connor/ordinary-diffeq
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Updated reports after bug fix

Browse Source
This commit is contained in:
connor committed 2026-09-29 18:21:40 -04:00
1 parent c20a090036
commit 2053aaa68e
3 files changed
+64 -62
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

VERN7_BENCHMARK_REPORT.md
+44 -40
View File
@@ -1,6 +1,6 @@
# Vern7 Performance Benchmark Report
**Date**: 2026-09-29 (re-measured; first run 2025-10-24)
**Date**: 2026-09-29
**Test System**: Linux 7.2.6-arch2-1
**Optimization Level**: Release build with full optimizations
@@ -8,10 +8,10 @@
Vern7 demonstrates **substantial performance advantages** over lower-order methods (BS3 and DP5) at tight tolerances (1e-8 to 1e-12), achieving:
- **2.9x faster** than DP5 at 1e-10 tolerance (exponential problem)
- **4.5x faster** than DP5 in harmonic oscillator
- **9.9x faster** than DP5 for orbital mechanics
- **4.4x faster** than DP5 in harmonic oscillator
- **1.1x faster** than DP5 for orbital mechanics
- **83x faster** than BS3 in harmonic oscillator
- **2.1x faster** than DP5 for interpolation workloads
- **2.3x faster** than DP5 for interpolation workloads
These results confirm Vern7's design goal: **maximum efficiency for high-accuracy requirements**.
@@ -23,9 +23,9 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Method | Time (μs) | Relative Speed | Speedup vs BS3 |
|--------|-----------|----------------|----------------|
| **Vern7** | **2.93** | **1.00x** (baseline) | **53.4x** |
| DP5 | 8.36 | 2.85x slower | 18.7x |
| BS3 | 156.63 | 53.4x slower | 1.0x |
| **Vern7** | **2.96** | **1.00x** (baseline) | **53.2x** |
| DP5 | 8.46 | 2.86x slower | 18.6x |
| BS3 | 157.35 | 53.2x slower | 1.0x |
**Analysis**:
- Vern7 is **2.9x faster** than DP5 and **53x faster** than BS3
@@ -41,15 +41,15 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Method | Time (μs) | Relative Speed | Speedup vs BS3 |
|--------|-----------|----------------|----------------|
| **Vern7** | **19.27** | **1.00x** (baseline) | **82.9x** |
| DP5 | 87.02 | 4.52x slower | 18.4x |
| BS3 | 1,597.5 | 82.9x slower | 1.0x |
| **Vern7** | **19.33** | **1.00x** (baseline) | **83.2x** |
| DP5 | 84.19 | 4.36x slower | 19.1x |
| BS3 | 1,608.7 | 83.2x slower | 1.0x |
**Analysis**:
- Vern7 is **4.5x faster** than DP5 and **83x faster** than BS3
- Vern7 is **4.4x faster** than DP5 and **83x faster** than BS3
- Smooth periodic problems like harmonic oscillators are ideal for high-order methods
- BS3 requires ~1.6ms due to tiny steps needed for tight tolerance
- DP5 needs ~87μs, still significantly more than Vern7's 19μs
- DP5 needs ~84μs, still significantly more than Vern7's 19μs
- Higher dimensionality (2D vs 1D) amplifies the advantage of larger steps
---
@@ -60,15 +60,18 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Method | Time (μs) | Relative Speed | Speedup |
|--------|-----------|----------------|---------|
| **Vern7** | **66.47** | **1.00x** (baseline) | **9.89x** |
| DP5 | 657.35 | 9.89x slower | 1.0x |
| **Vern7** | **83.88** | **1.00x** (baseline) | **1.13x** |
| DP5 | 94.68 | 1.13x slower | 1.0x |
**Analysis**:
- Vern7 is **9.9x faster** than DP5 for this challenging 6D problem
- Vern7 is **1.1x faster** than DP5 on this problem
- Step counts: Vern7 184, DP5 440. Vern7 takes 2.4x fewer steps, but each step
costs 16 stages against DP5's 7, so the advantage very nearly cancels
- Orbital mechanics requires tight tolerances to maintain energy conservation
- BS3 was too slow to include in the benchmark at this tolerance
- 6D problem with long integration time shows Vern7's scalability
- This represents realistic astrodynamics/orbital mechanics workloads
- Vern7's advantage here is in step count, which is what matters once the
right-hand side is expensive - a real gravity model rather than a two-body
point mass
---
@@ -78,11 +81,11 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Method | Time (μs) | Relative Speed | Notes |
|--------|-----------|----------------|-------|
| **Vern7** | **7.36** | **1.00x** (baseline) | Lazy extra stages |
| DP5 | 15.48 | 2.10x slower | Standard dense output |
| **Vern7** | **7.28** | **1.00x** (baseline) | Lazy extra stages |
| DP5 | 16.73 | 2.30x slower | Standard dense output |
**Analysis**:
- Vern7 with lazy computation is **2.1x faster** than DP5
- Vern7 with lazy computation is **2.3x faster** than DP5
- First interpolation triggers lazy computation of 6 extra stages (k11-k16)
- Subsequent interpolations reuse cached extra stages (~10ns RefCell overhead)
- Despite computing extra stages, Vern7 is still faster overall due to:
@@ -90,8 +93,8 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
2. Higher accuracy interpolation (7th order vs 5th order)
- Lazy computation adds minimal overhead, amortized over 100 interpolations
- Each benchmark iteration builds a fresh `Solution`, so the lazy stages are
computed once per iteration rather than served from the `RefCell` cache; this
is the honest cost of interpolating into a solution you have just produced
computed once per iteration rather than served from the `RefCell` cache - the
cost of interpolating into a solution you have just produced
---
@@ -103,11 +106,11 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Tolerance | DP5 (μs) | Vern7 (μs) | Speedup | Winner |
|-----------|----------|------------|---------|--------|
| 1e-6 | 1.95 | 1.57 | 1.24x | Vern7 |
| 1e-7 | 2.76 | 2.09 | 1.32x | Vern7 |
| 1e-8 | 4.16 | 2.26 | 1.84x | Vern7 |
| 1e-9 | 6.23 | 2.92 | 2.13x | **Vern7** |
| 1e-10 | 9.39 | 3.93 | 2.39x | **Vern7** |
| 1e-6 | 1.94 | 1.51 | 1.29x | Vern7 |
| 1e-7 | 2.82 | 1.93 | 1.46x | Vern7 |
| 1e-8 | 4.20 | 2.29 | 1.83x | Vern7 |
| 1e-9 | 6.25 | 2.97 | 2.11x | **Vern7** |
| 1e-10 | 9.52 | 3.89 | 2.45x | **Vern7** |
### Performance Scaling Chart (Conceptual)
@@ -130,8 +133,8 @@ Time (μs)
```
**Analysis**:
- At **moderate tolerances (1e-6)**: Vern7 is 1.2x faster
- At **tight tolerances (1e-10)**: Vern7 is 2.4x faster
- At **moderate tolerances (1e-6)**: Vern7 is 1.3x faster
- At **tight tolerances (1e-10)**: Vern7 is 2.5x faster
- **Crossover point**: Vern7 becomes increasingly advantageous as tolerance tightens
- DP5's time scales roughly quadratically with tolerance
- Vern7's time scales more slowly (higher order = larger steps)
@@ -160,22 +163,23 @@ Time (μs)
### Lazy Computation Impact
The lazy computation of extra stages (k11-k16) provides:
- **Minimal overhead**: ~6μs to compute 6 extra stages
- **Minimal overhead**: the 6 extra stages are computed once per step that is interpolated into
- **Cache efficiency**: Extra stages computed once per interval, reused for multiple interpolations
- **Memory efficiency**: Only computed when interpolation is requested
- **Performance**: Despite extra computation, still 1.65x faster than DP5 for interpolation workloads
- **Performance**: Despite extra computation, still 2.3x faster than DP5 for interpolation workloads
### Step Size Comparison
Estimated step sizes at 1e-10 tolerance for exponential problem:
Measured at 1e-10 tolerance on the exponential problem (`y' = y`, t = 0 to 4):
| Method | Avg Step Size | Steps Required | Function Evals |
|--------|---------------|----------------|----------------|
| BS3 | ~0.002 | ~2000 | ~8000 |
| DP5 | ~0.01 | ~400 | ~2400 |
| **Vern7** | ~0.05 | **~80** | **~800** |
| BS3 | 0.0014 | 2,916 | 11,664 |
| DP5 | 0.0354 | 113 | 791 |
| **Vern7** | 0.1379 | **29** | **290** |
**Vern7 requires ~3x fewer function evaluations than DP5.**
**Vern7 requires 2.7x fewer function evaluations than DP5**, taking 3.9x fewer
steps at 10 stages each against DP5's 7.
---
@@ -188,7 +192,7 @@ Our Rust implementation achieves performance comparable to Julia's highly-optimi
| Step computation | Highly optimized, FSAL | Optimized, no FSAL |
| Lazy interpolation | ✓ | ✓ |
| Stage caching | RefCell-based | RefCell-based (~10ns) |
| Memory allocation | Minimal | Minimal |
| Memory allocation | Cached in the integrator | Two heap allocations per step |
| Relative speed | Baseline | ~Comparable |
**Note**: Direct comparison difficult due to different hardware and problems, but algorithmic approach is identical.
@@ -216,8 +220,8 @@ Our Rust implementation achieves performance comparable to Julia's highly-optimi
Vern7 successfully achieves its design goal of being the **most efficient method for high-accuracy non-stiff problems**. The implementation with lazy computation of extra stages provides:
- ✅ **2-9x speedup** over DP5 at tight tolerances
- ✅ **50x+ speedup** over BS3 at tight tolerances
- ✅ **1.1-4.4x speedup** over DP5 at tight tolerances
- ✅ **53-83x speedup** over BS3 at tight tolerances
- ✅ **Efficient lazy interpolation** with minimal overhead
- ✅ **Full 7th-order accuracy** for both steps and interpolation
- ✅ **Memory-efficient caching** with RefCell
@@ -228,7 +232,7 @@ The results validate the effort invested in implementing the complex 16-stage in
## Appendix: Benchmark Configuration
**Hardware**: Not specified (Linux system)
**Hardware**: Linux 7.2.6-arch2-1
**Compiler**: rustc (release mode, full optimizations)
**Measurement Tool**: Criterion.rs v0.7.0
**Sample Size**: 100 samples per benchmark
benches/BS3_VS_DP5_RESULTS.md
+19 -21
View File
@@ -1,6 +1,6 @@
# BS3 vs DP5 Benchmark Results
Generated: 2026-09-29 (re-measured; first run 2025-10-23)
Generated: 2026-09-29
## Summary
@@ -10,7 +10,7 @@ Comprehensive performance comparison between **BS3** (Bogacki-Shampine 3rd order
### Overall Performance Comparison
**DP5 is consistently faster than BS3 across all tested scenarios**, typically by a factor of **1.6x to 4.8x**.
**DP5 is consistently faster than BS3 across all tested scenarios**, typically by a factor of **1.2x to 5.1x**.
This might seem counterintuitive since BS3 uses fewer stages (4 vs 7), but several factors explain DP5's superior performance:
@@ -32,8 +32,8 @@ Despite being slower in these benchmarks, BS3 still has value:
| Method | Time | Ratio |
|--------|------|-------|
| **BS3** | 2.96 µs | 2.08x slower |
| **DP5** | 1.43 µs | baseline |
| **BS3** | 2.99 µs | 2.21x slower |
| **DP5** | 1.35 µs | baseline |
Simple 1D problem with smooth exponential solution.
@@ -41,8 +41,8 @@ Simple 1D problem with smooth exponential solution.
| Method | Time | Ratio |
|--------|------|-------|
| **BS3** | 27.45 µs | 2.26x slower |
| **DP5** | 12.16 µs | baseline |
| **BS3** | 27.37 µs | 2.76x slower |
| **DP5** | 9.92 µs | baseline |
2D conservative system with periodic solution.
@@ -50,8 +50,8 @@ Simple 1D problem with smooth exponential solution.
| Method | Time | Ratio |
|--------|------|-------|
| **BS3** | 113.66 µs | 3.65x slower |
| **DP5** | 31.15 µs | baseline |
| **BS3** | 113.44 µs | 3.81x slower |
| **DP5** | 29.74 µs | baseline |
Nonlinear 2D system with trigonometric terms.
@@ -59,8 +59,8 @@ Nonlinear 2D system with trigonometric terms.
| Method | Time | Ratio |
|--------|------|-------|
| **BS3** | 102.04 µs | 1.56x slower |
| **DP5** | 65.52 µs | baseline |
| **BS3** | 98.02 µs | 5.07x slower |
| **DP5** | 19.32 µs | baseline |
Higher-dimensional problem with gravitational dynamics.
@@ -68,8 +68,8 @@ Higher-dimensional problem with gravitational dynamics.
| Method | Time (solve + 100 interpolations) | Ratio |
|--------|-----------------------------------|-------|
| **BS3** | 16.28 µs | 4.70x slower |
| **DP5** | 3.46 µs | baseline |
| **BS3** | 16.34 µs | 4.67x slower |
| **DP5** | 3.50 µs | baseline |
BS3 uses cubic Hermite interpolation, DP5 uses optimized 5th order interpolation.
@@ -79,14 +79,11 @@ Performance across different tolerance levels (`y' = -y` problem):
| Tolerance | BS3 Time | DP5 Time | Ratio (BS3/DP5) |
|-----------|----------|----------|-----------------|
| 1e-3 | 1.15 µs | 0.92 µs | 1.25x |
| 1e-4 | 2.01 µs | 1.19 µs | 1.69x |
| 1e-5 | 3.41 µs | 1.45 µs | 2.36x |
| 1e-6 | 6.72 µs | 1.92 µs | 3.50x |
| 1e-7 | 13.55 µs | 2.82 µs | 4.80x |
Both 1e-6 and 1e-7 are now measured. The earlier run estimated 1e-6 from the
trend and did not complete 1e-7.
| 1e-3 | 1.10 µs | 0.91 µs | 1.21x |
| 1e-4 | 1.87 µs | 1.19 µs | 1.57x |
| 1e-5 | 3.46 µs | 1.44 µs | 2.40x |
| 1e-6 | 6.77 µs | 1.95 µs | 3.46x |
| 1e-7 | 13.73 µs | 2.79 µs | 4.91x |
**Observation**: The performance gap widens as tolerance tightens, because DP5's higher order allows it to take larger steps while maintaining accuracy.
@@ -96,7 +93,8 @@ trend and did not complete 1e-7.
1. **DP5 is the better default choice** for most problems requiring moderate to high accuracy
2. **Performance gap increases** with tighter tolerances (favoring DP5)
3. **Higher dimensions** slightly favor BS3 relative to DP5 (1.56x vs 3.65x slowdown)
3. **Higher dimensions favor DP5 most strongly** (5.07x on the 6D orbit against
2.21x on 1D exponential decay)
4. **Interpolation** strongly favors DP5 (4.7x faster)
### Implementation Quality
readme.md
+1 -1
View File
@@ -15,7 +15,7 @@ and field line tracing:
| **Vern7** (Verner) | 7(6) | 10+6 | 7th order | High accuracy (rtol ~ 1e-8 to 1e-12) |
**Performance at 1e-10 tolerance:**
- Vern7: **2.9-9.9x faster** than DP5
- Vern7: **1.1-4.4x faster** than DP5
- Vern7: **53-83x faster** than BS3
See [benchmark report](VERN7_BENCHMARK_REPORT.md) for detailed performance analysis.