connor/ordinary-diffeq
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Updated reports after bug fix

Browse Source
This commit is contained in:
connor committed 2026-09-29 18:21:40 -04:00
1 parent c20a090036
commit 2053aaa68e
3 files changed
+64 -62
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

VERN7_BENCHMARK_REPORT.md
+44 -40
View File
@@ -1,6 +1,6 @@
# Vern7 Performance Benchmark Report
**Date**: 2026-09-29 (re-measured; first run 2025-10-24)
**Date**: 2026-09-29
**Test System**: Linux 7.2.6-arch2-1
**Optimization Level**: Release build with full optimizations
@@ -8,10 +8,10 @@
Vern7 demonstrates **substantial performance advantages** over lower-order methods (BS3 and DP5) at tight tolerances (1e-8 to 1e-12), achieving:
- **2.9x faster** than DP5 at 1e-10 tolerance (exponential problem)
- **4.5x faster** than DP5 in harmonic oscillator
- **9.9x faster** than DP5 for orbital mechanics
- **4.4x faster** than DP5 in harmonic oscillator
- **1.1x faster** than DP5 for orbital mechanics
- **83x faster** than BS3 in harmonic oscillator
- **2.1x faster** than DP5 for interpolation workloads
- **2.3x faster** than DP5 for interpolation workloads
These results confirm Vern7's design goal: **maximum efficiency for high-accuracy requirements**.
@@ -23,9 +23,9 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Method | Time (μs) | Relative Speed | Speedup vs BS3 |
|--------|-----------|----------------|----------------|
| **Vern7** | **2.93** | **1.00x** (baseline) | **53.4x** |
| DP5 | 8.36 | 2.85x slower | 18.7x |
| BS3 | 156.63 | 53.4x slower | 1.0x |
| **Vern7** | **2.96** | **1.00x** (baseline) | **53.2x** |
| DP5 | 8.46 | 2.86x slower | 18.6x |
| BS3 | 157.35 | 53.2x slower | 1.0x |
**Analysis**:
- Vern7 is **2.9x faster** than DP5 and **53x faster** than BS3
@@ -41,15 +41,15 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Method | Time (μs) | Relative Speed | Speedup vs BS3 |
|--------|-----------|----------------|----------------|
| **Vern7** | **19.27** | **1.00x** (baseline) | **82.9x** |
| DP5 | 87.02 | 4.52x slower | 18.4x |
| BS3 | 1,597.5 | 82.9x slower | 1.0x |
| **Vern7** | **19.33** | **1.00x** (baseline) | **83.2x** |
| DP5 | 84.19 | 4.36x slower | 19.1x |
| BS3 | 1,608.7 | 83.2x slower | 1.0x |
**Analysis**:
- Vern7 is **4.5x faster** than DP5 and **83x faster** than BS3
- Vern7 is **4.4x faster** than DP5 and **83x faster** than BS3
- Smooth periodic problems like harmonic oscillators are ideal for high-order methods
- BS3 requires ~1.6ms due to tiny steps needed for tight tolerance
- DP5 needs ~87μs, still significantly more than Vern7's 19μs
- DP5 needs ~84μs, still significantly more than Vern7's 19μs
- Higher dimensionality (2D vs 1D) amplifies the advantage of larger steps
---
@@ -60,15 +60,18 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Method | Time (μs) | Relative Speed | Speedup |
|--------|-----------|----------------|---------|
| **Vern7** | **66.47** | **1.00x** (baseline) | **9.89x** |
| DP5 | 657.35 | 9.89x slower | 1.0x |
| **Vern7** | **83.88** | **1.00x** (baseline) | **1.13x** |
| DP5 | 94.68 | 1.13x slower | 1.0x |
**Analysis**:
- Vern7 is **9.9x faster** than DP5 for this challenging 6D problem
- Vern7 is **1.1x faster** than DP5 on this problem
- Step counts: Vern7 184, DP5 440. Vern7 takes 2.4x fewer steps, but each step
costs 16 stages against DP5's 7, so the advantage very nearly cancels
- Orbital mechanics requires tight tolerances to maintain energy conservation
- BS3 was too slow to include in the benchmark at this tolerance
- 6D problem with long integration time shows Vern7's scalability
- This represents realistic astrodynamics/orbital mechanics workloads
- Vern7's advantage here is in step count, which is what matters once the
right-hand side is expensive - a real gravity model rather than a two-body
point mass
---
@@ -78,11 +81,11 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Method | Time (μs) | Relative Speed | Notes |
|--------|-----------|----------------|-------|
| **Vern7** | **7.36** | **1.00x** (baseline) | Lazy extra stages |
| DP5 | 15.48 | 2.10x slower | Standard dense output |
| **Vern7** | **7.28** | **1.00x** (baseline) | Lazy extra stages |
| DP5 | 16.73 | 2.30x slower | Standard dense output |
**Analysis**:
- Vern7 with lazy computation is **2.1x faster** than DP5
- Vern7 with lazy computation is **2.3x faster** than DP5
- First interpolation triggers lazy computation of 6 extra stages (k11-k16)
- Subsequent interpolations reuse cached extra stages (~10ns RefCell overhead)
- Despite computing extra stages, Vern7 is still faster overall due to:
@@ -90,8 +93,8 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
2. Higher accuracy interpolation (7th order vs 5th order)
- Lazy computation adds minimal overhead, amortized over 100 interpolations
- Each benchmark iteration builds a fresh `Solution`, so the lazy stages are
computed once per iteration rather than served from the `RefCell` cache; this
is the honest cost of interpolating into a solution you have just produced
computed once per iteration rather than served from the `RefCell` cache - the
cost of interpolating into a solution you have just produced
---
@@ -103,11 +106,11 @@ These results confirm Vern7's design goal: **maximum efficiency for high-accurac
| Tolerance | DP5 (μs) | Vern7 (μs) | Speedup | Winner |
|-----------|----------|------------|---------|--------|
| 1e-6 | 1.95 | 1.57 | 1.24x | Vern7 |
| 1e-7 | 2.76 | 2.09 | 1.32x | Vern7 |
| 1e-8 | 4.16 | 2.26 | 1.84x | Vern7 |
| 1e-9 | 6.23 | 2.92 | 2.13x | **Vern7** |
| 1e-10 | 9.39 | 3.93 | 2.39x | **Vern7** |
| 1e-6 | 1.94 | 1.51 | 1.29x | Vern7 |
| 1e-7 | 2.82 | 1.93 | 1.46x | Vern7 |
| 1e-8 | 4.20 | 2.29 | 1.83x | Vern7 |
| 1e-9 | 6.25 | 2.97 | 2.11x | **Vern7** |
| 1e-10 | 9.52 | 3.89 | 2.45x | **Vern7** |
### Performance Scaling Chart (Conceptual)
@@ -130,8 +133,8 @@ Time (μs)
```
**Analysis**:
- At **moderate tolerances (1e-6)**: Vern7 is 1.2x faster
- At **tight tolerances (1e-10)**: Vern7 is 2.4x faster
- At **moderate tolerances (1e-6)**: Vern7 is 1.3x faster
- At **tight tolerances (1e-10)**: Vern7 is 2.5x faster
- **Crossover point**: Vern7 becomes increasingly advantageous as tolerance tightens
- DP5's time scales roughly quadratically with tolerance
- Vern7's time scales more slowly (higher order = larger steps)
@@ -160,22 +163,23 @@ Time (μs)
### Lazy Computation Impact
The lazy computation of extra stages (k11-k16) provides:
- **Minimal overhead**: ~6μs to compute 6 extra stages
- **Minimal overhead**: the 6 extra stages are computed once per step that is interpolated into
- **Cache efficiency**: Extra stages computed once per interval, reused for multiple interpolations
- **Memory efficiency**: Only computed when interpolation is requested
- **Performance**: Despite extra computation, still 1.65x faster than DP5 for interpolation workloads
- **Performance**: Despite extra computation, still 2.3x faster than DP5 for interpolation workloads
### Step Size Comparison
Estimated step sizes at 1e-10 tolerance for exponential problem:
Measured at 1e-10 tolerance on the exponential problem (`y' = y`, t = 0 to 4):
| Method | Avg Step Size | Steps Required | Function Evals |
|--------|---------------|----------------|----------------|
| BS3 | ~0.002 | ~2000 | ~8000 |
| DP5 | ~0.01 | ~400 | ~2400 |
| **Vern7** | ~0.05 | **~80** | **~800** |
| BS3 | 0.0014 | 2,916 | 11,664 |
| DP5 | 0.0354 | 113 | 791 |
| **Vern7** | 0.1379 | **29** | **290** |
**Vern7 requires ~3x fewer function evaluations than DP5.**
**Vern7 requires 2.7x fewer function evaluations than DP5**, taking 3.9x fewer
steps at 10 stages each against DP5's 7.
---
@@ -188,7 +192,7 @@ Our Rust implementation achieves performance comparable to Julia's highly-optimi
| Step computation | Highly optimized, FSAL | Optimized, no FSAL |
| Lazy interpolation | ✓ | ✓ |
| Stage caching | RefCell-based | RefCell-based (~10ns) |
| Memory allocation | Minimal | Minimal |
| Memory allocation | Cached in the integrator | Two heap allocations per step |
| Relative speed | Baseline | ~Comparable |
**Note**: Direct comparison difficult due to different hardware and problems, but algorithmic approach is identical.
@@ -216,8 +220,8 @@ Our Rust implementation achieves performance comparable to Julia's highly-optimi
Vern7 successfully achieves its design goal of being the **most efficient method for high-accuracy non-stiff problems**. The implementation with lazy computation of extra stages provides:
- ✅ **2-9x speedup** over DP5 at tight tolerances
- ✅ **50x+ speedup** over BS3 at tight tolerances
- ✅ **1.1-4.4x speedup** over DP5 at tight tolerances
- ✅ **53-83x speedup** over BS3 at tight tolerances
- ✅ **Efficient lazy interpolation** with minimal overhead
- ✅ **Full 7th-order accuracy** for both steps and interpolation
- ✅ **Memory-efficient caching** with RefCell
@@ -228,7 +232,7 @@ The results validate the effort invested in implementing the complex 16-stage in
## Appendix: Benchmark Configuration
**Hardware**: Not specified (Linux system)
**Hardware**: Linux 7.2.6-arch2-1
**Compiler**: rustc (release mode, full optimizations)
**Measurement Tool**: Criterion.rs v0.7.0
**Sample Size**: 100 samples per benchmark