Comparative benchmarks #19

New Issue

2026-09-30T11:08:03-04:00

connor commented

2026-09-30 11:08:03 -04:00

Measured comparison against nyx-space and Orekit with matched force models, published as numbers in the readme. Where we are slower, the reason is documented rather than left open.

M1.

Measured comparison against `nyx-space` and Orekit with matched force models, published as numbers in the readme. Where we are slower, the reason is documented rather than left open. M1.

connor added this to the M1 milestone 2026-09-30 11:24:58 -04:00

connor added the size/L label 2026-09-30 11:24:58 -04:00

connor commented

2026-10-03 13:13:22 -04:00

Done, against both references. Numbers are in the readme.

One day of arc, steps and milliseconds, minimum of thirty runs after warmup:

scenario	astrokit	nyx-space	Orekit
two-body	1752 / 0.49 ms	2151 / 7.06 ms	1932 / 4.60 ms
EGM96 8x8	1968 / 31.9	2158 / 147.5	1938 / 34.2
EGM96 20x20	2473 / 54.5	2158 / 193.6	1940 / 52.9
Sun and Moon	1794 / 35.9	2152 / 79.7	1932 / 36.1
radiation pressure	2067 / 28.0	2570 / 166.9	2035 / 133.6
drag, NRLMSISE-00	1859 / 52.0	2452 / 385.0	2092 / 357.2

Faster than both on every scenario. The step counts are printed alongside because the tolerances are not literally equal, and they span about a third across the three tools, so a ratio of times is close to a ratio of per-step cost.

What is not like for like, stated so the table is not over-read. Orekit's radiation pressure registers eclipse event detectors and root-finds the shadow boundaries, which neither of the other two do. The JVM is warmed ten times before anything is recorded; a cold run measures the bytecode interpreter and is out by an order of magnitude. The nyx timings carry a control that skips building the trajectory, which is within noise on every row.

nyx is slower than Orekit on every scenario, which was not expected of a Rust library against a JVM. Part of it is structural: nyx's integrated state is Const<90>, nine components plus an 81-element STM, carried on every step whether or not an STM was asked for, where this crate integrates six. That explains the two-body end of the table (14x) and not the gravity end (3.7x), which is still unaccounted for.

Where the time actually goes, measured on this side. Vern7 takes ten stages per accepted step, and benches/reference_scenarios.rs now benchmarks one derivative evaluation beside the propagation it explains:

the integrator's own fixed overhead is 0.24 us per step, read off two-body where it dominates
once any real force model is in the list it is under 2% of the step; the force models are 93-95%
at degree 8 the field itself is 153 ns of a 1505 ns evaluation, so the ANISE frame rotation costs nine times the physics, and still twice it at degree 20

That last one is the open performance question and is deliberately not acted on — hoisting the rotation is not valid as written, since the ten stages span up to 100 s and the Earth turns 0.4 degrees in that. Interpolating it across the step would be.

Reproduce with cargo bench --bench reference_scenarios, scripts/generate-orekit-fixtures.sh --time and scripts/generate-nyx-fixtures.sh --time.

Ready.

Done, against both references. Numbers are in the readme. One day of arc, steps and milliseconds, minimum of thirty runs after warmup: | scenario | astrokit | nyx-space | Orekit | |---|---|---|---| | two-body | 1752 / 0.49 ms | 2151 / 7.06 ms | 1932 / 4.60 ms | | EGM96 8x8 | 1968 / 31.9 | 2158 / 147.5 | 1938 / 34.2 | | EGM96 20x20 | 2473 / 54.5 | 2158 / 193.6 | 1940 / 52.9 | | Sun and Moon | 1794 / 35.9 | 2152 / 79.7 | 1932 / 36.1 | | radiation pressure | 2067 / 28.0 | 2570 / 166.9 | 2035 / 133.6 | | drag, NRLMSISE-00 | 1859 / 52.0 | 2452 / 385.0 | 2092 / 357.2 | Faster than both on every scenario. The step counts are printed alongside because the tolerances are not literally equal, and they span about a third across the three tools, so a ratio of times is close to a ratio of per-step cost. **What is not like for like, stated so the table is not over-read.** Orekit's radiation pressure registers eclipse event detectors and root-finds the shadow boundaries, which neither of the other two do. The JVM is warmed ten times before anything is recorded; a cold run measures the bytecode interpreter and is out by an order of magnitude. The nyx timings carry a control that skips building the trajectory, which is within noise on every row. **nyx is slower than Orekit on every scenario**, which was not expected of a Rust library against a JVM. Part of it is structural: nyx's integrated state is `Const<90>`, nine components plus an 81-element STM, carried on every step whether or not an STM was asked for, where this crate integrates six. That explains the two-body end of the table (14x) and not the gravity end (3.7x), which is still unaccounted for. **Where the time actually goes, measured on this side.** Vern7 takes ten stages per accepted step, and `benches/reference_scenarios.rs` now benchmarks one derivative evaluation beside the propagation it explains: - the integrator's own fixed overhead is **0.24 us per step**, read off two-body where it dominates - once any real force model is in the list it is **under 2%** of the step; the force models are 93-95% - at degree 8 the field itself is 153 ns of a 1505 ns evaluation, so **the ANISE frame rotation costs nine times the physics**, and still twice it at degree 20 That last one is the open performance question and is deliberately not acted on — hoisting the rotation is not valid as written, since the ten stages span up to 100 s and the Earth turns 0.4 degrees in that. Interpolating it across the step would be. Reproduce with `cargo bench --bench reference_scenarios`, `scripts/generate-orekit-fixtures.sh --time` and `scripts/generate-nyx-fixtures.sh --time`. Ready.

connor closed this issue

2026-10-03 13:14:07 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: connor/astrokit#19