ordinary-diffeq/roadmap/features/42-callback-effects.md

Feature: Callback Effects That Can Change the State

Status: completed (2026-10-01), extended with owned callbacks (2026-10-01)

Overview

Continuous callbacks located their crossings correctly and then threw away what the effect did about them. In Problem::solve:

callback.effect(&mut self.ode);   // writes to ode.y
}
}
self.ode.y = new_y;               // and here it goes
self.ode.t += curr_step;

The effect ran on the pre-step state, and the two lines after the callback loop overwrote ode.y with the state the integrator had produced. Anything the effect wrote to the state was lost. stop worked only because it writes t_end instead.

Reproduced with dy/dt = 1 from 0 to 10, y0 = 0, an event of 5 - y[0] and an effect of ode.y = Vector1::new(100.0): the solution ended at y = 10, exactly as it does with an empty effect, where it should end near 105.

So the machinery was in place for the easy half of event handling — detect a condition and stop — and could not do the other half: detect a condition, change the state, and carry on.

Why This Feature Matters

A rotating rigid body parameterized by Modified Rodrigues Parameters has a singularity at a full turn. It is avoided by swapping the parameter vector for the shadow set -sigma / |sigma|^2 — the same attitude, different parameters — whenever |sigma| exceeds 1. The event is 1 - |sigma|, the effect is that substitution, and the integration has to continue from the substituted state. Any arc longer than a half turn crosses it, so this is the common case rather than an edge case.

The same shape of problem covers a body entering or leaving shadow, a vehicle whose mass or area changes partway through a run, and a bouncing ball. All of them are a state discontinuity located by an event, which is the one thing the callbacks could not express.

What Was Done

The effect runs after the crossing is committed

The event branch now commits the shortened step before calling the effect:

1. the step is shortened to land on the crossing, and the state there is recorded;
2. ode.t and ode.y are set to that crossing;
3. the effect is called, with the ODE at the crossing;
4. whatever the effect leaves in ode.y is what the integration continues from.

Detection still reads the state at the start of the step — an event function is asked about both ends — so nothing can be committed until every callback has been asked. That is why this is a reordering of the branch rather than a swap of two lines.

effect(&mut self)

Callback::effect takes &mut self, so a callback can record that it has fired, count its firings, or keep anything else it needs across them. Problem holds Vec<&'a mut dyn Callback<T, D, P>> and with_callback takes &'a mut dyn Callback, which is the breaking part of the change: a registered callback has to be declared mut and passed as &mut.

FnCallback still holds its two closures by shared reference and so still has no state of its own; a callback that needs state implements Callback on a struct, which is what Discrete Callbacks will do with its has_fired field.

Problem lost its #[derive(Clone)], which a &mut field cannot support. It was never used.

A terminal effect ends the solution at the crossing

ODE::finished existed, was never set and was never read. It now means what its name says: ODE::terminate sets it, the solve loop checks it, and stop is ode.terminate().

Behaviorally this is close to what stop already did. Writing t_end = ode.t also ended the loop, and because the step had already been shortened to the crossing, the last recorded point was the crossing — the solution did not in fact run a step past the event. What the old stop did do was mutate the problem's own end time, so a solution could no longer report the t_end it was asked for. Now t_end is left alone and finished says why the solution is short.

Both sides of a jump are recorded

Where an effect changes the state, the trajectory genuinely has two values at that time, and the solution holds both: the same t appears twice in times, with the state at the crossing and the state the effect left. The dense entry between them spans zero time and is never interpolated into — interpolate at exactly that time hits the search's exact-match branch, and walks to the last entry at that time, so it reports the state the effect left. That is the value the trajectory carries forward, and it is the one a caller asking "where is the body at this instant" wants; the value it arrived with is the entry before it in states.

This is what DifferentialEquations.jl does, where the two are u_left and u_right.

The controller is reset across a jump

Controller grew a required reset method, which discards the error history and leaves the next step guess alone. solve calls it when an effect changes the state. The history is a claim about how the error behaves along a smooth solution, and across a discontinuity it is not about anything.

Keyed on the state changing rather than on the effect running at all, so that a callback which only observes does not throw away a good history every time it fires. An effect that changes only the parameters is a discontinuity the solver cannot see, and does not reset the controller.

PIDController::reset already existed as an inherent method and became the trait implementation; PIController::reset restores factor_old to its initial value.

Two events in one step

Defined and tested: the earliest crossing in the step fires, the step ends there, and any later event is still ahead and is found again on the following step. A tie goes to registration order.

The old behavior was to walk the callbacks in registration order, each shortening what the last one left. That could apply a later event's effect and then truncate the step to before it, so the effect was applied at a time the solution never reached. One pass to find the earliest crossing costs an extra root find only when more than one callback crosses in the same step.

Crossings at exactly zero, and not firing twice at one

The test was before * after < 0.0, which misses a crossing that starts or ends exactly on zero. Now:

• before == 0.0 is a crossing at the step's own start, located at h = 0;
• after == 0.0 is a crossing at the step's end;
• opposite signs are located with Brent as before.

The first of those needs a guard, because an effect is free to leave its event function sitting on the zero it fired at. The shadow set swap does exactly that: |sigma| is 1 on both sides of it, so 1 - |sigma| is still zero after the effect. Without a guard the callback is re-detected at the start of the next step, located at h = 0, fired again, forever.

The guard is that the step beginning at a firing searches from EVENT_PROBE_FRACTION of itself — a millionth — rather than from its start, for that callback alone. Both the before sign and Brent's bracket start there, so the event just handled is out of reach while a second crossing later in the same step is still found. solve tracks the time each callback last fired to know when to do this.

The first attempt at this guard was to skip a callback for the whole step that begins at its firing, which is simpler and wrong: a bouncing ball bounced once and then fell through the floor, because the step after the bounce contained the next landing and the callback was not looking. That is in the test suite now.

A fraction of a step is not a satisfying thing to have to choose, and this is the least bad option found rather than a good one. Alternatives considered: an absolute tolerance on the event value, which is unit-dependent and no better founded; taking the sign from the step before the firing, which still leaves Brent bracketing a root at h = 0; and refusing a located crossing below some h, which is the same arbitrary fraction one step further down. If something cleaner turns up, this is the place to change it.

Follow-up: a problem can own a callback (0.7.0)

Problem held Vec<&'a mut dyn Callback>, so something else had to own each callback and outlive the problem. That is fine from a function body and impossible from a type that stores the problem — and a wrapper must store it, because solve(&mut self) -> Solution<'_, ..> hands back a solution that borrows from it, so the problem has to live somewhere that outlives the solution, and a field is the only such place. Such a wrapper could not then register a callback of its own:

struct Wrapper<'a, ..> {
    my_callback: MyCallback,           // owned here
    problem: Option<Problem<'a, ..>>,  // and borrowed from, here
}

That is self-referential, and moving the wrapper would dangle the reference, so the compiler is right to reject it. The workaround in the dependent crate was a zero-sized callback, leaked to get a &'static mut.

with_owned_callback(Box<dyn Callback<T, D, P> + 'o>) is the addition. Storage is one list of either kind:

enum Registered<'a, 'o, T: Scalar, const D: usize, P> {
    Borrowed(&'a mut dyn Callback<T, D, P>),
    Owned(Box<dyn Callback<T, D, P> + 'o>),
}

Registered implements Callback itself, forwarding both methods, which is why none of the event handling in solve changed: the immutable scan for the earliest crossing goes through event and the single effect that fires goes through effect, whichever kind is holding it. Root finding, jump recording, the controller reset and EVENT_PROBE_FRACTION are untouched.

The second lifetime, which was not in the plan

Problem gained a lifetime parameter: Problem<'a, 'o, T, D, S, P, C>. This was meant to be purely additive and it cannot be, for a reason worth writing down.

Dropping a Box<dyn Callback> runs a destructor the compiler cannot see into, so dropck requires everything that box's lifetime covers to be alive when the problem is dropped. With one lifetime for both kinds, that requirement lands on the borrowed callbacks too — and it is type-based, so it applies even to a problem that owns no callback at all. The first attempt, a single 'a, failed exactly here:

error[E0502]: cannot borrow `once.fired` as immutable because it is also borrowed as mutable
  |
  | mutable borrow might be used here, when `problem` is dropped

That is test_callback_can_remember_firing, reading a borrowed callback's own state after the solve, which is the only way a caller learns what a stateful callback did and the whole reason the borrowed form exists. Keeping it means the owned box needs a lifetime of its own, so that only 'o has to outlive the drop and NLL can still end a borrowed callback's borrow at the problem's last use.

The cost is arity: code that names the type explicitly needs Problem<'a, '_, ..>. Both lifetimes are inferred, so nothing that only calls Problem::new(..).with_callback(..).solve() is affected.

Not done: getting owned callbacks back

Deliberately skipped. Downcasting a Box<dyn Callback> to its concrete type needs Any, which means a 'static bound and an as_any method on every implementor — a cost paid by all callbacks for a need that has two cheaper answers already: stay borrowed, or share an Rc<Cell<_>>, which can be read while the solution is still alive.

Measurement

The benches register no callbacks, so what they exercise is the list's element type changing from a reference to a two-variant enum, on a list that is empty. Measured by alternating old and new rather than running one after the other, because the first attempt at it reported a 4% improvement, which nothing in this change could produce: between-sitting drift on this machine is larger than the effect being looked for.

bench	old	new
bench_orbit	356.06, 358.91 us	357.98, 355.18 us
bench_simple_1d	877.7, 881.1 ns	877.4, 895.7 ns
bench_interpolation_1d	2.2683, 2.2104 us	2.2444, 2.2496 us

Two runs of the same binary differ by up to 2.6% here, and the old and new ranges overlap on all three. No measurable change.

Testing

• an owned callback producing the same times and states as the identical borrowed one, element for element;
• an owned stateful callback escalating on its second firing, read off the trajectory since its state cannot be read directly;
• owned and borrowed callbacks on one problem with their events at the same time, where the final state distinguishes registration order, and the borrowed one's count is read after the solve;
• a boxed FnCallback, which holds two closure references, confirming the + 'o rather than 'static;
• the wrapper from the top of this section, compiling and solving.

Known Limitations

• An even number of crossings inside one step is still missed: the two ends of the step agree in sign and nothing looks in between. Documented on Problem::solve.
• A crossing inside the first millionth of the step that begins at a firing is missed, which is the cost of the no-refiring guard. It takes an event that recurs almost immediately after one was handled.
• The crossing time is located in real arithmetic and carries no derivative. Deliberate: differentiating through Brent's iterations gives the derivative of the iterative procedure rather than the sensitivity of the event, which is wrong without being loud about it. The state at the crossing comes from one ordinary step of the integrator, so it does carry derivatives.
• An effect that changes the parameters rather than the state does not reset the controller, and does not record two points.

Testing

In src/problem.rs:

• the reproduction above, ending near 105 rather than 10;
• both sides of the jump present in times/states, interpolation either side of it following the branch it is on, and interpolation at the jump itself giving the state the effect left;
• stop ending the solution at the crossing, with nothing past it, t_end untouched and finished set;
• a callback counting its own firings and acting only on the first, which needs the &mut self;
• two events in one step firing earliest-first, with the registration order reversed to show it does not matter;
• an effect that lands back on its own event firing once there rather than looping, and firing again when the event genuinely recurs;
• a bouncing ball, bouncing at the three analytic bounce times inside the interval and interpolating correctly inside each arc;
• an event function already zero at the start of the integration firing there;
• the controller reset asked for once across a jump and not at all for an effect that changes nothing, measured with a controller that counts the resets;
• a jump under dual numbers, where the derivative through the effect is exact (in src/lib.rs, with the autodiff tests);
• the Modified Rodrigues Parameter swap itself, against the closed form for a constant angular velocity: one swap, at exactly half a turn, |sigma| <= 1 throughout, and the final attitude equal to -cot(theta / 4) about the rotation axis.

Success Criteria

• An effect that writes to ode.y changes what is integrated from. ✅
• An effect can record something about itself. ✅
• A terminated solution ends at the crossing. ✅
• A jump is recorded on both sides. ✅
• The step size controller forgets its history across a jump. ✅
• The order of two events in one step is defined and tested. ✅
• No measurable change on the criterion benches. ✅ Measured before and after on the same machine in one sitting: bench_orbit -1.02%, bench_simple_1d +0.36%, bench_interpolation_1d -0.67%, all three within criterion's noise threshold. Which is the right reading of them: re-running the identical binary moved bench_simple_1d by -0.80% and bench_interpolation_1d by +1.47%, so run-to-run drift here is about a percent either way and is larger than the difference being looked for. No bench registers a callback, so what they measure is the no-callback path, which gains a loop over an empty slice and a finished check per step.

Effort: Medium