# resolved

Renders what a spacecraft camera sees: Earth, Moon, Sun and a target
spacecraft loaded from a mesh. The inputs are an epoch, the observer and target
states and attitudes, and a camera model. Images are radiometric: the renderer
works in radiance and converts to electrons and then DN through the camera's
optics and detector, so an image should match what a real camera with the same
parameters records.

It is built for closed-loop simulation, where each step renders one frame and
latency is the number that matters. The goal is high-fidelity 1–2 MP RGB in
about 10 ms end to end on a mid-range GPU (the reference is an RTX 3060);
100 ms is the ceiling, not the aim. Lighting fidelity and camera fidelity are
selected separately, and each tier is benchmarked, so a run can trade
fidelity for speed knowing what each step costs.

Rendering goes through wgpu. On a machine without a GPU, the same shaders run
on the CPU through Mesa's lavapipe (`vulkan-swrast` on Arch,
`mesa-vulkan-drivers` on Debian and Ubuntu).

## Performance and fidelity

Lighting and camera fidelity are chosen separately: lighting tiers L0 to L3,
camera tiers C0 to C3, set with `RendererOptions::fidelity` in the library and
`--lighting` and `--camera` on the command line. A tier is a fixed bundle of
features, and a tier that needs data (L1 needs an image of Earth's surface)
refuses to start without it. A row is added here as each tier is complete, with
its measured cost and a sample image from the same scene, so the cost of each
step up is visible at a glance.

| Tier | Lighting | Camera | RTX 3060 | lavapipe (CPU) | Sample |
|---|---|---|---|---|---|
| L0 / C0 | Lambertian surfaces, Earth's and the Moon's limbs exact, eclipse by Earth and the Moon, no shadows on the target, no light from Earth | Pinhole, absolute radiometry to 12-bit DN, no noise or blur | 1.8–2.1 ms | 8–65 ms | [![L0/C0](docs/images/l0-c0.png)](docs/images/l0-c0.png) |

Times are one 1920 x 1080 RGB frame end to end (`submit` to `wait`, readback
of the DN image included), median of criterion runs, over models from 12 to
752,000 triangles: the box-wing, Landsat 8 and ICESat-2. The GPU's own work is
0.3 to 0.6 ms of that; reading 12 MB back and copying it out is most of the
rest. lavapipe ran on the Intel Core Ultra 9 285K's 24 cores. The run-to-run
noise is about 3 %. The sample is the Landsat 8 example scene
(`examples/scenes/landsat8.toml`). `cargo bench -p resolved` reproduces the
numbers, and `resolved render --timings N` shows the per-pass split.

Planned tiers: L1 adds self-shadowing, Earthshine and albedo maps (M2); C1
and C2 add lens distortion, blur and detector noise (M3); L2 adds measured
materials and the atmosphere (M4); L3 is path traced (M5).

## Usage

```rust
use resolved::nalgebra::{UnitQuaternion, Vector3};
use resolved::radiometry::AU;
use resolved::{CameraConfig, Mesh, Pose, Renderer, Scene, Target};

# fn main() -> Result<(), resolved::Error> {
// Setup, once: device, shaders, spectra folded into the camera's channels.
let mut renderer = Renderer::new(CameraConfig::example(), Default::default())?;

// A model, uploaded once. Usually `Mesh::load(path, &options)` on an STL or glTF.
let stl = "solid panel
facet normal 0 0 0
outer loop
vertex 0 -1 -1
vertex 0 1 -1
vertex 0 0 1
endloop
endfacet
endsolid panel";
let panel = renderer.load_model(&Mesh::from_stl(&mut std::io::Cursor::new(stl), "panel", &Default::default())?)?;

// Each step: inertial positions in meters, Earth at the origin.
let target = Vector3::new(7.0e6, 0.0, 0.0);
let scene = Scene {
    earth_attitude: UnitQuaternion::identity(),
    sun: Vector3::new(AU, 0.0, 0.0),
    moon: Vector3::new(0.0, 3.844e8, 0.0),
    targets: vec![Target { model: panel, pose: Pose::new(target, UnitQuaternion::identity()) }],
};
let observer = Pose::looking_at(target + Vector3::new(20.0, 0.0, 0.0), target, Vector3::z()).unwrap();

let pending = renderer.submit(&scene, &observer)?;
// ... the simulation's own work for this step overlaps the GPU's here ...
let frame = pending.wait()?;

// 12-bit DN, channels r, g, b interleaved. The panel faces the Sun and the camera.
assert!(frame.dn_at(960, 540, 1) > 1000);
# Ok(())
# }
```

## Command line

`resolved render` renders a scene described in TOML: camera overrides, the
observer, Sun and Moon positions (or an epoch, with the `astrokit` feature) and
targets with their models. `examples/scenes/` has commented examples:
`boxwing.toml` and `boxwing-epoch.toml` use a small generated model that is
in the repository, and the rest use NASA spacecraft models (Landsat 8,
ICESat-2, OSTM/Jason-2, an SSL-1300 and ACE) that `scripts/fetch-models.sh`
downloads into `examples/models/nasa/`. That script needs Node (for `npx`),
because most NASA models are Draco-compressed and are decoded on the way in.
At lighting L1, Earth's surface comes from NASA's Blue Marble, which
`scripts/fetch-earth.sh` downloads and resizes (with ImageMagick) into
`data/earth/`, one image per month. The month comes from the scene's epoch, or
from `[appearance] earth_month` in a scene without one.

```sh
cargo run --release -p resolved-cli -- render \
    --scene examples/scenes/boxwing.toml \
    --out frame.png --preview preview.png --timings 50
```

- `--out` writes raw DN as a 16-bit PNG, shifted up to fill 16 bits as
  cameras usually store them: a 12-bit DN is the stored value divided by 16.
  It is linear, so it looks dark next to the preview.
- `--preview` writes an 8-bit sRGB image for looking at, with the bias as
  black and saturation as white.
- `--timings N` renders N frames and reports setup and per-frame times.
- `--lighting L0|L1` and `--camera C0` pick the fidelity tiers (default L0
  and C0). A tier the scene or the data cannot support stops with an error
  naming what is missing.
- `--data DIR` says where the tiers' data is (`earth/bmng-2004-MM.jpg`).
  Without it the CLI tries `RESOLVED_DATA`, then `./data`, then the
  repository's `data/`.
- `--adapter` takes `auto`, `gpu`, `cpu` or part of an adapter's name;
  `resolved adapters` lists them.
- `--kernels DIR` says where astrokit's kernels are, for scenes given by
  epoch. Without it the CLI tries `ASTROKIT_KERNELS`, then `data/kernels`,
  then the sibling `../astrokit` checkout it was built against.

## Features

- `astrokit`: build a `Scene` from an epoch with
  [astrokit](https://git.rcjohnstone.com/connor/astrokit)'s ephemerides (JPL
  DE440s for the Sun and Moon, ITRF93 for Earth's orientation), through
  `ephemeris::Ephemeris`. astrokit is not on crates.io, so this expects a
  checkout at `../astrokit` next to this repository, and its kernels from its
  `scripts/fetch-kernels.sh` (or `ASTROKIT_KERNELS` pointing at them).

## Status

M1 in progress: the library renders Earth, Moon, Sun and targets to DN
through `Renderer`, and the CLI renders scene files; benchmarks are still to
come. The work is
tracked as Gitea milestones, each split into one issue per change:

| Milestone | Scope |
|---|---|
| M1 | Scene API, headless wgpu, analytic Earth/Moon/Sun, glTF and STL targets, Lambert shading with eclipse, pinhole camera, radiance to DN, CLI, benchmark baseline |
| M2 | Self-shadowing, Earthshine and Moonshine, Earth and Moon albedo maps, Sun limb darkening |
| M3 | Lens distortion, PSF, detector noise, saturation and blooming, stars |
| M4 | BRDF and spectral materials, atmospheric scattering, clouds, lunar terrain |
| M5 | Portable ray tracing, path tracing, label outputs for ML |
| M6 | Validation against lunar, Earth and Mitsuba references; motion blur, rolling shutter, stray light; multi-band and 4k |

## Conventions

- SI units throughout: meters, seconds, radians, watts.
- Scene state is f64 in an inertial frame (GCRF). The GPU only sees f32
  coordinates relative to the camera.

## Layout

- `crates/resolved`: the library.
- `crates/resolved-cli`: the `resolved` command, for rendering without writing
  a host program.
- `examples/`: scene files and the box-wing model
  (`scripts/make-example-model.py` regenerates it). The NASA scenes are
  generated by `scripts/make-example-scenes.py`.
- `scripts/`: generators for compiled-in data and examples.
- `docs/optics.md`: the optics and radiometry the renderer models, explained
  for readers without an optics background.

## Development

Point git at the committed hooks once per clone:

```sh
git config core.hooksPath hooks
```

`pre-commit` runs formatting, clippy and the library unit tests; `pre-push`
runs the full test suite, including doctests, and the documentation build.

The integration tests render on both a hardware GPU and lavapipe, so running
them needs a Vulkan driver for the GPU and lavapipe installed. Set
`WGPU_BACKEND`, `WGPU_ADAPTER_NAME` and the other standard wgpu environment
variables to steer adapter discovery.

Benchmarks are end-to-end frame times (`submit` then `wait`, readback
included) on the GPU and on lavapipe, at two resolutions, for the box-wing and,
once fetched, Landsat 8 and ICESat-2:

```sh
cargo bench -p resolved
```

For where a frame's time goes, `resolved render --timings N` adds GPU
timestamps around each pass (`RendererOptions::profile` in the library).

To see what the GPU tests render, set `RESOLVED_TEST_IMAGES=1`. Each test then
writes a PNG per device to `target/tmp/test-images/`, scaled so a white
surface facing the Sun is full white:

```sh
RESOLVED_TEST_IMAGES=1 cargo test --test bodies --test mesh
```

## License

MIT