# resolved

Renders what a spacecraft camera sees: Earth, Moon, Sun and a target
spacecraft loaded from a mesh. The inputs are an epoch, the observer and target
states and attitudes, and a camera model. Images are radiometric: the renderer
works in radiance and converts to electrons and then DN through the camera's
optics and detector, so an image should match what a real camera with the same
parameters records.

It is built for closed-loop simulation, where each step renders one frame and
latency is the number that matters. The goal is high-fidelity 1–2 MP RGB in
about 10 ms end to end on a mid-range GPU (the reference is an RTX 3060);
100 ms is the ceiling, not the aim. Lighting fidelity and camera fidelity are
selected separately, and each tier is benchmarked, so a run can trade
fidelity for speed knowing what each step costs.

Rendering goes through wgpu. On a machine without a GPU, the same shaders run
on the CPU through Mesa's lavapipe (`vulkan-swrast` on Arch,
`mesa-vulkan-drivers` on Debian and Ubuntu).

## Performance and fidelity

Lighting and camera fidelity are chosen separately: lighting tiers L0 to L3,
camera tiers C0 to C3, set with `RendererOptions::fidelity` in the library and
`--lighting` and `--camera` on the command line. A tier is a fixed bundle of
features, and a tier that needs data (L1 needs an image of Earth's surface)
refuses to start without it. A row is added here as each tier is complete, with
its measured cost and a sample image from the same scene, so the cost of each
step up is visible at a glance.

| Tier | Lighting | Camera | RTX 3060 | lavapipe (CPU) | Sample |
|---|---|---|---|---|---|
| L0 / C0 | Lambertian surfaces, Earth's and the Moon's limbs exact, eclipse by Earth and the Moon, no shadows on the target, no light from Earth | Pinhole, absolute radiometry to 12-bit DN, no noise or blur | 1.8–2.1 ms | 8–65 ms | [![L0/C0](docs/images/l0-c0.png)](docs/images/l0-c0.png) |

Times are one 1920 x 1080 RGB frame end to end (`submit` to `wait`, readback
of the DN image included), median of criterion runs, over models from 12 to
752,000 triangles: the box-wing, Landsat 8 and ICESat-2. The GPU's own work is
0.3 to 0.6 ms of that; reading 12 MB back and copying it out is most of the
rest. lavapipe ran on the Intel Core Ultra 9 285K's 24 cores. The run-to-run
noise is about 3 %. The sample is the Landsat 8 example scene
(`examples/scenes/landsat8.toml`). `cargo bench -p resolved` reproduces the
numbers, and `resolved render --timings N` shows the per-pass split.

Planned tiers: L1 adds Earth's surface, shadows on the target, Earthshine
and Moonshine, the Moon's albedo map, the Sun's limb darkening and eclipses
seen on Earth and the Moon (M2); C1 and C2 add pixel-area sampling, lens
distortion, blur and detector noise (M3); L2 adds measured materials and the
atmosphere (M4); L3 is path traced (M5).

## Usage

```rust
use resolved::nalgebra::{UnitQuaternion, Vector3};
use resolved::radiometry::AU;
use resolved::{CameraConfig, Mesh, Pose, Renderer, Scene, Target};

# fn main() -> Result<(), resolved::Error> {
// Setup, once: device, shaders, spectra folded into the camera's channels.
let mut renderer = Renderer::new(CameraConfig::example(), Default::default())?;

// A model, uploaded once. Usually `Mesh::load(path, &options)` on an STL or glTF.
let stl = "solid panel
facet normal 0 0 0
outer loop
vertex 0 -1 -1
vertex 0 1 -1
vertex 0 0 1
endloop
endfacet
endsolid panel";
let panel = renderer.load_model(&Mesh::from_stl(&mut std::io::Cursor::new(stl), "panel", &Default::default())?)?;

// Each step: inertial positions in meters, Earth at the origin.
let target = Vector3::new(7.0e6, 0.0, 0.0);
let scene = Scene {
    earth_attitude: UnitQuaternion::identity(),
    sun: Vector3::new(AU, 0.0, 0.0),
    moon: Vector3::new(0.0, 3.844e8, 0.0),
    targets: vec![Target { model: panel, pose: Pose::new(target, UnitQuaternion::identity()) }],
};
let observer = Pose::looking_at(target + Vector3::new(20.0, 0.0, 0.0), target, Vector3::z()).unwrap();

let pending = renderer.submit(&scene, &observer)?;
// ... the simulation's own work for this step overlaps the GPU's here ...
let frame = pending.wait()?;

// 12-bit DN, channels r, g, b interleaved. The panel faces the Sun and the camera.
assert!(frame.dn_at(960, 540, 1) > 1000);
# Ok(())
# }
```

## Command line

`resolved render` renders a scene described in TOML: camera overrides, the
observer, Sun and Moon positions (or an epoch, with the `astrokit` feature) and
targets with their models. `examples/scenes/` has commented examples:
`boxwing.toml` and `boxwing-epoch.toml` use a small generated model that is
in the repository, and the rest use NASA spacecraft models (Landsat 8,
ICESat-2, OSTM/Jason-2, an SSL-1300 and ACE) that `scripts/fetch-models.sh`
downloads into `examples/models/nasa/`. That script needs Node (for `npx`),
because most NASA models are Draco-compressed and are decoded on the way in.
At lighting L1, Earth's surface comes from NASA's Blue Marble, which
`scripts/fetch-earth.sh` downloads and resizes (with ImageMagick) into
`data/earth/`, one image per month. The month comes from the scene's epoch, or
from `[appearance] earth_month` in a scene without one.

```sh
cargo run --release -p resolved-cli -- render \
    --scene examples/scenes/boxwing.toml \
    --out frame.png --preview preview.png --timings 50
```

- `--out` writes raw DN as a 16-bit PNG, shifted up to fill 16 bits as
  cameras usually store them: a 12-bit DN is the stored value divided by 16.
  It is linear, so it looks dark next to the preview.
- `--preview` writes an 8-bit sRGB image for looking at, with the bias as
  black and saturation as white.
- `--timings N` renders N frames and reports setup and per-frame times.
- `--lighting L0|L1` and `--camera C0` pick the fidelity tiers (default L0
  and C0). A tier the scene or the data cannot support stops with an error
  naming what is missing.
- `--data DIR` says where the tiers' data is (`earth/bmng-2004-MM.jpg`).
  Without it the CLI tries `RESOLVED_DATA`, then `./data`, then the
  repository's `data/`.
- `--adapter` takes `auto`, `gpu`, `cpu` or part of an adapter's name;
  `resolved adapters` lists them.
- `--kernels DIR` says where astrokit's kernels are, for scenes given by
  epoch. Without it the CLI tries `ASTROKIT_KERNELS`, then `data/kernels`,
  then the sibling `../astrokit` checkout it was built against.

## Features

- `astrokit`: build a `Scene` from an epoch with
  [astrokit](https://git.rcjohnstone.com/connor/astrokit)'s ephemerides (JPL
  DE440s for the Sun and Moon, ITRF93 for Earth's orientation), through
  `ephemeris::Ephemeris`. astrokit is not on crates.io, so this expects a
  checkout at `../astrokit` next to this repository, and its kernels from its
  `scripts/fetch-kernels.sh` (or `ASTROKIT_KERNELS` pointing at them).

## Status

M1 is complete: the library renders Earth, Moon, Sun and targets to DN
through `Renderer`, the CLI renders scene files, and the benchmarks give the
L0 / C0 baseline above. M2 is in progress: lighting L1 so far draws Earth's
surface from Blue Marble, shadows on the target, and light from Earth and the
Moon on it. The work is tracked as
Gitea milestones, each split into one issue per change:

| Milestone | Scope |
|---|---|
| M1 | Scene API, headless wgpu, analytic Earth/Moon/Sun, glTF and STL targets, Lambert shading with eclipse, pinhole camera, radiance to DN, CLI, benchmark baseline |
| M2 | Earth surface image, self-shadowing, Earthshine and Moonshine, Moon albedo map, Sun limb darkening, lunar and solar eclipses on the bodies, lower readback overhead |
| M3 | Pixel-area sampling, lens distortion, PSF, detector noise, saturation and blooming, stars, apparent positions |
| M4 | BRDF and spectral materials, atmospheric scattering, clouds, lunar terrain |
| M5 | Portable ray tracing, path tracing, label outputs for ML |
| M6 | Validation against lunar, Earth and Mitsuba references; motion blur, rolling shutter, stray light; multi-band and 4k |

## Conventions

- SI units throughout: meters, seconds, radians, watts.
- Scene state is f64 in an inertial frame (GCRF). The GPU only sees f32
  coordinates relative to the camera.

## Layout

- `crates/resolved`: the library.
- `crates/resolved-cli`: the `resolved` command, for rendering without writing
  a host program.
- `examples/`: scene files and the box-wing model
  (`scripts/make-example-model.py` regenerates it). The NASA scenes are
  generated by `scripts/make-example-scenes.py`.
- `scripts/`: generators for compiled-in data and examples.
- `docs/optics.md`: the optics and radiometry the renderer models, explained
  for readers without an optics background.

## Development

Point git at the committed hooks once per clone:

```sh
git config core.hooksPath hooks
```

`pre-commit` runs formatting, clippy and the library unit tests; `pre-push`
runs the full test suite, including doctests, and the documentation build.

The integration tests render on both a hardware GPU and lavapipe, so running
them needs a Vulkan driver for the GPU and lavapipe installed. Set
`WGPU_BACKEND`, `WGPU_ADAPTER_NAME` and the other standard wgpu environment
variables to steer adapter discovery.

Benchmarks are end-to-end frame times (`submit` then `wait`, readback
included) on the GPU and on lavapipe, at two resolutions, for the box-wing and,
once fetched, Landsat 8 and ICESat-2:

```sh
cargo bench -p resolved
```

For where a frame's time goes, `resolved render --timings N` adds GPU
timestamps around each pass (`RendererOptions::profile` in the library).

To see what the GPU tests render, set `RESOLVED_TEST_IMAGES=1`. Each test then
writes a PNG per device to `target/tmp/test-images/`, scaled so a white
surface facing the Sun is full white:

```sh
RESOLVED_TEST_IMAGES=1 cargo test --test bodies --test mesh
```

## License

MIT