226 lines
10 KiB
Markdown
226 lines
10 KiB
Markdown
# resolved
|
||
|
||
Renders what a spacecraft camera sees: Earth, Moon, Sun and a target
|
||
spacecraft loaded from a mesh. The inputs are an epoch, the observer and target
|
||
states and attitudes, and a camera model. Images are radiometric: the renderer
|
||
works in radiance and converts to electrons and then DN through the camera's
|
||
optics and detector, so an image should match what a real camera with the same
|
||
parameters records.
|
||
|
||
It is built for closed-loop simulation, where each step renders one frame and
|
||
latency is the number that matters. The goal is high-fidelity 1–2 MP RGB in
|
||
about 10 ms end to end on a mid-range GPU (the reference is an RTX 3060);
|
||
100 ms is the ceiling, not the aim. Lighting fidelity and camera fidelity are
|
||
selected separately, and each tier is benchmarked, so a run can trade
|
||
fidelity for speed knowing what each step costs.
|
||
|
||
Rendering goes through wgpu. On a machine without a GPU, the same shaders run
|
||
on the CPU through Mesa's lavapipe (`vulkan-swrast` on Arch,
|
||
`mesa-vulkan-drivers` on Debian and Ubuntu).
|
||
|
||
## Performance and fidelity
|
||
|
||
Lighting and camera fidelity are chosen separately: lighting tiers L0 to L3,
|
||
camera tiers C0 to C3, set with `RendererOptions::fidelity` in the library and
|
||
`--lighting` and `--camera` on the command line. A tier is a fixed bundle of
|
||
features, and a tier that needs data (L1 needs an image of Earth's surface)
|
||
refuses to start without it. A row is added here as each tier is complete, with
|
||
its measured cost and a sample image from the same scene, so the cost of each
|
||
step up is visible at a glance.
|
||
|
||
| Tier | Lighting | Camera | RTX 3060 | lavapipe (CPU) | Sample |
|
||
|---|---|---|---|---|---|
|
||
| L0 / C0 | Lambertian surfaces, Earth's and the Moon's limbs exact, eclipse by Earth and the Moon, no shadows on the target, no light from Earth | Pinhole, absolute radiometry to 12-bit DN, no noise or blur | 0.9–1.3 ms | 6–69 ms | [](docs/images/l0-c0.png) |
|
||
|
||
Times are one 1920 x 1080 RGB frame end to end (`submit` to `wait_view`, with
|
||
the DN image ready to read), median of criterion runs, over models from 12 to
|
||
752,000 triangles: the box-wing, Landsat 8 and ICESat-2. The GPU's own work is
|
||
0.3 to 0.6 ms of that. `wait`, which copies the 12 MB image into a new `Vec`,
|
||
takes 1.6 to 2.0 ms on the RTX 3060 and 7.7 to 82 ms on lavapipe. lavapipe ran
|
||
on the Intel Core Ultra 9 285K's 24 cores. Run to run, the RTX 3060's times
|
||
vary by about 5 %, and lavapipe's by up to 15 % between sessions. The sample is the Landsat 8 example scene
|
||
(`examples/scenes/landsat8.toml`). `cargo bench -p resolved` reproduces the
|
||
numbers, and `resolved render --timings N` shows the per-pass split.
|
||
|
||
L1 adds Earth's surface, shadows on the target, Earthshine and Moonshine,
|
||
the Moon's albedo map and phase curve, the Sun's limb darkening and eclipses
|
||
seen on Earth and the Moon; its row comes with its measurements. Planned: C1
|
||
and C2 add pixel-area sampling, lens distortion, blur and detector noise; L2
|
||
adds measured materials and the atmosphere; L3 is path traced.
|
||
|
||
## Usage
|
||
|
||
```rust
|
||
use resolved::nalgebra::{UnitQuaternion, Vector3};
|
||
use resolved::radiometry::AU;
|
||
use resolved::{CameraConfig, Mesh, Pose, Renderer, Scene, Target};
|
||
|
||
# fn main() -> Result<(), resolved::Error> {
|
||
// Setup, once: device, shaders, spectra folded into the camera's channels.
|
||
let mut renderer = Renderer::new(CameraConfig::example(), Default::default())?;
|
||
|
||
// A model, uploaded once. Usually `Mesh::load(path, &options)` on an STL or glTF.
|
||
let stl = "solid panel
|
||
facet normal 0 0 0
|
||
outer loop
|
||
vertex 0 -1 -1
|
||
vertex 0 1 -1
|
||
vertex 0 0 1
|
||
endloop
|
||
endfacet
|
||
endsolid panel";
|
||
let panel = renderer.load_model(&Mesh::from_stl(&mut std::io::Cursor::new(stl), "panel", &Default::default())?)?;
|
||
|
||
// Each step: inertial positions in meters, Earth at the origin.
|
||
let target = Vector3::new(7.0e6, 0.0, 0.0);
|
||
let moon = Vector3::new(0.0, 3.844e8, 0.0);
|
||
let scene = Scene {
|
||
earth_attitude: UnitQuaternion::identity(),
|
||
sun: Vector3::new(AU, 0.0, 0.0),
|
||
moon,
|
||
// Which way the Moon's surface map faces; an ephemeris gives it exactly.
|
||
moon_attitude: Scene::moon_facing_earth(moon),
|
||
targets: vec![Target { model: panel, pose: Pose::new(target, UnitQuaternion::identity()) }],
|
||
};
|
||
let observer = Pose::looking_at(target + Vector3::new(20.0, 0.0, 0.0), target, Vector3::z()).unwrap();
|
||
|
||
let pending = renderer.submit(&scene, &observer)?;
|
||
// ... the simulation's own work for this step overlaps the GPU's here ...
|
||
let frame = pending.wait()?;
|
||
// (`pending.wait_view()` lends the image where the GPU left it instead of copying it,
|
||
// saving 0.6 ms or more at 1920 x 1080; the next `submit` waits until it is dropped.)
|
||
|
||
// 12-bit DN, channels r, g, b interleaved. The panel faces the Sun and the camera.
|
||
assert!(frame.dn_at(960, 540, 1) > 1000);
|
||
# Ok(())
|
||
# }
|
||
```
|
||
|
||
## Command line
|
||
|
||
`resolved render` renders a scene described in TOML: camera overrides, the
|
||
observer, Sun and Moon positions (or an epoch, with the `astrokit` feature) and
|
||
targets with their models. `examples/scenes/` has commented examples:
|
||
`boxwing.toml` and `boxwing-epoch.toml` use a small generated model that is
|
||
in the repository, `lunar-eclipse.toml` and `solar-eclipse.toml` show two
|
||
real eclipses at lighting L1, `full-moon.toml` the Moon's map, and the rest use NASA spacecraft models
|
||
(Landsat 8, ICESat-2, OSTM/Jason-2, an SSL-1300 and ACE) that
|
||
`scripts/fetch-models.sh` downloads into `examples/models/nasa/`. That script needs Node (for `npx`),
|
||
because most NASA models are Draco-compressed and are decoded on the way in.
|
||
At lighting L1, Earth's surface comes from NASA's Blue Marble, which
|
||
`scripts/fetch-earth.sh` downloads and resizes (with ImageMagick) into
|
||
`data/earth/`, one image per month. The month comes from the scene's epoch, or
|
||
from `[appearance] earth_month` in a scene without one. The Moon's surface
|
||
comes from the LRO Camera's normalized-reflectance mosaic, which
|
||
`scripts/fetch-moon.py` downloads (1.2 GB) and assembles (with numpy and Pillow)
|
||
into `data/moon/wac-hapke-643.png`. The Moon is turned by the scene's epoch, or
|
||
by `moon_attitude` in a scene without one (by default facing Earth, which can
|
||
be off by up to about 10 degrees).
|
||
|
||
```sh
|
||
cargo run --release -p resolved-cli -- render \
|
||
--scene examples/scenes/boxwing.toml \
|
||
--out frame.png --preview preview.png --timings 50
|
||
```
|
||
|
||
- `--out` writes raw DN as a 16-bit PNG, shifted up to fill 16 bits as
|
||
cameras usually store them: a 12-bit DN is the stored value divided by 16.
|
||
It is linear, so it looks dark next to the preview.
|
||
- `--preview` writes an 8-bit sRGB image for looking at, with the bias as
|
||
black and saturation as white.
|
||
- `--timings N` renders N frames and reports setup and per-frame times.
|
||
- `--lighting L0|L1` and `--camera C0` pick the fidelity tiers (default L0
|
||
and C0). A tier the scene or the data cannot support stops with an error
|
||
naming what is missing.
|
||
- `--data DIR` says where the tiers' data is (`earth/bmng-2004-MM.jpg` and
|
||
`moon/wac-hapke-643.png`).
|
||
Without it the CLI tries `RESOLVED_DATA`, then `./data`, then the
|
||
repository's `data/`.
|
||
- `--adapter` takes `auto`, `gpu`, `cpu` or part of an adapter's name;
|
||
`resolved adapters` lists them.
|
||
- `--kernels DIR` says where astrokit's kernels are, for scenes given by
|
||
epoch. Without it the CLI tries `ASTROKIT_KERNELS`, then `data/kernels`,
|
||
then the sibling `../astrokit` checkout it was built against.
|
||
|
||
## Features
|
||
|
||
- `astrokit`: build a `Scene` from an epoch with
|
||
[astrokit](https://git.rcjohnstone.com/connor/astrokit)'s ephemerides (JPL
|
||
DE440s for the Sun and Moon, ITRF93 for Earth's orientation), through
|
||
`ephemeris::Ephemeris`. astrokit is not on crates.io, so this expects a
|
||
checkout at `../astrokit` next to this repository, and its kernels from its
|
||
`scripts/fetch-kernels.sh` (or `ASTROKIT_KERNELS` pointing at them).
|
||
|
||
## Status
|
||
|
||
Lighting L0 and L1 and camera C0 are in: the library renders Earth, Moon, Sun
|
||
and targets to DN through `Renderer`, and the CLI renders scene files. Lighting
|
||
L1 draws Earth's surface from Blue Marble and the Moon's from the LRO Camera's
|
||
map, shadows on the target, light from Earth and the Moon on it, eclipses on
|
||
Earth and the Moon, and the Sun's limb darkening. Planned, in order:
|
||
|
||
- Camera: pixel-area sampling, lens distortion, PSF, detector noise,
|
||
saturation and blooming, stars, apparent positions.
|
||
- Lighting: BRDF and spectral materials, atmospheric scattering, clouds, lunar
|
||
terrain.
|
||
- Portable ray tracing, path tracing, and label outputs for ML.
|
||
- Validation against lunar, Earth and Mitsuba references; motion blur, rolling
|
||
shutter, stray light; multi-band and 4k.
|
||
|
||
## Conventions
|
||
|
||
- SI units throughout: meters, seconds, radians, watts.
|
||
- Scene state is f64 in an inertial frame (GCRF). The GPU only sees f32
|
||
coordinates relative to the camera.
|
||
|
||
## Layout
|
||
|
||
- `crates/resolved`: the library.
|
||
- `crates/resolved-cli`: the `resolved` command, for rendering without writing
|
||
a host program.
|
||
- `examples/`: scene files and the box-wing model
|
||
(`scripts/make-example-model.py` regenerates it). The NASA scenes are
|
||
generated by `scripts/make-example-scenes.py`.
|
||
- `scripts/`: generators for compiled-in data and examples.
|
||
- `docs/optics.md`: the optics and radiometry the renderer models, explained
|
||
for readers without an optics background.
|
||
|
||
## Development
|
||
|
||
Point git at the committed hooks once per clone:
|
||
|
||
```sh
|
||
git config core.hooksPath hooks
|
||
```
|
||
|
||
`pre-commit` runs formatting, clippy and the library unit tests; `pre-push`
|
||
runs the full test suite, including doctests, and the documentation build.
|
||
|
||
The integration tests render on both a hardware GPU and lavapipe, so running
|
||
them needs a Vulkan driver for the GPU and lavapipe installed. Set
|
||
`WGPU_BACKEND`, `WGPU_ADAPTER_NAME` and the other standard wgpu environment
|
||
variables to steer adapter discovery.
|
||
|
||
Benchmarks are end-to-end frame times (`submit` then `wait`, readback
|
||
included) on the GPU and on lavapipe, at two resolutions, for the box-wing and,
|
||
once fetched, Landsat 8 and ICESat-2:
|
||
|
||
```sh
|
||
cargo bench -p resolved
|
||
```
|
||
|
||
For where a frame's time goes, `resolved render --timings N` adds GPU
|
||
timestamps around each pass (`RendererOptions::profile` in the library).
|
||
|
||
To see what the GPU tests render, set `RESOLVED_TEST_IMAGES=1`. Each test then
|
||
writes a PNG per device to `target/tmp/test-images/`, scaled so a white
|
||
surface facing the Sun is full white:
|
||
|
||
```sh
|
||
RESOLVED_TEST_IMAGES=1 cargo test --test bodies --test mesh
|
||
```
|
||
|
||
## License
|
||
|
||
MIT
|