resolved/docs/optics.md

# Optics and radiometry guide

This explains the physics the renderer models, for a reader who knows
orbital mechanics but not optics. Each section says which code implements
it. When a feature adds a new optical effect, it adds or updates a section
here in the same change.

Numbers in the examples come from `CameraConfig::example()` unless stated
otherwise: 1920 x 1080 pixels, 5.5 um pitch, 50 mm focal length, f/4, three
color channels, 12-bit output.

## The chain from Sun to number

Every pixel value comes from the same sequence:

1. The Sun emits power across a range of wavelengths.
2. Some of it lands on a surface (Earth, the Moon, the target), which
   reflects part of it in various directions.
3. The fraction reflected toward the camera enters the lens aperture. The
   optics focus it onto one pixel, losing a little to absorption.
4. The pixel converts some of the arriving photons into electrons. The
   fraction depends on wavelength and on the color filter over that pixel.
5. The electrons pile up during the exposure, up to a limit.
6. The readout electronics turn the electron count into an integer, the
   digital number (DN).

The renderer follows this chain with physical units at every step. Image
brightness therefore comes from the scene and the camera settings, not from
a tuned scale factor. Change the exposure time, and the image changes
exactly as a real camera's would.

## Geometry: the pinhole camera

Code: `camera.rs` (`Intrinsics`), `scene.rs`.

A pinhole camera maps each direction in space to one point on the focal
plane. Put the pinhole at the origin, looking along +Z. A point at camera
coordinates (x, y, z) lands on the focal plane at (f x / z, f y / z), where
f is the focal length. Dividing by the pixel pitch p converts focal-plane
distance to pixels:

```text
u = (f / p) * x / z + cx
v = (f / p) * y / z + cy
```

`f / p` is the focal length in pixels. (cx, cy) is the principal point,
where the boresight hits the image, normally the image center. For the
example camera, f / p = 50 mm / 5.5 um = 9091 pixels.

Some quantities that follow from that:

- **IFOV, the instantaneous field of view.** The angle one pixel covers:
  p / f = 110 microradians. At 10 m that is 1.1 mm per pixel; at 7000 km
  it is 770 m per pixel. This sets the smallest detail the camera can
  resolve.
- **FOV, the field of view.** The angle the whole image covers:
  2 atan(width / 2 / (f / p)), about 12.1 x 6.8 degrees here. A longer
  focal length or smaller sensor gives a narrower field of view and finer
  IFOV.

Real lenses are not perfect pinholes. They bend straight lines slightly
(distortion, planned in M3) and spread each point into a small blur (the
point spread function, also M3). The pinhole model is the starting point
those effects are added to.

### Coordinate conventions

The camera frame has +Z along the boresight, +X toward increasing column u,
and +Y toward increasing row v. This is the computer-vision convention: row
0 is the top of the image, so +Y points down in the picture. Pixel (i, j)
covers u in [i, i+1) and v in [j, j+1), so its center is at (i + 0.5,
j + 0.5).

### Why the GPU sees camera-relative coordinates

GPUs compute in 32-bit floats, which carry about 7 significant digits. A
spacecraft 7000 km from Earth's center, stored in Earth-centered f32
coordinates, can only be placed to the nearest 0.5 m. A target 10 m away
would jitter by several pixels. So all positions stay in 64-bit floats on
the CPU. They are made relative to the camera there, and converted to
32-bit only after that. A target 10 m from the camera is then placed to
about a micrometer.

### Depth

When two things overlap in the image, the nearer one must win. The GPU
keeps a depth value per pixel for this. The renderer stores near / z,
which is 1 at the near clip distance and falls toward 0 with distance.
Combined with how floating-point numbers are spaced, this gives roughly the
same relative precision at every distance. That matters in a scene that
contains a 10 m target, Earth at thousands of kilometers, and the Sun at
150 million kilometers. The scheme is called reversed-Z with an infinite
far plane.

## Radiometric quantities

Code: `spectrum.rs`, `radiometry.rs`.

Light has a few distinct measures, and the camera measures a specific one.

- **Irradiance, E, in W/m^2.** Power arriving per unit area of a surface.
  Sunlight at 1 AU delivers about 1361 W/m^2 to a surface facing the Sun.
  It falls off with the square of the distance from the Sun. A surface
  tilted by angle theta from the Sun direction receives E cos(theta).
- **Solid angle, in steradians (sr).** The 2D version of an angle: the area
  a cone cuts out of a unit sphere. The full sphere is 4 pi sr. One pixel
  of the example camera covers (110 urad)^2, about 1.2e-8 sr.
- **Radiance, L, in W/m^2/sr.** Power per unit area per unit solid angle,
  traveling in a particular direction. It is what a camera pixel responds
  to: how bright a surface looks from a given direction.

Radiance has a property that explains a lot of what a camera sees: it does
not change along a ray through empty space. Move a camera twice as far from
a wall, and each pixel sees four times as much wall area. But each square
meter now contributes a quarter of the power, so the pixel value is
unchanged. An object that covers many pixels has the same surface
brightness at any distance; it just covers fewer pixels farther away.

That stops being true once the object is smaller than a pixel. The pixel
then also sees dark space around the object, so the signal falls as one
over distance squared. That is the regime of a distant satellite seen as a
point (space situational awareness), as opposed to the resolved imagery
this renderer is mostly for.

### Reflection: albedo and Lambertian surfaces

A Lambertian surface reflects light equally in every direction. Matte
paint, the Moon (roughly) and cloud tops behave close to this. If such a
surface receives irradiance E and reflects a fraction rho (its albedo), its
radiance in every direction is:

```text
L = rho * E / pi
```

The pi comes from integrating the reflected radiance over the hemisphere
above the surface. A white Lambertian sheet facing the Sun at 1 AU, with
rho = 1, has a radiance of about 1361 / pi = 433 W/m^2/sr, summed over all
wavelengths. Earth's average albedo is about 0.3.

Real spacecraft surfaces are not Lambertian. Multilayer insulation (MLI)
and solar cells are partly mirror-like and produce bright glints at
particular angles. These are modeled by a BRDF, a function giving reflected
radiance for every pair of incoming and outgoing directions. That arrives
in M4.

## Wavelength: spectra, channels and photons

Code: `spectrum.rs` (`Spectrum`), `camera.rs` (`Channel`).

Everything above varies with wavelength. Sunlight peaks around 500 nm and
extends from the ultraviolet into the infrared. Surfaces reflect different
wavelengths differently; that is what color is. Optics transmit some
wavelengths better than others. The sensor's sensitivity also depends on
wavelength.

The renderer describes each of these as a `Spectrum`: a value as a function
of wavelength, in meters.

- **Quantum efficiency (QE).** The fraction of arriving photons that become
  electrons in a pixel. A color camera puts red, green and blue filters
  over different pixels, so each channel has its own QE curve. The example
  camera uses idealized Gaussian curves centered at 620, 540 and 460 nm,
  80 nm wide at half height, peaking at 60 %.
- **Transmission (tau).** The fraction of light the lenses pass, about 0.9
  for the example camera.

Detectors count photons, not energy. A photon of wavelength lambda carries
energy h c / lambda (h is Planck's constant, c the speed of light). So one
watt of red light delivers more photons than one watt of blue. The
renderer converts power to photons per wavelength before applying QE.

### The solar spectrum

Code: `radiometry.rs` (`solar_spectrum`), generated into `solar_data.rs` by
`scripts/fetch-solar-spectrum.sh`.

The renderer uses a measured solar spectrum, the TSIS-1 Hybrid Solar
Reference Spectrum (Coddington et al. 2021). It is the current
international reference, built mainly from the TSIS-1 instruments on the
International Space Station and filled in with a high-resolution model of
the solar absorption lines. It gives the irradiance at 1 AU, outside the atmosphere,
every 1 nm from 202 to 2730 nm. Its uncertainty is 0.3 % from 460 to
2365 nm. Over that range it sums to 1325.8 W/m^2. The other 35 W/m^2 of the
1361 W/m^2 total is at wavelengths silicon detectors do not see.

The Sun is often approximated as a 5772 K blackbody: a smooth curve with
the right total power. The real spectrum has thousands of absorption lines
from the Sun's outer layers, and its shape differs from the smooth curve.
For the example camera's channels, the blackbody gives 4.6 % (red), 7.5 %
(green) and 9.7 % (blue) fewer electrons than the measured spectrum. That
is far outside what absolute radiometry can tolerate.

### Folding spectra into per-channel constants

Code: `radiometry.rs` (`Radiometry`).

Doing all of this wavelength by wavelength for every pixel would be slow.
Most factors do not depend on the pixel: the solar spectrum, the optics,
and the QE curves. The renderer multiplies and integrates those over
wavelength once, when the camera is set up. That yields a few numbers per
channel, and the GPU only multiplies by them.

The central number is the solar flux of each channel k:

```text
S_k = integral of E_sun(lambda) * tau(lambda) * QE_k(lambda) * lambda / (h c) d lambda
```

It is the rate of electrons per square meter that sunlight at 1 AU would
produce if it fell straight onto the detector through the optics. For the
example camera, S_k is about 2.4e20 electrons per second per m^2 in red and
green, and 2.0e20 in blue.

The renderer then works in channel radiance: scene radiance weighted the
same way (transmission, QE, photons per joule), in electrons per second
per m^2 per steradian. A Lambertian surface with albedo rho, lit at angle i
from its normal, at distance d from the Sun in AU, has channel radiance:

```text
L_k = rho * cos(i) * S_k / (pi * d^2)
```

The solar disk has channel radiance S_k / Omega_sun, where Omega_sun is the
Sun's solid angle at 1 AU, 6.79e-5 sr (it is 0.53 degrees across). Seen
from anywhere, the Sun's surface has that same radiance, about 2.0e7
W/m^2/sr in total. Looked at directly, it puts about 5e8 electrons into a
pixel of the example camera's green channel in 0.1 ms, 26,000 times the
full well.

A surface whose reflectance varies with wavelength is reduced to an
effective albedo per channel. That is the albedo of a flat-spectrum surface
that would give the same signal under sunlight
(`Radiometry::effective_albedo`). This step is where the error of the
shortcut comes in. Two spectra that both vary inside one channel's band
interact in ways a single number per channel cannot capture. One example is
a surface whose reflectance changes steeply across green, lit partly by
sunlight and partly by bluish light scattered from Earth's atmosphere.
Rendering more, narrower channels reduces that error, and the code keeps
the channel count generic for that reason.

## From radiance to electrons: the camera equation

Code: `radiometry.rs` (`Radiometry::electrons`); the sensor pass, #9.

A lens with focal length f and aperture diameter D has f-number
N = f / D. The example camera is f/4: a 50 mm lens with a 12.5 mm aperture.
For a scene radiance L, the irradiance on the focal plane at the image
center is:

```text
E_image = pi * L * tau / (4 N^2)
```

The aperture collects light over a solid angle proportional to (D / f)^2 =
1 / N^2. So opening the aperture one "stop", which divides N by the square
root of 2, doubles the light. Image irradiance is independent of distance
for the same reason radiance is.

Away from the center, image irradiance falls off as cos^4(theta), where
theta is the angle from the boresight. Two factors come from the longer
path to the off-axis pixel. One comes from the aperture appearing
foreshortened. One comes from light reaching the pixel at a slant. At the
corner of the example camera, theta is 6.9 degrees, and the falloff is
3 %. Real lenses add more falloff of their own, called vignetting.

The electrons a pixel collects in one channel are then:

```text
electrons = E_image * (pixel area) * (exposure time) * (photons per joule) * QE
```

The last two factors are wavelength-dependent. In channel radiance they are
already folded in, so the camera equation reduces to

```text
electrons = L_k * pi * (pixel area) * (exposure time) / (4 N^2) * cos^4(theta)
```

For the example camera, pi * A * t / (4 N^2) is 1.48e-16 m^2 sr s.

## From electrons to DN: the detector

Code: `camera.rs` (`Detector`); the sensor pass, #9.

- **Full well.** The maximum number of electrons a pixel holds. Beyond it
  the pixel saturates and reads its maximum value whatever the light.
  Brighter scenes do not look brighter. In real sensors, excess charge can
  also spill into neighboring pixels (blooming, M3).
- **Gain.** Electrons per DN. With a gain of 5, every 5 electrons add 1 to
  the output.
- **Bias.** A fixed offset added to every pixel, so that noise around zero
  signal does not get clipped at 0. The example uses 64 DN.
- **ADC bits.** The output resolution. 12 bits gives values 0 to 4095.

So DN = bias + electrons / gain, rounded and clipped to the ADC range.

### Worked example

Take a white Lambertian surface facing the Sun at 1 AU, seen by the example
camera's green channel. Integrating the solar spectrum through the green QE
curve, f/4 optics, 5.5 um pixels and a 0.1 ms exposure gives 11,139
electrons. Red gets 11,408 and blue 9,626. (A test in `radiometry.rs`
checks the green figure.)

- DN = 64 + 11,139 / 5, about 2292, a little over half the 12-bit range.
- The full well is 20,000 electrons, so this surface is at 56 % of
  saturation.
- Earth, with albedo near 0.3, reads about 64 + 3,342 / 5, about 732 DN.

Picking the exposure time is the same trade a photographer makes. A longer
exposure saturates the sunlit target. A shorter one leaves the side of the
target lit only by Earthshine lost in noise once noise is modeled (M3).

## How shapes become pixels

Two techniques turn 3D shapes into an image. The renderer uses one for the
bodies and the other for the target.

- **Mesh.** A surface approximated by many small flat triangles that share
  corners (vertices). A cube needs 12 triangles. A curved surface needs
  many more, and it is still faceted up close. Each vertex stores a position
  and usually a normal, the direction the surface faces there. STL and glTF
  files are meshes, and so is the target spacecraft.
- **Rasterization.** The GPU's built-in way of drawing a mesh. Each
  triangle's corners are projected into the image with the pinhole
  equations above. The GPU then finds every pixel whose center falls inside
  the projected triangle, and runs a small program (a fragment shader) for
  each of those pixels to compute its value. It is fast because it only
  touches the pixels each triangle covers, and GPUs have dedicated hardware
  for it. It can only draw triangles.
- **Ray casting.** The reverse direction: for each pixel, follow the line
  of sight from the camera through the pixel center and solve for where it
  first meets a surface. For a sphere or an ellipsoid that is a quadratic
  equation with an exact answer, so there are no triangles and no facets.
  Every pixel does the work, whether or not it ends up seeing the body.

The bodies use ray casting, because their shapes have exact equations and
their edges are what limb-based navigation measures. The target uses
rasterization, because a spacecraft model has no equation, only its mesh.
Both write into the same depth image, so they sort against each other
correctly. Ray casting a mesh is also possible, by testing rays against
its triangles. That is what M5's ray tracer does, because following rays
gives exact shadows and light bouncing between surfaces. Rasterization can
only approximate those, for example with the shadow maps planned for M2.

## Earth, Moon and Sun

Code: `bodies.rs`, `gpu/bodies.rs`, `shaders/bodies.wgsl`.

The three bodies are not drawn as triangle meshes; they are ray cast. For
every pixel, the shader sends a ray from the camera through the pixel's
center and solves for where it meets each body's exact shape. The nearest hit wins. This
gives an exact edge at any zoom, at the cost of a few arithmetic operations
per pixel. A mesh would show its facets along the limb, the edge of the
disk, which is the feature horizon-based navigation measures.

- **Earth** is the WGS84 ellipsoid: 6378.137 km equatorial radius,
  6356.752 km polar. The 21 km difference matters at fine pixel scales. A
  camera with 5.5 urad pixels at 200,000 km sees the north limb 19 pixels
  away from where a spherical Earth would put it. A test checks the limb
  against the exact tangent to the ellipsoid to within a pixel.
- **The Moon** is a sphere of 1737.4 km radius. From Earth's distance it
  is about 0.52 degrees across, 82 pixels in the example camera.
- **The Sun** is a sphere of 695,700 km radius: 0.53 degrees across, about
  85 pixels in the example camera. Its disk has the solar radiance from the
  folding section above, and it is drawn only where no nearer body covers
  it. So Earth eclipses it naturally.

### Shading

Earth and the Moon are Lambertian, with one albedo each, set by
`Appearance`: 0.3 for Earth and 0.12 for the Moon, flat across wavelength.
Each point's radiance is rho cos(i) S_k / (pi d^2), with i the angle between
the surface normal and the Sun. The night side has i past 90 degrees and is
black. The Sun's distance d is taken from each body's center, since the
difference across a body is a few parts in 10^5.

This gets the overall brightness and the phase (how much of the disk is
lit) right. It misses some things that are visible in real images:

- Earth's oceans, land and clouds differ in brightness by a factor of
  several. An albedo map comes in M2.
- The full Moon looks almost uniformly bright out to its edge. A Lambertian
  sphere instead darkens toward the edge, because the light there arrives
  and leaves at grazing angles. Lunar soil scatters light back toward its
  source much more strongly than Lambert predicts, and a dedicated lunar
  reflectance model (M4) fixes that. The Moon's maria (the dark plains of
  old basalt lava, the "seas" seen with the naked eye) and its bright
  highlands also differ in albedo by about a factor of two (M2 albedo
  map).
- Earth's atmosphere softens the limb and adds a thin bright band around it
  (M4). Here Earth's edge is a sharp line.

### Depth from the bodies

The pass also writes each hit's depth, near / z, as described in the
geometry section. The target spacecraft, drawn after the bodies, uses the
same depth buffer. So a target in front of Earth covers it, and a target
behind Earth's limb is hidden.

## The target model and its materials

Code: `model.rs`.

The target spacecraft is a triangle mesh loaded from STL or glTF. Each
piece (an STL file, or a glTF primitive) is a part with one material, and
for now a material is a Lambertian albedo spectrum, as for Earth and the
Moon.

Surface normals decide how bright each triangle is, through the cos(i)
factor. STL files store a normal per triangle, but exporters often write
zeros or wrong values, so the loader ignores them. It recomputes each
normal from the order of the triangle's corners: counterclockwise as seen
from outside. Every triangle is then lit as a flat facet, which matches the
flat panels and boxes most spacecraft are built from. glTF files that store
normals keep them.

glTF describes a material's color as red, green and blue reflectances (its
"base color"). Three numbers do not define a spectrum: very different
spectra can look the same color to the eye. The loader turns them into a
stepped spectrum, blue below 490 nm, green from 500 to 580 nm, and red above
590 nm. That is a reasonable guess for painted surfaces seen by an RGB
camera. It is a poor one for a camera whose channels differ from the eye's,
or for anything in the near infrared, where the red value simply continues.
For absolute radiometry of a real spacecraft, give the important materials
measured reflectance spectra (M4 adds those, and BRDFs).

STL has no units. CAD tools usually export millimeters, so loading a CAD
model usually needs `scale: 0.001`. glTF is always meters.

## Drawing the target

Code: `gpu/mesh.rs`, `shaders/mesh.wgsl`.

The target's mesh is rasterized after the bodies, into the same radiance
and depth images. Each pixel the target covers is kept only if the target
is nearer than what is already there. That one rule handles a target in
front of Earth, a target behind Earth's limb, and parts of the target
hiding other parts.

Each point on the target is shaded like the bodies: albedo times cos(i)
times the sunlight that reaches it. A panel modeled with no thickness, such
as a solar array, has one surface but two sides. Whichever side faces the
camera is the side that is lit. A closed shape never shows its inside, so
this only matters for thin parts.

### Eclipses

Code: `eclipse.rs`.

The Sun is a disk, not a point. So as a spacecraft passes into Earth's
shadow, the light fades over a few seconds instead of switching off:

- **Umbra.** Earth covers the whole solar disk. No direct sunlight.
- **Penumbra.** Earth covers part of the disk. 7000 km from Earth's
  center, on the night side, the penumbra is about 65 km wide; a satellite
  crosses it in under 10 seconds.
- **Annular.** A body small enough to sit inside the solar disk blocks a
  ring-shaped bite. From orbit this happens with the Moon.

The renderer finds the visible fraction of the solar disk from the overlap
of two circles on the sky. The circles are the Sun's disk and the
occulter's, each with its apparent angular radius. It multiplies the
target's sunlight by that fraction. The model is the standard one for
satellites (Montenbruck and Gill), the same as astrokit's. A test checks it
against tracing lines from the target to thousands of points spread over
the solar disk.

The fraction is computed once per target, at its center. The whole target
dims together, which is accurate unless the target is a sizable fraction of
the penumbra's width. Light reaching the target from Earth itself
(Earthshine), which matters most when the target is in shadow, comes in M2.

## Not modeled yet

| Effect | Where |
|---|---|
| Light from Earth and the Moon illuminating the target (Earthshine, Moonshine) | M2 |
| Earth's shadow on the Moon (lunar eclipses) | M2 |
| Earthshine on the Moon's night side | M4 |
| Shadows the target casts on itself | M2 |
| The Sun as a disk that is darker toward its edge (limb darkening) | M2 |
| Lens distortion | M3 |
| Blur from diffraction and lens imperfections (point spread function) | M3 |
| Noise: photon shot noise, read noise, dark current, pixel-to-pixel variation | M3 |
| Saturation spilling into neighbors (blooming) | M3 |
| Non-Lambertian materials (BRDFs), glints, lunar backscatter | M4 |
| Earth and Moon albedo maps | M2 |
| Earth's atmosphere: scattering, the bright limb | M4 |
| Motion blur, rolling shutter, stray light | M6 |