Files
resolved/docs/optics.md
T

2338 lines
127 KiB
Markdown

# Optics and radiometry guide
This explains the physics the renderer models, for a reader who knows
orbital mechanics but not optics. Each section says which code implements
it. When a feature adds a new optical effect, it adds or updates a section
here in the same change.
Numbers in the examples come from `CameraConfig::example()` unless stated
otherwise: 1920 x 1080 pixels, 5.5 um pitch, 50 mm focal length, f/4, three
color channels, 12-bit output.
## The chain from Sun to number
Every pixel value comes from the same sequence:
1. The Sun emits power across a range of wavelengths.
2. Some of it lands on a surface (Earth, the Moon, the target), which
reflects part of it in various directions.
3. The fraction reflected toward the camera enters the lens aperture. The
optics focus it onto the detector, losing a little to absorption. Each
point of the scene lands as a small spot rather than a point.
4. The pixel converts some of the arriving photons into electrons. The
fraction depends on wavelength and on the color filter over that pixel.
5. The electrons pile up during the exposure, up to a limit.
6. The readout electronics turn the electron count into an integer, the
digital number (DN).
The renderer follows this chain with physical units at every step. Image
brightness therefore comes from the scene and the camera settings, not from
a tuned scale factor. Change the exposure time, and the image changes
exactly as a real camera's would.
## Geometry: the pinhole camera
Code: `camera.rs` (`Intrinsics`), `scene.rs`.
A pinhole camera maps each direction in space to one point on the focal
plane. Put the pinhole at the origin, looking along +Z. A point at camera
coordinates (x, y, z) lands on the focal plane at (f x / z, f y / z), where
f is the focal length. Dividing by the pixel pitch p converts focal-plane
distance to pixels:
```text
u = (f / p) * x / z + cx
v = (f / p) * y / z + cy
```
`f / p` is the focal length in pixels. (cx, cy) is the principal point,
where the boresight hits the image, normally the image center. For the
example camera, f / p = 50 mm / 5.5 um = 9091 pixels.
Some quantities that follow from that:
- **IFOV, the instantaneous field of view.** The angle one pixel covers:
p / f = 110 microradians. At 10 m that is 1.1 mm per pixel; at 7000 km
it is 770 m per pixel. This sets the smallest detail the camera can
resolve.
- **FOV, the field of view.** The angle the whole image covers:
2 atan(width / 2 / (f / p)), about 12.1 x 6.8 degrees here. A longer
focal length or smaller sensor gives a narrower field of view and finer
IFOV.
Real lenses are not perfect pinholes. They bend straight lines slightly
(distortion, next) and spread each point into a small blur (the point
spread function, below). The pinhole model is the starting point those
effects are added to.
### Lens distortion
Code: `camera.rs` (`BrownConrady`, `Intrinsics`); `gpu/warp.rs`,
`gpu/blur.rs`, `shaders/blur.wgsl`, `shaders/distortion.wgsl`.
A real lens moves each point a little away from where the pinhole puts it,
by an amount that grows toward the edge of the image. Straight lines in the
scene come out curved. When points move outward more the farther out they
are, the corners stretch (pincushion distortion); when they move inward,
the image bulges (barrel distortion). A lens element slightly off center
adds a smaller sideways shift (tangential distortion).
The standard model, Brown-Conrady, works on the pinhole's normalized
coordinates x = X / Z and y = Y / Z, with r^2 = x^2 + y^2:
```text
x_d = x (1 + k1 r^2 + k2 r^4 + k3 r^6) + 2 p1 x y + p2 (r^2 + 2 x^2)
y_d = y (1 + k1 r^2 + k2 r^4 + k3 r^6) + p1 (r^2 + 2 y^2) + 2 p2 x y
u = (f / p) x_d + cx, v = (f / p) y_d + cy
```
k1, k2 and k3 are the radial terms, positive for pincushion and negative
for barrel; p1 and p2 are the tangential terms. This is OpenCV's convention,
so the coefficients a camera calibration produces apply directly (OpenCV
lists them in the order k1, k2, p1, p2, k3). For the example camera, k1 =
0.1 alone moves the corners 1.6 px outward.
That matters for navigation, not just for looks. A limb fit, a star
identification or a target's centroid all assume where each pixel looks;
an unmodeled 1.6 px is a 0.2 mrad pointing error at the corner, which at
36,000 km is 6 km. `Intrinsics::project` and `Intrinsics::ray` include the
distortion (the inverse by Newton's method, to 1e-10 px), so navigation code
and the renderer share one model; the projection matches OpenCV's to 1e-9
px.
Rendering it, from camera tier C1 (C0 is always a pinhole): the GPU draws
the target's triangles with a linear projection, which keeps straight edges
straight, and moving their corners alone would leave long edges straight
when they should curve. So the renderer draws the whole frame without
distortion, the bodies and targets sorted together as usual, on a slightly
larger and offset area that covers everything the lens bends into view.
Then it resamples it: for each pixel of the focal plane it finds the
undistorted point the lens bends onto that pixel and reads the render
there, interpolating between the four nearest render pixels. The blur's
first pass (below) does this as it reads its input, so the distorted image
is never written out. Radiance is
the same at both ends of a ray, so moving it is all distortion does to an
extended scene. The resampling works on the finer image C1 renders (below),
before the blur, which is where the lens puts it in a real camera.
Two effects of distortion fall on the detector instead. The cos^4 falloff
uses each pixel's real line of sight, the undistorted one. And where the
lens stretches the image, the same light covers more pixels: each pixel's
signal is divided by how much the distortion stretches area there (the
determinant of its derivative), 0.6 % at the corners for k1 = 0.1.
Measured with coefficients of 0.3, -0.2, 0.1 (radial) and 0.001, -0.0008
(tangential), which move the corners about 4 px: a small Moon's light lands
centered where `Intrinsics::project` puts it to within 0.004 px across the
frame, and a long straight edge follows the predicted curve to within 0.07
px in every row and 0.02 px averaged over rows. The interpolation adds a
little blur where it falls between render pixels, at most a variance of
1/16 px^2 at 2 x 2 rendering (computed), which takes sigma 0.5 px to 0.56 px
at worst.
Distortion costs 0.25 to 0.32 ms per 1920 x 1080 frame on an RTX 3060 at
2 x 2 rendering, and 1.1 to 1.3 ms at 3840 x 2160, almost all of it the
blur's first pass reading through it, and under 0.1 ms the larger
undistorted render. A separate pass writing the distorted image out and the
blur reading it back cost 0.7 to 0.8 ms and 2.8 to 3.3 ms: at those sizes
both are limited by how fast the card moves the image through memory. On
lavapipe distortion adds 10 to 11 ms to 1920 x 1080 frames.
### Coordinate conventions
The camera frame has +Z along the boresight, +X toward increasing column u,
and +Y toward increasing row v. This is the computer-vision convention: row
0 is the top of the image, so +Y points down in the picture. Pixel (i, j)
covers u in [i, i+1) and v in [j, j+1), so its center is at (i + 0.5,
j + 0.5).
### Why the GPU sees camera-relative coordinates
GPUs compute in 32-bit floats, which carry about 7 significant digits. A
spacecraft 7000 km from Earth's center, stored in Earth-centered f32
coordinates, can only be placed to the nearest 0.5 m. A target 10 m away
would jitter by several pixels. So all positions stay in 64-bit floats on
the CPU. They are made relative to the camera there, and converted to
32-bit only after that. A target 10 m from the camera is then placed to
about a micrometer.
### Depth
When two things overlap in the image, the nearer one must win. The GPU
keeps a depth value per pixel for this. The renderer stores near / z,
which is 1 at the near clip distance and falls toward 0 with distance.
Combined with how floating-point numbers are spaced, this gives roughly the
same relative precision at every distance. That matters in a scene that
contains a 10 m target, Earth at thousands of kilometers, and the Sun at
150 million kilometers. The scheme is called reversed-Z with an infinite
far plane.
## Radiometric quantities
Code: `spectrum.rs`, `radiometry.rs`.
Light has a few distinct measures, and the camera measures a specific one.
- **Irradiance, E, in W/m^2.** Power arriving per unit area of a surface.
Sunlight at 1 AU delivers about 1361 W/m^2 to a surface facing the Sun.
It falls off with the square of the distance from the Sun. A surface
tilted by angle theta from the Sun direction receives E cos(theta).
- **Solid angle, in steradians (sr).** The 2D version of an angle: the area
a cone cuts out of a unit sphere. The full sphere is 4 pi sr. One pixel
of the example camera covers (110 urad)^2, about 1.2e-8 sr.
- **Radiance, L, in W/m^2/sr.** Power per unit area per unit solid angle,
traveling in a particular direction. It is what a camera pixel responds
to: how bright a surface looks from a given direction.
Radiance has a property that explains a lot of what a camera sees: it does
not change along a ray through empty space. Move a camera twice as far from
a wall, and each pixel sees four times as much wall area. But each square
meter now contributes a quarter of the power, so the pixel value is
unchanged. An object that covers many pixels has the same surface
brightness at any distance; it just covers fewer pixels farther away.
That stops being true once the object is smaller than a pixel. The pixel
then also sees dark space around the object, so the signal falls as one
over distance squared. That is the regime of a distant satellite seen as a
point (space situational awareness), as opposed to the resolved imagery
this renderer is mostly for.
### Reflection: albedo and Lambertian surfaces
A Lambertian surface reflects light equally in every direction. Matte
paint, the Moon (roughly) and cloud tops behave close to this. If such a
surface receives irradiance E and reflects a fraction rho (its albedo), its
radiance in every direction is:
```text
L = rho * E / pi
```
The pi comes from integrating the reflected radiance over the hemisphere
above the surface. A white Lambertian sheet facing the Sun at 1 AU, with
rho = 1, has a radiance of about 1361 / pi = 433 W/m^2/sr, summed over all
wavelengths. Earth's average albedo is about 0.3.
Real spacecraft surfaces are not Lambertian. Multilayer insulation (MLI)
and solar cells are partly mirror-like and produce bright glints at
particular angles. These are modeled by a BRDF, a function giving reflected
radiance for every pair of incoming and outgoing directions. BRDFs are not
modeled yet.
## Wavelength: spectra, channels and photons
Code: `spectrum.rs` (`Spectrum`), `camera.rs` (`Channel`).
Everything above varies with wavelength. Sunlight peaks around 500 nm and
extends from the ultraviolet into the infrared. Surfaces reflect different
wavelengths differently; that is what color is. Optics transmit some
wavelengths better than others. The sensor's sensitivity also depends on
wavelength.
The renderer describes each of these as a `Spectrum`: a value as a function
of wavelength, in meters.
- **Quantum efficiency (QE).** The fraction of arriving photons that become
electrons in a pixel. A color camera puts red, green and blue filters
over different pixels, so each channel has its own QE curve. The example
camera uses idealized Gaussian curves centered at 620, 540 and 460 nm,
80 nm wide at half height, peaking at 60 %.
- **Transmission (tau).** The fraction of light the lenses pass, about 0.9
for the example camera.
Detectors count photons, not energy. A photon of wavelength lambda carries
energy h c / lambda (h is Planck's constant, c the speed of light). So one
watt of red light delivers more photons than one watt of blue. The
renderer converts power to photons per wavelength before applying QE.
### The solar spectrum
Code: `radiometry.rs` (`solar_spectrum`), generated into `solar_data.rs` by
`scripts/fetch-solar-spectrum.sh`.
The renderer uses a measured solar spectrum, the TSIS-1 Hybrid Solar
Reference Spectrum (Coddington et al. 2021). It is the current
international reference, built mainly from the TSIS-1 instruments on the
International Space Station and filled in with a high-resolution model of
the solar absorption lines. It gives the irradiance at 1 AU, outside the atmosphere,
every 1 nm from 202 to 2730 nm. Its uncertainty is 0.3 % from 460 to
2365 nm. Over that range it sums to 1325.8 W/m^2. The other 35 W/m^2 of the
1361 W/m^2 total is at wavelengths silicon detectors do not see.
The Sun is often approximated as a 5772 K blackbody: a smooth curve with
the right total power. The real spectrum has thousands of absorption lines
from the Sun's outer layers, and its shape differs from the smooth curve.
For the example camera's channels, the blackbody gives 4.6 % (red), 7.5 %
(green) and 9.7 % (blue) fewer electrons than the measured spectrum. That
is far outside what absolute radiometry can tolerate.
### Folding spectra into per-channel constants
Code: `radiometry.rs` (`Radiometry`).
Doing all of this wavelength by wavelength for every pixel would be slow.
Most factors do not depend on the pixel: the solar spectrum, the optics,
and the QE curves. The renderer multiplies and integrates those over
wavelength once, when the camera is set up. That yields a few numbers per
channel, and the GPU only multiplies by them.
The central number is the solar flux of each channel k:
```text
S_k = integral of E_sun(lambda) * tau(lambda) * QE_k(lambda) * lambda / (h c) d lambda
```
It is the rate of electrons per square meter that sunlight at 1 AU would
produce if it fell straight onto the detector through the optics. For the
example camera, S_k is about 2.4e20 electrons per second per m^2 in red and
green, and 2.0e20 in blue.
The renderer then works in channel radiance: scene radiance weighted the
same way (transmission, QE, photons per joule), in electrons per second
per m^2 per steradian. A Lambertian surface with albedo rho, lit at angle i
from its normal, at distance d from the Sun in AU, has channel radiance:
```text
L_k = rho * cos(i) * S_k / (pi * d^2)
```
The solar disk has channel radiance S_k / Omega_sun, where Omega_sun is the
Sun's solid angle at 1 AU, 6.79e-5 sr (it is 0.53 degrees across). Seen
from anywhere, the Sun's surface has that same radiance, about 2.0e7
W/m^2/sr in total. Looked at directly, it puts about 5e8 electrons into a
pixel of the example camera's green channel in 0.1 ms, 26,000 times the
full well.
A surface whose reflectance varies with wavelength is reduced to an
effective albedo per channel. That is the albedo of a flat-spectrum surface
that would give the same signal under sunlight
(`Radiometry::effective_albedo`). This step is where the error of the
shortcut comes in. Two spectra that both vary inside one channel's band
interact in ways a single number per channel cannot capture. One example is
a surface whose reflectance changes steeply across green, lit partly by
sunlight and partly by bluish light scattered from Earth's atmosphere.
Rendering more, narrower channels reduces that error, and the code keeps
the channel count generic for that reason.
## From radiance to electrons: the camera equation
Code: `radiometry.rs` (`Radiometry::electrons`); `gpu/sensor.rs`, `shaders/sensor.wgsl`.
A lens with focal length f and aperture diameter D has f-number
N = f / D. The example camera is f/4: a 50 mm lens with a 12.5 mm aperture.
For a scene radiance L, the irradiance on the focal plane at the image
center is:
```text
E_image = pi * L * tau / (4 N^2)
```
The aperture collects light over a solid angle proportional to (D / f)^2 =
1 / N^2. So opening the aperture one "stop", which divides N by the square
root of 2, doubles the light. Image irradiance is independent of distance
for the same reason radiance is.
Away from the center, image irradiance falls off as cos^4(theta), where
theta is the angle from the boresight. Two factors come from the longer
path to the off-axis pixel. One comes from the aperture appearing
foreshortened. One comes from light reaching the pixel at a slant. At the
corner of the example camera, theta is 6.9 degrees, and the falloff is
3 %. Real lenses add more falloff of their own, called vignetting.
The electrons a pixel collects in one channel are then:
```text
electrons = E_image * (pixel area) * (exposure time) * (photons per joule) * QE
```
The last two factors are wavelength-dependent. In channel radiance they are
already folded in, so the camera equation reduces to
```text
electrons = L_k * pi * (pixel area) * (exposure time) / (4 N^2) * cos^4(theta)
```
For the example camera, pi * A * t / (4 N^2) is 1.48e-16 m^2 sr s.
## From electrons to DN: the detector
Code: `camera.rs` (`Detector::dn`, `Detector::variance`); `gpu/sensor.rs`,
`shaders/sensor.wgsl`.
- **Full well.** The maximum number of electrons a pixel holds. Beyond it
the pixel saturates and reads its maximum value whatever the light.
Brighter scenes do not look brighter. In real sensors, excess charge can
also spill into neighboring pixels (blooming, below).
- **Gain.** Electrons per DN. With a gain of 5, every 5 electrons add 1 to
the output.
- **Bias.** A fixed offset added to every pixel, so that noise around zero
signal does not get clipped at 0. The example uses 64 DN.
- **ADC bits.** The output resolution. 12 bits gives values 0 to 4095.
So DN = bias + min(electrons, full well) / gain, rounded and clipped to the
ADC range. The renderer does this on the GPU for every pixel and channel,
as the last step of a frame, and reads back only the resulting integers.
From camera tier C1, noise enters between the electrons and the gain (see
Noise below).
Two consequences of these numbers show up in every image:
- **Saturation is not always the ADC's maximum.** In the example camera the
full well fills first: 20,000 / 5 + 64 = 4064 DN, below the 12-bit maximum
of 4095. A saturated pixel there reads 4064. Real cameras are designed
either way, and an algorithm that detects saturation by looking for 4095
would miss it in this one.
- **Black is not zero.** A pixel that receives no light reads the bias,
64 DN, so empty space in a raw frame displays as dark gray.
Raw frames are also not white-balanced. The camera reports what each
channel collected, and the channels collect different amounts from the same
gray surface. In the example camera, sunlight on a gray surface gives about
15 % fewer electrons in blue than in red, so gray things look slightly
yellow-green in a raw image. A white balance step, applied after capture,
would scale the channels to make them equal. The renderer leaves it out,
because it simulates what the sensor records.
### Worked example
Take a white Lambertian surface facing the Sun at 1 AU, seen by the example
camera's green channel. Integrating the solar spectrum through the green QE
curve, f/4 optics, 5.5 um pixels and a 0.1 ms exposure gives 11,139
electrons. Red gets 11,408 and blue 9,626. (A test in `radiometry.rs`
checks the green figure.)
- DN = 64 + 11,139 / 5, about 2292, a little over half the 12-bit range.
- The full well is 20,000 electrons, so this surface is at 56 % of
saturation.
- Earth, with albedo near 0.3, reads about 64 + 3,342 / 5, about 732 DN.
- The same white surface at the image corner, 6.9 degrees off axis, gets
cos^4 of that, 97 %: 10,819 electrons, 2228 DN.
The sensor tests check the 2292, 2228, 732, 4064 and 64 DN figures
exactly, on the GPU and on lavapipe.
Picking the exposure time is the same trade a photographer makes. A longer
exposure saturates the sunlit target. A shorter one leaves the side of the
target lit only by Earthshine lost in noise.
### Noise
Code: `Detector::variance`; `sensor.wgsl` (`NOISE`).
A real image is noisy, and the noise decides what can be detected at all.
From camera tier C1 the renderer adds three kinds, following the EMVA 1288
standard's model of a camera:
- **Shot noise.** Light arrives as photons at random times, so a pixel
expecting N electrons collects a Poisson-distributed count with mean N and
standard deviation sqrt(N). It is a property of light, not of the camera:
a perfect detector still has it. A pixel expecting 10,000 electrons varies
by 100 (1 %); one expecting 100 varies by 10 (10 %).
- **Dark current.** The detector makes electrons of its own from heat,
`Detector::dark_current` per second, which add to the light's and carry
shot noise of their own. It grows with temperature, roughly doubling every
6 to 8 degrees C; the value given is for the detector's operating
temperature.
- **Read noise.** Reading a pixel out adds a few electrons of random error,
`Detector::read_noise` RMS, to every pixel in every frame, light or not.
In order: the light's electrons and the dark electrons together are drawn
as one Poisson count (a sum of Poisson counts is itself one); the count is
clipped to the full well; read noise is added; then the gain, bias and
rounding to DN as before. Below a mean of 50 electrons the count is drawn
exactly. Above it a Gaussian of the same mean and variance takes its place,
which a Poisson count's shape is already close to and which costs the same
at any level. With read noise the DN can fall below the bias, which is what
the bias is for.
Each pixel's variance in electrons^2 is then
variance = (electrons + dark current x exposure) + read noise^2 + gain^2 / 12
where the last term is the rounding to whole DN. Divided by gain^2 it is in
DN^2. Plotted against the mean signal in DN, the variance is a straight line
with slope 1 / gain and intercept (read noise^2 + dark shot noise) / gain^2:
the photon transfer curve, the standard way to measure a camera's gain and
read noise from flat fields alone.
For the example camera (5 electrons read noise, 20 electrons per second dark
current, gain 5) at 0.1 ms, the signal-to-noise ratio, signal over the
square root of the variance, is:
| What | Electrons | SNR |
|---|---|---|
| A white surface facing the Sun, green channel | 11,139 | 105 |
| Earth, albedo 0.3 | 3,342 | 58 |
| A white surface lit by 1 % of sunlight, as Earthshine can | 111 | 9.5 |
| A V = 0 star, over the 4 pixels it covers | 56 | 4.4 |
| A V = 4 star, over 4 pixels | 1.4 | 0.13 |
So at the exposures a sunlit target needs, the brightest stars are just
detectable and the rest are lost in read noise. At 50 ms, a star tracker's
exposure, a V = 6 star reaches an SNR of 7.5.
Dark current hardly matters at these exposures: 20 electrons per second is
0.002 electrons in 0.1 ms. It matters for long ones: 1 electron in 50 ms,
100 in 5 s, with 10 electrons of shot noise.
**Reproducible randomness.** The noise comes from a counter-based generator:
a hash (Jarzynski and Olano's pcg4d) of the pixel and channel, the frame
number and a 64-bit seed (`RendererOptions::noise_seed`), turned into
uniform numbers, then into Gaussians (Box-Muller) or a Poisson count (by
inverting its distribution). Nothing is carried from one pixel or frame to
the next, so every value is computed independently on the GPU. The renderer
numbers its frames from 0 (`Frame::number`), and
`Renderer::set_frame_number` renders a frame again with exactly the same
noise. Different frame numbers or seeds give independent noise. The hash is
identical on every device, but the logarithms and cosines that shape it may
differ in the last bit, so the GPU and lavapipe agree statistically rather
than pixel for pixel.
**Noise-free images.** A detector with no read noise, no dark current, shot
noise off and no fixed pattern (`Detector::noiseless`) renders C1 without
noise, exactly as before noise existed, for ground truth and ML labels. The
renderer then compiles the noise out entirely. At C0 there is never noise.
Measured, identically on the GPU and lavapipe: flat fields at six levels
from 100 to 16,000 electrons, two frames each, give half the variance of
their difference within 2 % of the formula (0.982 to 1.008 of it, against a
sampling error of 0.6 %), and the photon transfer curve's slope is 1 / gain
to 0.1 %. A dark frame of 100 electrons reads 20.005 DN above the bias, with
its variance within 0.6 %. Counts below 50 match the Poisson probabilities
count by count. Consecutive frames, different seeds and neighboring pixels
are uncorrelated (|r| at most 0.005 over 196,608 values). 400 faint stars of
300 electrons each vary as predicted to 0.4 %, an SNR of 7.35. Noise costs nothing measurable on an
RTX 3060 (0.476 ms for the sensor pass at 1920 x 1080 with or without it,
1.89 ms at 3840 x 2160); on lavapipe it adds 1.5 ms at 1920 x 1080 and
9.6 ms at 3840 x 2160.
### Fixed pattern: PRNU and DSNU
Code: `fixed_pattern.rs` (`FixedPattern`); `sensor.wgsl` (`PATTERN`).
No two pixels of a real detector are quite alike, and the differences are
the same in every frame. That sets them apart from the noise above, which is
new each frame:
- **PRNU (photo-response nonuniformity).** Each pixel converts light to
electrons a little more or less efficiently than average, typically by
0.5 to 1.5 % RMS. Under even light the image is mottled by that much.
- **DSNU (dark signal nonuniformity).** Each pixel's dark current differs
from the average, by tens of percent. A few pixels, from defects in the
silicon, make tens to hundreds of times the average: hot pixels.
From camera tier C1 each value (pixel and channel) has a response factor,
which multiplies its light's electrons (stars' included), and a dark factor,
which multiplies the dark current, before the shot noise and the full well.
The example camera has 1 % PRNU, 10 % DSNU, and 1 value in 10,000 hot at 50
times the dark current (`Detector::prnu`, `dsnu`, `hot_pixel_fraction`,
`hot_pixel_multiple`). Because the dark factor scales the dark current, the
dark pattern grows with the exposure, as a real one does: at 0.1 ms the
example camera's hot pixels make 0.1 electrons, invisible; over 5 s, 5,000.
A camera made for long exposures is calibrated: its hot pixels are found in
dark frames and mapped out. The long-exposure example scenes (the Pleiades
and the GEO telescopes) leave 1 value in a million hot instead
(`hot_pixels = [1e-6, 50]` in a scene file's `[camera]`).
The pattern comes from `Detector::pattern_seed`, apart from the noise's
seed, so one camera keeps its pattern whatever the frame or noise: the same
hash as the noise, of the seed and the value's index, gives a Gaussian
response and dark factor and decides whether the value is hot. The shader
computes it for each value as it goes, so it takes no memory and no setup.
`FixedPattern::generate` computes the same pattern on the CPU, for tests or
to correct images with. A pattern measured from a real camera can be used
instead (`FixedPattern::from_calibration`, `RendererOptions::fixed_pattern`):
a flat, a frame of even light, and a dark, a frame with none, each the
average of many so their own noise is small, bias removed; each is divided
by its mean.
This is why real images are flat-fielded and dark-subtracted before use:
divided by a flat and with a dark frame taken off, which removes the
pattern and leaves the noise. It matters for detection in particular. A hot
pixel in a long exposure is a bright, sharp point that stays put from frame
to frame, just like a faint star or a distant target; only its
single-pixel sharpness (a real point source is spread by the PSF) or a dark
frame gives it away.
Measured, on the GPU and lavapipe: with noise off, a flat field and dark
frames at two exposures read the CPU's pattern to the rounding (within
0.505 DN of the unrounded value), the hot pixels are the same ones, and the
spreads are the requested 1 % and 10 % to 2 %. With noise on, at 10,000
electrons the pattern and the noise are equal parts of the spread, and two
frames or two noise seeds correlate by the predicted 0.50, while another
pattern seed shares nothing. Calibration maps are applied to the rounding.
The pattern costs nothing measurable on an RTX 3060; on lavapipe it adds
about 2.3 ms at 1920 x 1080 and 6.3 ms at 3840 x 2160. A loaded pattern
takes 8 bytes per value, 199 MB at 3840 x 2160 in RGB.
### Blooming
Code: `Blooming`; `sensor.wgsl` (`BLOOMING`: `collect_charge`,
`bloom_columns`, `read_out`).
A saturated pixel's extra charge does not vanish. Where it goes depends on
the sensor, and matters whenever something in or near the frame is far past
the full well: the Sun at the example camera's 0.1 ms is about 26,000 full
wells at the center of its disk, and a sunlit specular part can be hundreds.
- **CCDs** move charge down columns to read it out, and the channels that
hold it are only walled off from each other across the columns. Excess
charge spills along the column, up and down, filling each pixel it
reaches before passing on: the familiar vertical streak through the Sun
or a glint, which can cross a whole frame and hide a target in it.
- **CMOS sensors** read each pixel on its own, and most have an
anti-blooming drain that carries excess charge away. What escapes it
spills into the pixels around.
From camera tier C1, `Detector::blooming` chooses: `None` (the default: the
pixel clips and the excess is lost), `Column { efficiency }` or
`Neighbors { efficiency }`, where the efficiency is the fraction of the
excess that spreads and the rest drains. Along a column, half goes up and
half down; each pixel it reaches takes what it has room for and passes the
rest on, until it runs out or leaves the frame. Into the neighbors, each of
the 8 takes an eighth, and what overfills them drains.
How far a streak runs is the excess over the full well, times the
efficiency, divided among pixels that each hold one full well. For a bright
star only a pixel or two saturate, by a little, and the streak is short and
a pixel or two wide. The Sun is different. Its disk is about 85 pixels
across in the example camera, so the streak is 85 pixels wide, and at 0.4 ms
each pixel of it holds about 100,000 full wells: a column through the middle
of the disk has some 8 million full wells of excess. With no drain
(efficiency 1), half of that runs each way, enough to fill thousands of
frames' worth of column, so every pixel in those columns saturates and the
rest runs off the edges: a white band the width of the Sun, top to bottom
(`examples/scenes/sun-glare.toml`). That is what a CCD without an
anti-blooming drain records. A drain rated for 100 to 1,000 times
overexposure, as is typical, still leaves the Sun at about 100,000 times;
only an efficiency of about 5e-5 would shorten this streak to about 200
pixels each way.
Blooming happens to the charge, so it comes after the shot noise (and the
fixed pattern) and before the read noise and the ADC. With blooming on, the
sensor pass works in three steps rather than one: it writes each value's
charge to a buffer, spreads it, and reads it out. Along columns the spread
is a walk up and down each column, one at a time; the first step marks the
columns that hold any excess, and the walk skips the rest, which in most
frames is all of them. The CPU reference in the tests does the same walk in
f64.
Measured, on the GPU and lavapipe: a single overfull pixel on an even
background fills the pixels it should along its column to the electron,
charge that leaves the frame is lost, and nothing else changes; the
neighbors share equally; below the full well, or at efficiency 0, blooming
changes no value. The Sun at 2e-5 efficiency, in a 256 x 256 frame of the
example optics, streaks 22 pixels each way from the middle of its disk,
matching the CPU column by column to 0 DN. Column blooming costs 0.23 ms per
1920 x 1080 frame on an RTX 3060 with nothing past the full well and 0.75 ms
with the Sun in the frame (1.0 and 2.2 ms at 3840 x 2160); into the
neighbors, 0.24 ms (1.0 ms). On lavapipe, about 3 ms and 10 to 12 ms at
1920 x 1080.
## How shapes become pixels
Two techniques turn 3D shapes into an image. The renderer uses one for the
bodies and the other for the target.
- **Mesh.** A surface approximated by many small flat triangles that share
corners (vertices). A cube needs 12 triangles. A curved surface needs
many more, and it is still faceted up close. Each vertex stores a position
and usually a normal, the direction the surface faces there. STL and glTF
files are meshes, and so is the target spacecraft.
- **Rasterization.** The GPU's built-in way of drawing a mesh. Each
triangle's corners are projected into the image with the pinhole
equations above. The GPU then finds every pixel whose center falls inside
the projected triangle, and runs a small program (a fragment shader) for
each of those pixels to compute its value. It is fast because it only
touches the pixels each triangle covers, and GPUs have dedicated hardware
for it. It can only draw triangles.
- **Ray casting.** The reverse direction: for each pixel, follow the line
of sight from the camera through the pixel center and solve for where it
first meets a surface. For a sphere or an ellipsoid that is a quadratic
equation with an exact answer, so there are no triangles and no facets.
Every pixel does the work, whether or not it ends up seeing the body.
The bodies use ray casting, because their shapes have exact equations and
their edges are what limb-based navigation measures. The target uses
rasterization, because a spacecraft model has no equation, only its mesh.
The meshes are drawn first, with their depth. The bodies pass then casts its
rays and compares depths at every pixel, so the two sort against each other
correctly: a target in front of Earth covers it, and a target behind the limb
is hidden. Ray casting a mesh is also possible, by testing rays against
its triangles. A ray tracer, planned, does that, because following rays
gives exact shadows and light bouncing between surfaces. Rasterization can
only approximate those, for example with the shadow maps lighting L1 uses
(below).
### Pixel area
At camera tier C0, both techniques decide each pixel's value from one
point, the pixel's center. A real pixel collects light over its whole area,
about 5.5 um square on the detector. Where a pixel straddles an edge, say
half on a sunlit panel and half on dark space, the real pixel reads about
half the panel's value. At C0 the renderer reads either the full value or
none, depending on which side the center falls.
This shows in two ways. Edges are stair-stepped instead of graded. And
anything thinner than a pixel, like the booms of the example satellite seen
from 40 m, breaks into dashes, because only some pixel centers land on it.
A regular pattern finer than the pixels, like the cells of a solar array,
beats against the pixel grid and shows as curved bands (moiré). For an
algorithm that finds edges or measures brightness at sub-pixel precision,
the graded edge is the signal, so this matters for limb navigation and
pose estimation.
Camera tier C1 samples each pixel at several points and averages them
(supersampling), in a way that costs little where nothing is crossing the
pixel:
- **Meshes** use the GPU's multisampling. The rasterizer tests each
triangle against 4 points in every pixel, and keeps a depth for each, but
runs the fragment shader once per pixel. A pixel half covered by a panel
then gets the panel's value in 2 of its 4 samples, and reads half. The
covered fraction comes in steps of a quarter, and the light is computed
at one point inside the covered part (the centroid of the covered
samples), so a pixel on an edge is still shaded as one surface.
- **Bodies** are ray cast, so the bodies pass first decides whether a body's
edge could cross the pixel: whether the line of sight through its center
passes within one pixel's angle of the edge of Earth, the Moon or the Sun.
Most pixels do not, and cast one ray as at C0. A pixel on an edge casts a
grid of 4 x 4 rays spread over its area and averages them. A body only a
few pixels across needs a finer grid, because its brightness falls off
toward its edge and a coarse grid misjudges that: a 4 x 4 grid gets a
full Moon 0.45 pixels wide 45 % wrong in total light. So the grid has at
least 8 rays across the body's radius, up to 64 x 64 for one smaller than
a pixel, which keeps the total within 1 %. Only a handful of pixels pay
for that.
- **Both together.** Where a target and a body share a pixel, each of the
grid's rays takes the mesh sample nearest to it, and keeps the target if
that sample is nearer than the body along the ray. So a target seen just
past Earth's limb takes its share of the pixel, and Earth keeps the rest.
A grid of 4 x 4 is still a coarse estimate of an area. For an edge nearly
aligned with the pixel grid, its 4 columns of rays resolve the covered
fraction in quarters, so a pixel on the limb can be off by up to an eighth
of the difference across the edge (measured 0.10 on Earth's limb from low
orbit, against up to a half at C0). Patterns that stagger the rays fix
aligned edges but do worse at other angles; the regular grid had the
smallest error on average over all angles. At C1 the point spread function
(below) then spreads every edge over a few pixels, and C1 also renders each
pixel as 2 x 2 finer ones by default, each sampled this way, which divides
these errors further.
At one rendered pixel per image pixel, C1 costs 0.4 to 0.5 ms more per
1920 x 1080 frame on an RTX 3060, and 1.2 to 1.4 ms at 3840 x 2160, measured on Landsat 8, ICESat-2 and the moonrise
scene. On the CPU through lavapipe it costs much more, 2.6 to 8 times the
mesh pass at C0 (ICESat-2's 752,000 triangles go from 74 to 580 ms per
frame), because lavapipe appears to run the fragment shader for every
sample rather than every pixel. The 4 samples of radiance and depth also
take 80 bytes per pixel of GPU memory: 166 MB at 1920 x 1080 and 660 MB at
3840 x 2160.
## Blur: the point spread function
Code: `camera.rs` (`Psf`); `gpu/blur.rs`, `shaders/blur.wgsl`; from camera
C2, `psf.rs`, `gpu/convolve.rs`, `gpu/fft.rs`, `shaders/convolve.wgsl`,
`shaders/fft.wgsl`.
No camera focuses a point to a point. Light from one point of the scene
lands on the detector as a small spot, and the shape of that spot is the
point spread function (PSF). Several things widen it:
- **Diffraction.** Light passing through a round aperture spreads into a
pattern of rings (the Airy pattern) whose size depends only on the
wavelength and the f-number. At f/4 and 550 nm the first dark ring is
2.7 um from the center, half a pixel of the example camera.
- **Aberrations and defocus.** Real lenses bring different parts of the
aperture, different wavelengths and different parts of the field to
slightly different points.
- **The detector.** Electrons freed deep in a pixel can drift into its
neighbors before they are collected (charge diffusion).
- **Jitter.** The camera shakes a little during the exposure.
Each of these blurs the image of the one before it, and many such blurs
together come out close to a Gaussian, a bell curve in 2D. So a common model
of a camera's PSF is a Gaussian with one width, its standard deviation
sigma, measured in pixels. A spot with sigma 0.5 px is 1.2 px wide at half
its peak, and a point centered on a pixel puts about 47 % of its light in
that pixel and the rest in its neighbors; at 0.7 px, 28 %. Longer
wavelengths diffract more, so sigma is set per channel and usually grows
toward the red.
How wide the spot should be depends on the camera. Published measurements,
converted to a Gaussian's sigma with the pixel's own area taken out:
- Earth-observation imagers are built sharp: about 0.2 px for Landsat 8's
OLI, the sharpest, and 0.35 to 0.55 px for WorldView-3, Pléiades Neo and
Sentinel-2's specification. The Lunar Reconnaissance Orbiter's cameras are
specified in the same range.
- Navigation and small-satellite cameras are softer: about 0.6 to 0.9 px
for DART's DRACO camera in flight, and 0.55 to 0.7 px for Satellogic's
imagers.
- Star trackers are defocused so each star covers several pixels. The one
measured in flight here (on Lunar Flashlight) has a sharp core and wide
wings that no single Gaussian matches: its spot is as wide at half its
peak as a 0.4 px Gaussian, and spreads its light as widely as a 1.25 px
one.
The imagers' numbers are for the whole system, including the atmosphere
and image processing, so the optics alone are somewhat sharper. The example
camera uses 0.5 px in every channel, a sharp camera, not derived from its
optics; diffraction alone at f/4 would give about 0.17 px.
Without blur, a star or a distant spacecraft lands in one pixel or in none,
and every edge is as sharp as the pixel grid allows. Algorithms that find
the center of a star or a target to a fraction of a pixel (centroiding)
need the spot spread over more than one pixel, so they cannot be tested on
an unblurred image.
The blur is applied to radiance on the focal plane, before light becomes
electrons. That order matters for anything bright enough to saturate. The
Sun's image is tens of thousands of times brighter than a full well.
Blurred first, its spot spreads that light into the neighboring pixels,
which also fill, so the saturated region is wider than the Sun's disk, as
in a real camera. Clipped first and then blurred, the saturated region
would shrink back inside the disk and its edge would fade over a pixel or
two, as if the Sun were barely brighter than the full well. The blur's
weights sum to 1, so it moves light without adding or removing any.
### Blurring before the pixels average
The blur has to act on the light where it lands, before each pixel averages
it. Blurring an image of pixel averages afterward is not the same once the
spot is about as small as a pixel: a point of light that fell anywhere in a
pixel has become that pixel's value, and where in the pixel it fell is
gone. Its blurred spot should lean toward wherever it was, and no blur of
the averages can tell which way. Measured with sigma 0.4 px, points of
light blurred after averaging put up to 51 % of their light in the wrong
pixel, and edges were off by up to 12 % of their step.
So from camera tier C1 the renderer draws the scene finer than the pixels,
n x n rendered pixels to each image pixel, blurs that by a Gaussian PSF set
in `CameraConfig::psf`, and only then averages each n x n block into one
pixel. Each rendered pixel still collects light over its own area the way
the previous section describes, so the finer image is itself made of
small-pixel averages, and the same error remains at a smaller scale:
| Rendered pixels per pixel | Points, worst pixel | Edges, worst pixel |
|---|---|---|
| 1 x 1 | 51 % of the light | 12 % of the step |
| 2 x 2 | 19 % | 2.2 % |
| 4 x 4 | 4.5 % | 0.5 % |
(sigma 0.4 px, points at 16 and edges at 129 positions within a pixel; the
worst edges lie on a pixel boundary.) C1 renders 2 x 2 by default;
`RendererOptions::supersampling` sets 1 to 4. Point sources need better
than that, so stars are drawn with the blurred spot computed exactly over
each pixel instead (see Stars).
A 2D Gaussian is the product of a 1D Gaussian across and one down, and
averaging an n x n block is likewise an average across and one down. So the
blur runs as two passes, one along rows and one along columns, each taking
a weighted sum of the rendered pixels out to 4 sigma on either side
(beyond which a Gaussian holds less than 1e-4 of its light) and averaging
n of them into one pixel at once. Light from just outside the frame also
blurs into it, so the renderer draws a margin of that width around the
image, and the blur reads it and drops it. The weights are the Gaussian at
each rendered pixel's offset, which keeps a spot's width (its variance)
within 1 % of sigma squared once sigma is 0.65 rendered pixels or more:
0.33 px in image pixels at 2 x 2.
Rendering finer costs about n^2 times the drawing, and the blur grows with
it. To keep that down, the meshes are covered at 2 points per rendered pixel
rather than 4 when rendering finer: at 2 x 2 that still puts samples at 4
distinct offsets across and down each image pixel, as 4 samples did at
1 x 1, so mesh edges still grade in quarters or better. (A device that
cannot render 2 samples, such as lavapipe, uses 4.) GPU time per frame at
C1, from the first pass to the last, on an RTX 3060 (Landsat 8, ICESat-2
and the moonrise scene, at lighting L1, sigma 0.5 px):
| Rendered pixels per pixel | 1920 x 1080 | 3840 x 2160 |
|---|---|---|
| 1 x 1 | 1.9 to 2.5 ms | 6.1 to 6.9 ms |
| 2 x 2 (default) | 3.7 to 4.4 ms | 12.6 to 14.6 ms |
| 3 x 3 | 6.7 to 7.9 ms | 34 to 38 ms |
| 4 x 4 | 10.9 to 13.0 ms | 59 to 66 ms |
Of the 3.7 to 4.4 ms at 2 x 2, the bodies take 1.3 to 1.4 ms, the meshes
0.7 to 1.1 ms and the blur 1.0 ms. With 2 samples instead of 4 the meshes
and bodies took 0.3 to 0.6 ms less in all; with 1 sample, 0.7 to 1.0 ms
less, but a mesh edge then grades only in halves at 2 x 2, coarser than at
1 x 1. The rendered images take 60 bytes per rendered pixel (100 at 1 x 1,
with 4 samples), and the blur 16 bytes per image pixel per rendered row:
about 0.27, 0.60, 1.25 and 2.15 GB at 1920 x 1080 for 1 x 1 to 4 x 4, and
four times that at 3840 x 2160. The renderer reports running out of memory
as an error. On the CPU through lavapipe, 2 x 2 adds 9 to 13 ms to
1920 x 1080 frames of 116 to 680 ms, and 3 x 3 at 3840 x 2160 runs out of
memory.
### Diffraction and the PSF's wings (camera C2)
A Gaussian falls to nothing within a few sigma. A real camera's spot does
not: diffraction's rings fade only as the cube of the distance, and dust and
rough lens surfaces scatter a little light much farther. Next to most of a
scene that faint halo, the PSF's wings, is invisible. Next to the Sun it is
not. In the sun-glare example scene each pixel of the Sun's disk collects
about 5e8 electrons in its 0.4 ms exposure, 26,000 full wells, over about
5,700 pixels. Diffraction at f/4 spreads a fraction of about
λN / (π^3 r^3) of a point's light into each pixel r pixels away (λN, the
wavelength times the f-number, is 0.39 px for green light in the example
camera), so 500 px from the Sun each pixel still gets about 300 electrons
from it, and 1000 px away about 40: a glow over the whole frame that raises
the background, can hide a faint target, and pulls a limb fit toward the
bright side.
Camera tier C2 models this. `Psf::Airy { obstruction }` is diffraction by
the camera's aperture: the Airy pattern, set by the f-number, with an
optional central obstruction such as a reflecting telescope's secondary
mirror (which moves light from the core into the rings). The pattern scales
with the wavelength, so a channel's spot is the pattern averaged over the
channel's band, each wavelength weighted by the photons sunlight gives it
there; across an 80 nm band the outer rings blur into a smooth halo.
`Psf::Measured { image, pixel_scale }` takes a spot measured in a lab or in
flight, the same over the whole field, one image per channel or one for
all. Measured: a 2 nm wide channel's first dark ring falls within 1 % of
1.22 λN, where the textbook puts it.
A kernel reaching hundreds of pixels is far too slow to apply pixel by
pixel. C2 instead uses the convolution theorem: transform the image into
its frequencies with a fast Fourier transform (FFT), multiply by the
kernel's frequencies, computed once at setup, and transform back. The cost
no longer depends on the kernel's size, only on the image's. The transform
runs on the GPU in compute shaders, sized to powers of two up to 4096 on a
side, with two channels sharing one complex transform, so three channels
take four transforms per frame. C2 applies every PSF this way, Gaussians
too, and draws stars into the focal plane before the transform, so they get
the same wings; each star's light is shared among its four nearest rendered
pixels, which keeps its center exact but can widen its spot by up to 0.25 px
RMS at 2 x 2.
As at C1, the transform acts on the scene rendered finer than the pixels,
before they average it. The kernel is the PSF integrated over a rendered
pixel twice, once for the pixel the light leaves and once for the one it
lands in, so a Gaussian at C2 differs slightly from C1's, which samples the
Gaussian at the rendered pixels' centers (by up to 0.34 % of the brightest
pixel, measured on the Moon). The transform treats the image as repeating,
so the renderer draws a margin around it as wide as the kernel reaches, and
the grid holds the image and that margin. The reach is therefore set by the
grid: at least `RendererOptions::psf_reach` (16 px by default), and as far
as the smallest grid that holds it allows. Light the kernel would carry
farther is added back to the spot instead, so none is lost, but the glow
stops at the reach, and with the Sun in the frame its edge shows in a
brightened preview.
| Image, rendered pixels per pixel | Grid | Reach | RTX 3060 | lavapipe |
|---|---|---|---|---|
| 1920 x 1080, 2 x 2 | 4096 x 4096 | 64 px | 30 ms | 560 ms |
| 1920 x 1080, 1 x 1 | 2048 x 2048 | 64 px | 5.8 ms | 213 ms |
| 1920 x 1080, 1 x 1, reach 1000 | 4096 x 4096 | 1088 px | 29 ms | 556 ms |
| 3840 x 2160, 1 x 1 | 4096 x 4096 | 128 px | 30 ms | 623 ms |
(The convolution pass alone, in the sun-glare scene at lighting L1. The
whole frame took 35 ms at 1920 x 1080 and 2 x 2, against 5.9 ms at C1.) A
grid of 4096 x 4096 takes 640 MB on the device. 3840 x 2160 at 2 x 2 would
need a grid 8192 across and is refused; 1 x 1 fits.
The transforms work in 32-bit floating point, which keeps about seven
significant digits, and their rounding error spreads over the whole grid in
proportion to its brightest values. With the Sun in the frame that leaves a
floor of about 10 electrons RMS, up to 165, across the frame (measured in
the sun-glare scene with a noiseless detector, where C1 reads exactly 0),
above the example detector's 5 electrons of read noise. A sunlit target at
half the full well leaves a floor near 1e-3 electrons, far below any noise.
Computing the brightest sources' wings separately, outside the transform,
would remove the Sun's floor; that is not done yet.
## Earth, Moon and Sun
Code: `bodies.rs`, `gpu/bodies.rs`, `shaders/bodies.wgsl`.
The three bodies are not drawn as triangle meshes; they are ray cast. For
every pixel, the shader sends a ray from the camera through the pixel's
center and solves for where it meets each body's exact shape. The nearest hit wins. This
gives an exact edge at any zoom, at the cost of a few arithmetic operations
per pixel. A mesh would show its facets along the limb, the edge of the
disk, which is the feature horizon-based navigation measures.
- **Earth** is the WGS84 ellipsoid: 6378.137 km equatorial radius,
6356.752 km polar. The 21 km difference matters at fine pixel scales. A
camera with 5.5 urad pixels at 200,000 km sees the north limb 19 pixels
away from where a spherical Earth would put it. A test checks the limb
against the exact tangent to the ellipsoid to within a pixel.
- **The Moon** is a sphere of 1737.4 km radius. From Earth's distance it
is about 0.52 degrees across, 82 pixels in the example camera.
- **The Sun** is a sphere of 695,700 km radius: 0.53 degrees across, about
85 pixels in the example camera. Averaged over its disk it has the solar
radiance from the folding section above; from lighting L1 its center is
brighter than its edge (see "The Sun's disk" below). It is drawn only
where no nearer body covers it, so Earth eclipses it naturally.
### Shading
Earth and the Moon are Lambertian, at lighting L0 each with one reflectance
spectrum over the whole body, set by `Appearance`. The Moon's is a flat 0.12
(from L1 it comes from a map; see "The Moon's surface"). Earth's
default is a stand-in shaped like cloud-free ocean seen from space: about
0.3 in the blue falling to 0.07 in the red, then back up in the near
infrared. Most of that blue is sunlight scattered by the atmosphere above
the ocean, the same scattering that makes the sky blue; the ocean itself is
dark. In the example camera it gives channel albedos of 0.09, 0.15 and 0.26
(red, green, blue), so Earth renders blue. Its overall brightness is below
the real Earth's average of about 0.3, which includes bright clouds.
Each point's radiance is rho cos(i) S_k / (pi d^2), with i the angle between
the surface normal and the Sun. The night side has i past 90 degrees and is
black. The Sun's distance d is taken from each body's center, since the
difference across a body is a few parts in 10^5.
The total light a sphere sends a far-away camera has a closed form. A
Lambertian sphere of albedo rho and radius R, at distance D, delivers an
irradiance of (2/3) rho E (R/D)^2 Phi(alpha), where alpha is the phase angle
(Sun-sphere-camera) and Phi(alpha) = (sin alpha + (pi - alpha) cos alpha) / pi.
It falls from 1 at full phase to 0.61 at 60 degrees and 0.11 at 120 degrees.
Up close it is not exact: the near side of the sphere is nearer the camera
than its center, so it counts for more. At 40,000 km from the Moon (R/D =
0.043), the real total is 3 % above the formula. The renderer's image agrees
with the exact surface integral to 0.03 % or better (`tests/gpu/analytic.rs`).
This gets the overall brightness and the phase (how much of the disk is
lit) right. It misses some things that are visible in real images:
- Earth's oceans, land and clouds differ in brightness by a factor of
several. From lighting L1, Earth's surface comes from an image and its
clouds from another (next sections).
- The full Moon looks almost uniformly bright out to its edge. A Lambertian
sphere instead darkens toward the edge, because the light there arrives
and leaves at grazing angles. Lunar soil scatters light back toward its
source much more strongly than Lambert predicts, which is also why it
brightens so steeply toward full Moon. At lighting L1 the Moon's total
brightness is scaled to its measured phase curve ("The Moon's surface"),
but its edge still darkens; a dedicated lunar reflectance model would fix
that. The Moon's maria (the dark plains of
old basalt lava, the "seas" seen with the naked eye) and its bright
highlands also differ in albedo by about a factor of two, which from
lighting L1 the Moon's map shows.
- Earth's atmosphere softens the limb and adds a thin glowing band around it.
At lighting L0 Earth's edge is a sharp line; from L1 the atmosphere is drawn
("Earth's atmosphere").
### Earth's surface
Code: `earth.rs`, `shaders/bodies.wgsl`; data from `scripts/fetch-earth.sh`.
At lighting tier L1 (`RendererOptions::fidelity`, with the image in
`RendererOptions::earth_texture`; `--lighting L1` on the command line), Earth's
reflectance comes from an image of the whole surface instead of one spectrum. The image is NASA's
Blue Marble Next Generation: a cloud-free mosaic of the 2004 MODIS satellite
record, one image per month, so snow cover and vegetation follow the season.
The CLI picks the month from the scene's epoch.
The image is equirectangular: each column is a longitude and each row a
latitude. For every pixel that sees Earth, the shader finds the surface
point's geodetic latitude and longitude from the ellipsoid's normal there, in
Earth-fixed axes, and reads the image at that place. Earth's orientation from
the scene turns the continents with the epoch.
The image comes in two sizes:
- The global map, always there: NASA's 21600 x 10800 image (1 arcminute,
1.85 km per pixel at the equator), resized to 16384 x 8192 (2.45 km), the
largest texture lavapipe allows.
- A sharp map, optional: NASA's 86400 x 43200 image (15 arcseconds, 460 m),
made once per month by `scripts/fetch-earth.sh --sharp` (below).
Whether a map is fine or coarse depends on the view:
- From low orbit looking straight down, a camera pixel of the example camera
covers 77 m of ground. A texel of the global map spans 32 image pixels and
looks blurred; one of the sharp map, 6.
- From geostationary orbit, a camera pixel covers about 4 km, so each image
pixel averages about 2 x 2 texels of the global map.
For the second case the texture is stored with mipmaps: copies at half, a
quarter, an eighth of the size, and so on. The GPU reads from the copy whose
pixels best match the image pixel, so a far view shows the true average color
of a region rather than a shimmering sample of individual texture pixels.
The sharp map is too large for one texture on either device (lavapipe's limit
is 16384 on a side) and for one allocation on lavapipe (2 GB), and 30 GB as
16-bit floating point. So it is cut into 8 x 4 square tiles of 10800 texels,
each stored with a border of 4 texels copied from its neighbors, so that
blending near a tile's edge reads the right texels without a second tile, and
compressed to BC6H: blocks of 4 x 4 texels in 16 bytes, which the GPU decodes
as it reads them. BC6H keeps linear values with about the precision of 16-bit
floating point; on Blue Marble's July map around Sinai, 99 % of pixels are
within 1 to 2.5 % of the uncompressed value and the mean within 0.1 %. Three
mipmap levels are stored (460 m, 930 m, 1.85 km), 4.9 GB in all, in four
texture arrays of 8 tiles. A pixel reads the sharp map where it covers less
ground than the coarsest level's texels, the global map beyond, and the two
blended across half a level so the change does not show.
Its cost on the RTX 3060: 0.1 to 0.2 ms more in the bodies pass where it is
read, and 1.6 s more setup to upload it. On lavapipe, which decodes BC6H in
software, about 100 ms more in the bodies pass where it is read and 5 s more
setup, with 8 GB of memory at its peak. From geostationary orbit it is not
read, and costs nothing per frame.
The GPU and lavapipe choose mipmap levels and blend texels slightly
differently, so a textured Earth differs between them by about 1 DN on
average at lighting L1 and camera C1 (0.8 DN with the global map alone,
1.3 DN with the sharp map; 2.7 DN before the atmosphere, which lowers the
contrast). The tests of the maps compare each device with itself.
Two details matter for brightness:
- Averaging has to happen on reflectance, not on the encoded values image files
store (sRGB, which compresses the darks). Averaging encoded black and white
gives a reflectance of 21 %, not 50 %. So the texture is decoded to linear
reflectance once at load and kept in 16-bit floating point, and both the
mipmaps and the GPU's blending between neighboring texture pixels work on
those values. A coastline seen up close blends correctly from sea to land.
- Blue Marble is surface reflectance with no atmosphere, and open ocean is
dark, a few percent. From orbit the ocean looks bright blue because the air
above it scatters sunlight toward the camera. The renderer adds that air
itself, next.
The texture's red, green and blue are reflectances in the same three bands a
glTF base color uses (see the target model section), folded into the camera's
channels once at setup as a small matrix, so any channel layout works.
Blue Marble is a visual product built from calibrated MODIS reflectances, not a
calibrated reflectance product itself, so its values are close but not exact
reflectances. A calibrated alternative (MODIS's albedo product) is a possible
later refinement.
### Earth's atmosphere
Code: `atmosphere.rs`, `shaders/atmosphere.wgsl`, `shaders/bodies.wgsl`.
At lighting L1 Earth is seen through its atmosphere
(`Appearance::earth_atmosphere`, on by default). Air does three things to
light passing through it:
- **Molecules scatter it**, much more at short wavelengths: Rayleigh
scattering grows as 1 / wavelength^4, so blue light at 450 nm scatters 3.6
times as much as red at 620 nm. Straight up from the ground, air scatters
about 10 % of green light (an optical depth of 0.11 at 550 nm) and 21 % of
blue (0.24 at 450 nm). That is why the sky is blue and the setting Sun red.
- **Aerosols** (dust, sea salt, smoke droplets) scatter colors far more
evenly than air does, so they make haze white rather than blue, and mostly
forward, within a few tens of degrees of the light's direction. They sit
low: half of them in the lowest 0.8 km. Straight up they scatter or absorb
about as much light as the air does, an optical depth of 0.12 at 550 nm on
average over Earth, and far more over deserts and fires.
- **Ozone absorbs** light in a broad band centered near 600 nm (the Chappuis
band). From space it matters mainly at the limb, where the line of sight
crosses the ozone layer edge on, and tints the glow there.
What leaves the ground is reduced by the light scattered or absorbed out of
it on the way to the camera: the **transmittance**, between 0 and 1. And the
air along the line of sight sends sunlight it scatters toward the camera: the
**path radiance**, which adds to whatever is behind it. Both grow with the
amount of air the line of sight crosses, so the ground looks hazier and bluer
toward Earth's edge, where the line of sight is long and slanting. Past the
edge the line of sight crosses only air, which glows: the **limb**. There the
air is thin enough that the glow is in proportion to the density at the
lowest point of the line of sight, so it fades by a factor of e (2.7) for
every 8 km of height.
The ground itself is lit by the sunlight that reaches it, which the air has
also dimmed (more at a low Sun, and more in blue, so ground near sunset is lit
redder), and by **skylight**: the light the whole sky scatters down, about a
tenth of the Sun's in blue with the Sun overhead and less in red. Past the
terminator the sky still lights the ground for a while: twilight.
Light can also scatter more than once, from air to air or off the ground and
back. With the Sun high that adds a few tens of percent to the sky's
brightness.
Adding up the scattered light along every line of sight for every pixel would
be far too slow. Instead the renderer precomputes tables once, at setup, with
the method of Bruneton and Neyret (2008), in the form of Bruneton's 2017
implementation and with its atmosphere: what a point at a given height sees
looking in a given direction, with the Sun at a given angle, is a function of
four numbers (height, view angle from the zenith, Sun angle from the zenith,
and the angle between the view and the Sun), so it fits a table. Three tables:
- the scattered light (4D, stored as a 3D texture of 256 x 128 x 16 texels,
Rayleigh and aerosols separately);
- the transmittance from any point to the top of the atmosphere (256 x 64),
from which the transmittance between any two points follows;
- the skylight on the ground (64 x 16).
Each pixel that sees Earth then reads them about a dozen times: the
transmittance to the ground and the light scattered along the way (the light
scattered along the whole line of sight, less what lies beyond the ground),
and the sunlight and skylight on the ground there.
Light scattered more than once is approximated as Hillaire (2020) does:
assume the light arriving at a point after one scattering comes equally from
every direction and is the same at nearby points. Then all the higher orders
together follow from a small table per height and Sun angle, instead of being
computed order by order.
The air is Bruneton's, with more aerosols: Rayleigh scattering falling with
an 8 km scale height; aerosols with a 1.2 km scale height, an optical depth of
0.12 at 550 nm (the global annual mean of Kinne et al.'s 2013 climatology)
falling as wavelength^-1.3, 90 % of it scattering, with Bruneton's forward
phase function (asymmetry 0.8); and 300 Dobson units of ozone between 10 and
40 km. Bruneton's aerosol depth of 0.0053 is air cleaner than almost anywhere
on Earth: with it the ground looked bluer and sharper toward the limb than it
should. The top of the atmosphere is set
100 km up. Earth is the sphere of its equatorial radius, in axes stretched
along the polar axis, where its ellipsoid is that sphere (heights are off by
at most 0.3 % near the poles).
The tables are computed every 20 nm and folded into the camera's channels
with sunlight and each channel's QE, like every other spectrum
("Folding spectra into per-channel constants"). The scattered light folds
exactly. Transmittance does not: a channel whose band spans very different
transmittances gets their average weighted by sunlight, which is wrong when
the light passing through is a different color from sunlight. It matters most
for blue light along long, grazing paths. Setup takes 0.25 s on the 24-core
desktop CPU these numbers come from, and 1.1 s on four of its cores.
Tests: the tables against direct integrals, in `atmosphere.rs` (the amount of
air from the ground straight up within 1e-4 of the scale height, Rayleigh
scattered light within 3.4e-4, aerosols within 0.31 %); the shaders against
the tables, on the GPU and lavapipe (`tests/gpu/atmosphere.rs`): the ground
seen at 60 degrees from its zenith and the limb at 40 and 50 km, within
2.5e-4. The limb fades by 3.38 over those 10 km, against 3.49 for the density
alone; the air between takes the rest.
Checked against a real satellite: GOES-16's full disk of 2024-07-01 at
15:00 UTC, in its blue (470 nm) and red (640 nm) bands, over 640,000 cloud-free
ocean pixels (NOAA's cloud mask), away from the Sun's glint. Over ocean,
which is dark, most of the light the satellite measures comes from the air,
and GOES sees the sea from straight above out to its limb: the same slanting
paths through the air as a camera in low orbit looking toward the horizon.
The tables, evaluated at each pixel's angles with Blue Marble's ocean color
and channels shaped like GOES's bands, against the measured reflectance
(the median in each band of viewing angle from the zenith):
| Viewing angle | GOES blue | Model | Clean air | GOES red | Model | Clean air |
|---|---|---|---|---|---|---|
| 20-40 degrees | 0.097 | 0.090 | 0.077 | 0.029 | 0.030 | 0.023 |
| 55-65 | 0.146 | 0.145 | 0.123 | 0.044 | 0.051 | 0.037 |
| 70-75 | 0.205 | 0.191 | 0.169 | 0.076 | 0.071 | 0.056 |
| 78-81 | 0.249 | 0.239 | 0.219 | 0.093 | 0.095 | 0.078 |
| 81-84 | 0.267 | 0.266 | 0.246 | 0.104 | 0.110 | 0.092 |
The model is within 7 % in blue (low: Blue Marble's ocean is darker in blue
than real open ocean) and 15 % in red out to 84 degrees, and the haze grows
toward the limb as measured. With Bruneton's clean air ("Clean air") it is 8
to 29 % too dark. NOAA's own retrieval put the aerosols' optical depth over
these pixels at 0.13 (median), close to the 0.12 the model assumes.
On the RTX 3060 the atmosphere adds 0.8 to 1.0 ms to the bodies pass at
1920 x 1080 with camera C1's 2 x 2 supersampling, and 14 to 27 ms on lavapipe.
Limitations:
- **The camera must be above the atmosphere**, over 100 km up; below that a
frame is refused.
- **Sunlight reaching the target**, and the Sun and Moon seen past the limb,
pass through the atmosphere as "Sunset through the atmosphere" describes;
targets, stars and Earth's own surface seen past the limb do not.
- **Clouds** are a separate layer, the next section.
- **The Moon's shadow** in a solar eclipse dims the air along the line of
sight as it dims the ground point behind it.
- **Light from Earth onto targets** (Earthshine) includes the atmosphere
over the ground, but not the glow beyond Earth's edge.
### Clouds
Code: `clouds.rs`, `shaders/clouds.wgsl`, `shaders/cloudy.wgsl`,
`shaders/earthshine.wgsl`. From lighting L1, with the atmosphere.
Blue Marble's surface is a cloud-free mosaic, but at any moment about two
thirds of Earth is under cloud, and clouds are the brightest thing on it.
Without them Earth looks too dark and too clear, and Earthshine on a target is
too low.
The renderer draws clouds as a thin layer 5 km above the ground. Real cloud
tops run from about 1 to 15 km; one height stands for them all. The layer's map
is the cloud layer of NASA's original Blue Marble (2002): Terra MODIS clouds
of 2001 July 29, with thermal infrared over the poles, 1 km per pixel, resized
to 16384 x 8192 like the Earth maps (`scripts/fetch-clouds.sh`). The same
clouds show whatever the date.
Each pixel's value is read as the cloud's diffuse reflectance there: what it
reflects of light arriving from every direction alike, from 0 for clear sky to 1.
Clouds absorb almost no visible light, so what a cloud does not reflect it passes
on. A thick cloud is bright and hides the ground. A thin one is dim and lets the
ground show through. The value gives the cloud's optical thickness tau (how many
times light would be scattered crossing it straight) by the two-stream result of
Lacis and Hansen (1974):
R = 0.13 tau / (1 + 0.13 tau)
The 0.13 is sqrt(3) / 2 times (1 - g), with g = 0.85 the cloud droplets' strong
forward scattering. So tau = 5 reflects 39 %, tau = 20 reflects 72 %, and tau =
50 reflects 87 %. Over Earth's area the map's mean reflectance is 0.21, and 62 %
of the area has more than 0.05. The map stores reflectance, so its mip levels
average reflectance, which keeps a far view of Earth as bright as a near one.
A cloud reflects more of a beam the more slanting it is: light entering at a low
angle scatters out of the top before it gets deep. A Monte Carlo of a cloud slab
(no absorption, the droplets' scattering as a Henyey-Greenstein function of g =
0.85, black below, 200,000 photons a case) gives:
| tau | Sun overhead | 60 degrees from it | 78 degrees | Diffuse (above) |
|---|---|---|---|---|
| 1 | 0.043 | 0.165 | 0.418 | 0.115 |
| 3 | 0.142 | 0.359 | 0.568 | 0.281 |
| 10 | 0.422 | 0.603 | 0.732 | 0.565 |
| 30 | 0.713 | 0.806 | 0.868 | 0.796 |
| 100 | 0.897 | 0.929 | 0.952 | 0.929 |
So the sunlight a cloud reflects, and the shadow it casts, come from tau and the
Sun's angle by the delta-Eddington method (Joseph, Wiscombe and Weinman 1976),
in closed form. It takes the forward peak of the droplets' scattering, a
fraction g^2 of it, as no scattering at all, and the rest as two streams of
light, up and down. It is within 0.08 of every case of the Monte Carlo, worst for
thin cloud under a low Sun, where the diffuse value is off by up to 0.30.
Averaged over the Sun's angles it is within 0.02 of the Monte Carlo's average,
as the diffuse value is. Sky light, which arrives from every direction, takes
the diffuse value.
For each pixel that sees the ground, the shader:
1. finds where the line of sight crosses the layer, and reads the cloud there.
In an oblique view that is not the cloud straight above the ground point,
so clouds show parallax against the ground;
2. finds where the sunlight reaching the ground point crossed the layer, and
dims the ground's sunlight by what that cloud passes on at the Sun's angle,
and its sky light by what it passes on of diffuse light. That is the cloud's
shadow, offset from the cloud away from the Sun: 5 km times the cotangent of
the Sun's elevation, 29 km with the Sun 10 degrees up;
3. adds the cloud's own light: the sunlight at 5 km, which passes through less
air than at the ground, times the cloud's reflectance at the Sun's angle, and
the sky light there times its diffuse reflectance;
4. adds what is below the cloud, the ground and the air between it and the
layer, times what the cloud passes on at the line of sight's angle. Then it
applies the air above the layer: its glow added, and its transmittance on
what is under it.
So high clouds are less hazed than the ground, the way they look from orbit.
The ray's entry into the atmosphere is looked up once and shared by the cloud
and the ground below it.
The Earthshine pass samples the same layer, so a target over cloud receives
more light. A plate 700 km up facing straight down, the Sun overhead, under
clouds of reflectance 0.5 everywhere, receives 2.0, 1.9 and 1.75 times its
cloud-free Earthshine in red, green and blue.
How bright Earth is as a whole tests the reading of the map's values as
reflectance. With clouds, rendered at phase angles from 0 to 180 degrees and
integrated over them, Earth reflects 0.26 to 0.28 of the sunlight in green
(the Bond albedo), from four sides in July. Without clouds it reflects 0.13 to
0.16. Measured, Earth's Bond albedo is about 0.30 (CERES, over the whole solar
spectrum). With the diffuse reflectance alone in place of delta-Eddington it was
0.27 to 0.29. The geometric albedo, the full disk's brightness seen from the
Sun's direction, is 0.17 to 0.19 in green here, against 0.43 for the real
Earth, because clouds and the ocean send back toward the Sun more light than a
Lambertian surface does.
Tests (`tests/gpu/clouds.rs`), on the GPU and lavapipe:
- A uniform layer over gray ground, for four reflectances from clear to opaque
and the Sun 30 and 70 degrees from the zenith, against the atmosphere's tables
and delta-Eddington in f64 at the cloud's height and the ground's (within
3.3e-4).
- A clear map against no map. Lavapipe agrees to 2.4e-7. The RTX 3060 differs
by up to 6.7e-3 in 0.3 % of pixels, 1.8e-6 on average: its texture filtering
steps by 1/256 between texels, and a coordinate that differs in its last bit
between the two pipelines can land on the next step.
- One opaque cloud with the Sun 10 degrees up, seen from 1000 km straight
above: the cloud and its shadow 28.7 km to the west land within 0.41 and
0.12 px of where they project, 261 px apart.
- The Earthshine above, against the ratio for Earth seen straight down (within
10 %; 3 to 8 % less, most in blue, as the slant views around the nadir see the
clouds pass on less of what is below them, and more of their light is the
air's).
Unit tests in `clouds.rs` check the reflectance formulas, delta-Eddington against
the Monte Carlo above, and the mip averaging.
The command-line tests check that a missing map names `scripts/fetch-clouds.sh`,
and that `[appearance] clouds = false` renders without one.
Cost: the map takes 360 MB of GPU memory. It is compiled into the bodies pass
only once set, so frames without clouds cost what they did. With Earth filling
1920 x 1080 at camera C1, the bodies pass on the RTX 3060 goes from 2.4 to 3.1 ms
over mostly clear Egypt, and from 2.5 to 3.7 ms over the cloudy Pacific,
7.2 ms a frame end to end. On lavapipe, the Pacific view goes from 49 to 64 ms.
Delta-Eddington's division and three exponentials per pixel add 0.03 ms to that
on the RTX 3060 and nothing measurable on lavapipe. Earthshine: unchanged
(0.045 ms).
Limitations:
- **One height**, and no thickness. Clouds stand neither above the limb nor
above one another, and cast no shadows on each other. Volumetric clouds are
planned for lighting L2.
- **Gray, and not absorbing.** The light a cloud reflects leaves it evenly in
every direction (Lambertian), though how much it reflects follows the Sun's
angle. Real clouds send more back toward the Sun and forward along the light,
and show a glory and bright rims at the terminator.
- **No light between the cloud and the ground**: the ground's light reflected
back down by the cloud's base is left out. It matters over snow and ice.
- **The sky light under a cloud** is dimmed by the cloud toward the Sun, not by
the clouds over the whole sky.
- **At lighting L1, one day's clouds**, whatever the scene's date. Lighting L2
draws the scene's day's (next section).
- **The map** is 2.45 km per pixel. Small cumulus are blurred into a gray
veil. The composite also has straight seams where its pieces meet, at
high latitudes, which show as clouds that end on a straight line (east of
Greenland in the solar eclipse example).
- **Without the atmosphere** (`Appearance::earth_atmosphere` off), no clouds.
### Clouds of the day (lighting L2)
Code: `scripts/fetch-dated-clouds.py`, the command line's choice of map. Lighting
L2.
At lighting L1 the clouds are always those of 2001 July 29. Lighting L2 draws the
clouds of the scene's day, from MODIS on Terra and Aqua, with the same layer and
shading as above. Only the map differs.
**Making a day's map.** NASA's Global Imagery Browse Services (GIBS) serve the
MODIS cloud products as images, one global composite per day. The cloud optical
thickness comes as colors that stand for values, in steps of about 4 % from 0.01
to 150 (GIBS's color map). `scripts/fetch-dated-clouds.py FIRST [LAST]` fetches
each day's layers as two hemispheres of 8192 x 8192 pixels (GIBS times out on the
whole map at once) and turns the colors back into numbers. All 252 colors in the
images are in the color map, so none were blended in the resampling. For each
pixel it takes the first optical thickness of:
1. Terra's standard retrieval;
2. Aqua's;
3. Terra's partly cloudy retrieval (PCL);
4. Aqua's PCL.
That thickness becomes reflectance by the formula above. A pixel with none of them
but a cloud fraction from either satellite is clear, because MODIS retrieves
thickness wherever its cloud mask finds cloud in daylight. A pixel with no data at
all, between passes or in polar night, takes the static map's value. The map is
written at the static map's size, 16384 x 8192, as `data/clouds/YYYY-MM-DD.png`.
Each day takes about two minutes and 0.3 GB of downloads, and keeps 20 MB.
For 2024 July 15 that gives:
- thickness over 56 % of Earth's area;
- clear over 43 %, including the dark Antarctic winter, where only the infrared
cloud fraction has data;
- no data over 1.3 %.
The mean reflectance is 0.247, against 0.207 for the static map. Fewer pixels are
cloudy, 55 % against 65 % north of 60 S, but those that are are brighter.
**Checking it.** Earth rendered at phase angles from 0 to 180 degrees, as for the
static map, reflects 0.30 to 0.32 of the sunlight in green (Bond albedo) with the
day's clouds, against 0.26 to 0.28 with the static map and about 0.30 measured.
Seen from 80,000 km over Africa, the day's clouds lie where MODIS's true-color
image of that day shows them: the storms over the North Atlantic, the band over
the Sahel, and the clouds over Europe. The Saharan dust in that image is aerosol,
not cloud, and is not drawn.
**Choosing the day.** The command line takes the day of the scene's epoch in UTC,
or `[appearance] cloud_date` in a scene without an epoch. The July example scenes
give 2024-07-15. A day whose map is missing is an error that names the command to
fetch it, and a scene with no day at all is an error too. Lighting L2 needs a
cloud map and Earth's atmosphere, and the renderer refuses it without them.
`Renderer::set_cloud_map` swaps one map for another between frames, for a run that
crosses days. It takes about 175 ms on the RTX 3060 and 240 ms on lavapipe, mostly
building the mip chain and uploading it, and the frames after it are those of a
renderer made with the new map. Rendering costs what it does at L1.
Tests: the day's choice from `cloud_date`, from the epoch on either side of UTC
midnight, and from a TT epoch half a minute after midnight, which is still the day
before in UTC; the errors for a missing map and for a scene with no day; L2's
refusal without clouds or without the atmosphere; and a swap that draws what a
fresh renderer draws, to the bit, on the GPU and lavapipe.
Limitations:
- **Seams.** Terra crosses the equator at 10:30 local time and Aqua at 13:30, so
where one satellite's data gives way to the other's, clouds that moved in three
hours end on a line. So do the edges of a single satellite's passes, made at
different times of the day.
- **One map per day.** The clouds of a whole UTC day are drawn at any time of it,
though each place is seen by MODIS only around late morning and early afternoon.
- **Thickness at 1 km, map at 2.45 km.** Broken clouds below that blur, as in the
static map.
- **Clear where MODIS saw no cloud,** including where its retrieval failed for
other reasons, such as sun glint; there the map has no cloud that may have been
there.
- **Polar night** takes the static map, which is dark there anyway.
### The Moon's surface
Code: `moon.rs`, `shaders/bodies.wgsl`; data from `scripts/fetch-moon.py`. From
lighting L1, with the map in `RendererOptions::moon_texture`.
At lighting L1 the Moon's reflectance comes from a map instead of one spectrum.
The map is the LRO Camera's Wide Angle Camera mosaic at 643 nm (red), built from
more than 100,000 images taken under every kind of lighting. The camera team
used a model of how each spot on the Moon reflects light at different angles
(Hapke's, fitted to the images; Sato et al. 2014) to work out what every spot
would look like under one standard geometry: the Sun 60 degrees from overhead,
seen straight down. The map's values are that reflectance, I/F, the ratio of
the surface's brightness to that of a perfect white surface facing the Sun.
Over 98 % of the Moon they lie between 0.019 in dark maria and 0.067 on the
brightest highlands and crater rays, and they average 0.042.
The map is 400 m per pixel and covers 70 degrees south to 70 degrees north;
the polar caps beyond are filled with the average, so they render as smooth
gray (6 % of the surface, seen only near the limb from Earth). It is resampled
to 16384 x 8192, 666 m per pixel, the largest texture lavapipe takes, and read
exactly as Earth's image is: by latitude and longitude in the Moon's own axes,
with mipmaps, and without a seam at 180 degrees.
The Moon's axes come from `Scene::moon_attitude`. The Moon keeps one face
toward Earth, but rocks back and forth by up to 8 degrees (libration), so
which features are near the limb changes from month to month. With an epoch,
the renderer uses the IAU's model of the Moon's rotation; it puts the point
facing Earth within 0.008 degrees (230 m) of JPL Horizons' high-precision
frame, under a third of a map pixel. Without one, `Scene::moon_facing_earth`
turns the Moon's prime meridian toward Earth and is off by the libration, up
to about 10 degrees.
Two choices turn the map into something the renderer can shade:
- **Brightness.** The renderer shades the Moon as Lambertian, starting
from the albedo that gives the map's reflectance at the map's geometry:
I/F / cos(60 degrees), on average 0.084. On its own that makes the Moon
far too dark at full: the real Moon is not Lambertian, and it brightens
steeply toward full Moon, as rough soil hides its own shadows when lit
from behind the viewer. A Lambert sphere with these albedos is about a
third as bright as the real full Moon, and slightly too bright past
quarter. So each point's albedo is multiplied by a phase factor, from the
phase angle between the Sun and the viewer seen from that point. The
factor is the whole Moon's measured brightness at that phase (Allen's
lunar phase curve, 0.026 g + 4e-9 g^4 magnitudes below full, g in
degrees) over a Lambert sphere's, scaled so that the map's full Moon has
the measured geometric albedo (the full Moon's brightness against a white
Lambertian disk of the same size), 0.121 in the V band. It is 2.8 at 4
degrees, 1.16 at 60, 0.89 at 90 and 0.75 at 120, and holds its 150 degree
value beyond. At 60 degrees it would be 1 if the map and the measured
curve agreed exactly; the 16 % between them is the Lambert shape's error
at that phase together with the two calibrations'.
Rendered whole in the example camera's green channel, the Moon is then
within 5 % of the measured curve from 2 to 30 degrees of phase, and
within 11 % out to 120 degrees. At the same phase, the side of the Moon
holding Oceanus Procellarum, the largest of the maria, is darker than the
other: 0.89 of the curve at 90 degrees against 1.06. The real Moon differs
between waxing and waning in the same way, which the curve averages out.
At 150 degrees it is 9-15 % bright.
Across the disk the light still falls off as Lambert's does, darkening
toward the edge, while the real full Moon looks nearly flat out to its
limb. A lunar reflectance model would fix that.
- **Color.** The map is one band. Its color comes from the Moon's average
spectrum across the WAC's seven bands, 321 to 689 nm, from the same mosaics
averaged over the whole Moon by area: relative to 643 nm, 0.42 at 321 nm, 0.61
at 415 nm, 0.87 at 566 nm and 1.08 at 689 nm. The Moon is distinctly red,
which is why it looks yellowish gray to the eye. Each pixel's
643 nm value scales that spectrum, which is folded into the camera's
channels once at setup: in the example camera the Moon's albedo in red,
green and blue is 0.96, 0.83 and 0.69 times its 643 nm value over cos 60.
Beyond 689 nm the spectrum continues the slope from 643 to 689 nm to 900 nm
(1.45) and holds there, roughly following real lunar soils, which flatten
into an absorption by iron-bearing minerals near 1000 nm; below 321 nm it
holds the 321 nm value. Real maria rich in titanium are up to about 10 %
bluer than this, and highlands slightly redder; the seven-band mosaic
could give each pixel its own color later. The real Moon also reddens
slightly with phase, which the phase factor leaves out.
Light the Moon sheds on targets (Moonshine) reads the same map, with the
phase factor for the angle between the Sun and the target.
Tests (`tests/gpu/moon_texture.rs`), with small synthetic maps, on the GPU and
lavapipe: the pixel under the camera reads the texel at its latitude and
longitude, and moves when the Moon's attitude does; the color is the spectrum's;
a uniform map draws exactly the Moon a flat albedo draws times the phase
factor, at every pixel; there
is no seam at 180 degrees; a far view averages a fine checker; Moonshine comes
from the part of the map that faces the target.
Cost: the map adds about 0.04 ms to the bodies pass on the RTX 3060, for working
out where to read it at every pixel (the read itself happens only where the
Moon is). On lavapipe the pass got faster, from 2.7 to 2.0 ms with the Moon
filling the view, because Earth's image is now also read only where Earth is.
Loading the map takes 0.5 s and uploading it 0.17 s; it occupies 360 MB on the
GPU.
### The Sun's disk
Code: `sun.rs`, `shaders/common.wgsl`, `shaders/bodies.wgsl`. From lighting
L1.
The Sun's disk is brighter at its center than at its edge: limb darkening.
Light leaving the Sun comes from a layer a few hundred kilometers deep in
which the gas gets hotter with depth. Looking at the center of the disk, the
line of sight goes straight down into hot gas; near the edge it runs through
the layer at a slant and sees higher, cooler gas, which shines less.
Where the line of sight meets the Sun's surface, call mu the cosine of the
angle between it and the surface's normal: 1 at the center of the disk, 0 at
the edge, 0.5 at 87 % of the way out. Neckel and Labs (1994) measured the
brightness across the disk at 30 wavelengths from 303 to 1099 nm, and
Hestroffer and Magnan (1998) showed that at each wavelength it is close to
mu^alpha times the center's, to about 1 %. Alpha is 0.94 at 303 nm, 0.48 at
580 nm and 0.26 at 1099 nm: blue light darkens much more toward the edge
than red. The renderer uses their values of alpha, interpolated between those
wavelengths and held constant outside them.
The total light from the disk does not change; that is the measured solar
spectrum. Only its spread does. A disk of brightness mu^alpha averages 2 /
(alpha + 2) of its center, so the center is (alpha + 2) / 2 times the mean.
Folded through the example camera's channels, the center is 1.22, 1.26 and
1.31 times the mean in red, green and blue, and 87 % of the way to the edge
(mu = 0.5) the disk is 0.70 of its center's brightness in green. The
folding is exact: for each channel the renderer weights every wavelength's
profile by that channel's solar flux and tabulates the result, so any channel
layout works.
It matters in two places:
- **The Sun in an image.** A camera that looks at the Sun almost always
saturates, but not with a dark filter, a Sun sensor, or once the lens's
blur, once modeled, spreads the Sun's light into the pixels around it.
- **Eclipses.** When Earth or the Moon covers part of the Sun, what counts is
how much light it hides, not how much of the disk's area. Near the umbra
only a sliver of the dim edge is left, so less light gets through than the
area says: 7000 km behind Earth, 25 km inside the penumbra's dark side,
4.5 % in green against 6.1 %. Near the outside of the penumbra the covered
part is dim edge, so more gets through: 88.2 % against 86.0 %. In an
annular solar eclipse the Moon covers the bright center and leaves the dim
ring: 1 % of the light reaches the ground, not the 2.4 % of the disk's area.
The renderer computes the light hidden by an occulter in two parts. Circles
around the Sun's center that the occulter covers completely count through a
table of the light inside each radius. Circles it covers in part count by
adding up, over eight radii, each circle's brightness times the length of its
arc inside the occulter, with the radii placed by a Gauss-Legendre rule; eight radii are within 2.4e-5
of what 32 give. The same
calculation runs on the CPU for each target's eclipse and on the GPU for
every pixel of Earth and the Moon in an eclipse, per channel.
Tests: every pixel of the Sun seen through a 250 mm lens (420 pixels across)
reads its mean radiance times the profile at its mu, to 6e-5, and the disk's
light summed over pixels is the channel's solar flux to 7e-5. The light
hidden by an occulter matches a fine sum over the disk to within 1e-4, and a
uniform disk gives the area overlap. In eclipses the tests of the sections below use
the limb-darkened light.
Cost: nothing measurable on the RTX 3060 unless an eclipse is in view. With
Earth filling the image and most of it in the Moon's penumbra, the bodies
pass takes 0.43 ms instead of 0.21 ms; lavapipe 9.5 ms instead of 5.0 ms.
Limitations:
- **The very edge.** Hestroffer and Magnan fitted the profiles over the
inner 90 % of the radius, where measurements are reliable. Right at the
edge mu^alpha falls to zero, faster than the measured Sun does, over the
outermost fraction of a percent of the radius.
- **Wavelengths outside 303 to 1099 nm** use the end values. The example
camera's channels are inside that range.
- **A quiet Sun.** Sunspots and bright faculae change the Sun's output by
about 0.1 % over the solar cycle and are not modeled.
- **Shadows on the target** (above) still treat the Sun as a square of
uniform brightness in their penumbra.
### Depth from the bodies
The pass also writes each hit's depth, near / z, as described in the
geometry section. The target spacecraft, drawn after the bodies, uses the
same depth buffer. So a target in front of Earth covers it, and a target
behind Earth's limb is hidden.
## The target model and its materials
Code: `model.rs`.
The target spacecraft is a triangle mesh loaded from STL or glTF. Each
piece (an STL file, or a glTF primitive) is a part with one material, and
for now a material is a Lambertian albedo spectrum, as for Earth and the
Moon.
Surface normals decide how bright each triangle is, through the cos(i)
factor. STL files store a normal per triangle, but exporters often write
zeros or wrong values, so the loader ignores them. It recomputes each
normal from the order of the triangle's corners: counterclockwise as seen
from outside. Every triangle is then lit as a flat facet, which matches the
flat panels and boxes most spacecraft are built from. glTF files that store
normals keep them.
glTF describes a material's color as red, green and blue reflectances (its
"base color"). Three numbers do not define a spectrum: very different
spectra can look the same color to the eye. The loader turns them into a
stepped spectrum, blue below 490 nm, green from 500 to 580 nm, and red above
590 nm. That is a reasonable guess for painted surfaces seen by an RGB
camera. It is a poor one for a camera whose channels differ from the eye's,
or for anything in the near infrared, where the red value simply continues.
For absolute radiometry of a real spacecraft, give the important materials
measured reflectance spectra (not supported yet).
Texture images are not used yet; each material renders with its base color
value alone. The loader removes textures before reading a glTF file, so a
model still loads when its textures are in a format or file the loader cannot
read. A model that relies on textures for its colors (one material with a
painted image) renders in that material's flat base color, often gray.
STL has no units. CAD tools usually export millimeters, so loading a CAD
model usually needs `scale: 0.001`. glTF is meters by its specification, but
models in the wild do not always follow it: of the NASA example models,
Landsat 8 and ICESat-2 are in meters, while the SSL-1300 needs a scale of 0.1
and ACE about 0.005. Checking a model's size against the real spacecraft's is
worth the minute it takes.
## Drawing the target
Code: `gpu/mesh.rs`, `shaders/mesh.wgsl`.
The target's mesh is rasterized after the bodies, into the same radiance
and depth images. Each pixel the target covers is kept only if the target
is nearer than what is already there. That one rule handles a target in
front of Earth, a target behind Earth's limb, and parts of the target
hiding other parts.
Each point on the target is shaded like the bodies: albedo times cos(i)
times the sunlight that reaches it. A panel modeled with no thickness, such
as a solar array, has one surface but two sides. Whichever side faces the
camera is the side that is lit. A closed shape never shows its inside, so
this only matters for thin parts.
### Shadows on the target
Code: `shadow.rs`, `gpu/shadow.rs`, `shaders/mesh.wgsl`. From lighting L1.
A solar array shades the bus behind it, an antenna dish shades its feed,
and a second spacecraft close by can shade the first. Rasterization draws
each triangle on its own, so by itself it cannot know that something else
stands between a point and the Sun. The renderer finds out with a shadow
map.
- **Shadow map.** Before the camera's view is drawn, the target is drawn
once more as the Sun sees it, looking straight down the Sun's rays. That
drawing keeps only depth: for each texel (a pixel of the map), how far
along the Sun's direction the first surface is. The Sun is far enough
away that its rays are parallel across a spacecraft, so the view has no
perspective, and a map 4096 texels across covers the target's bounding
sphere. For Landsat 8, whose sphere is 16.6 m across, one texel is 4 mm.
- **Lookup.** When the camera's view shades a point, it finds that point
in the map. If the map holds a surface nearer the Sun than the point, the
point is in shadow.
Each target gets its own map. Another target is drawn into that map when it
is sunward of this one and close enough across the Sun's rays to shade it.
A target off to the side is left out, since it cannot cast a shadow here.
The Sun is a disk half a degree across, so a shadow edge is not sharp. A
point near the edge sees part of the disk past the blocker, the same
penumbra as in an eclipse. The width of the penumbra is the distance from
blocker to surface times the tangent of the Sun's angular diameter, 0.93 %
at 1 AU. A solar array 2 m above the bus casts an edge 19 mm wide. One 10 m
away casts an edge 9 cm wide. The renderer reproduces this with
percentage-closer soft shadows:
1. Search the map around the point for blockers, and average their depth.
That gives the blocker-to-surface distance, and from it the penumbra
width.
2. Compare the point's depth against a 6 x 6 grid of map texels spread over
that width, and take the fraction that are not blocked as the share of
the Sun the point sees.
Both steps use fixed grids rather than random samples, so the image has no
noise and the same scene always gives the same pixels.
A surface can wrongly shadow itself, because the map only stores its depth
once per texel. Where the surface is tilted to the Sun, depth changes
across a texel, and half of each texel looks like it is behind itself. The
result is a pattern of dark stripes called shadow acne. The renderer
prevents it three ways. The stored depth is pushed away from the Sun in
proportion to the surface's tilt. The lookup is made from a point 1.5
texels off the surface, toward the Sun. And a blocker has to be more than 2
texels nearer the Sun to count.
Tests (`tests/gpu/shadows.rs`) check, on the GPU and lavapipe:
- A plate under a 2 m square: the shadow reads zero direct light, and its
edge falls where the square's edge projects, to within one texel.
- The measured penumbra width is within 2.2 % of 0.93 % of the occluder's
height, at heights of 1, 2 and 3 m.
- No pixel darkens compared with L0 on a plate tilted 80 and 87 degrees
from the Sun, or on a faceted sphere.
- A target sunward of another shades it; one to the side does not.
On the RTX 3060, shadows cost 0.14 ms of GPU time per frame for Landsat 8
(85,000 triangles) and 0.34 ms for ICESat-2 (752,000), plus about 0.15 ms
in the camera view's lookups. Maps 2048 texels across save about 0.15 ms.
Limitations:
- **Penumbra shape.** The grid averages a square of the map, so brightness
rises in a straight line across the penumbra. A real disk-shaped Sun
rises along an S-shaped curve: the full width is the same, but the middle
80 % of the change is 0.69 of the width rather than 0.8. Limb darkening
(from L1) would steepen it a little more, and is not modeled here either.
- **Resolution.** A detail smaller than a texel casts no reliable shadow. A
gap narrower than about 2 texels between a blocker and the surface it
shades (a few mm on Landsat 8) lets light through, because of the acne
margins above.
- **One shadow lookup per pixel.** Shadows are looked up once per pixel, even
at camera C1, so the edge of a shadow inside a pixel is not graded by its
4 samples; the penumbra's own width usually grades it.
- **Bodies.** Earth and the Moon do not use shadow maps. Their shadows on
the target are the eclipse factor below, applied to the whole target.
Light from Earth and the Moon (below) is not shadowed at all.
### Eclipses
Code: `eclipse.rs`.
The Sun is a disk, not a point. So as a spacecraft passes into Earth's
shadow, the light fades over a few seconds instead of switching off:
- **Umbra.** Earth covers the whole solar disk. No direct sunlight.
- **Penumbra.** Earth covers part of the disk. 7000 km from Earth's
center, on the night side, the penumbra is about 65 km wide; a satellite
crosses it in under 10 seconds.
- **Annular.** A body small enough to sit inside the solar disk blocks a
ring-shaped bite. From orbit this happens with the Moon.
The renderer finds the visible fraction of the solar disk from the overlap
of two circles on the sky. The circles are the Sun's disk and the
occulter's, each with its apparent angular radius. It multiplies the
target's sunlight by that fraction. The model is the standard one for
satellites (Montenbruck and Gill), the same as astrokit's. A test checks it
against tracing lines from the target to thousands of points spread over
the solar disk.
From lighting L1 the fraction is of the Sun's light rather than its area,
per channel, because the disk is brighter at its center (see "The Sun's
disk"); that makes a penumbra darker toward the umbra and brighter toward
the outside than the area alone says.
The renderer counts both Earth and the Moon as occulters. The fraction is
computed once per target, at its center. The whole target
dims together, which is accurate unless the target is a sizable fraction of
the penumbra's width. With Earth's atmosphere (lighting L1), Earth's part is
the light that comes through the air past its edge, the next section. Light
reaching the target from Earth itself is the section after the next two.
### Sunset through the atmosphere
Code: `atmosphere/grazing.rs`, `eclipse.rs` (`sun_visible_through_air`,
`moon_near_shadow`), `shaders/bending.wgsl`.
From low orbit, the Sun sets behind Earth's atmosphere before it sets behind
the ground. For the last 20 seconds or so the light reaching a spacecraft has
crossed hundreds of kilometers of air edge on, and three things happen to it:
- **The air takes light out of it**, as anywhere else, but crossing about 70
times as much air as a path straight up from the same height. Blue goes first, scattered away by air
molecules. Then, through the ozone layer 20 to 30 km up, ozone's absorption
band near 600 nm takes more of the red and green than the air takes of the
blue, so the light is bluish for a few seconds: the turquoise fringe at the
edge of Earth's shadow in photographs of lunar eclipses. Lower down, below
about 15 km, the air's scattering wins again and what is left is orange and
red, until aerosols near the ground take the rest.
- **The air bends it.** Air's refractive index is a little over 1 and grows
with its density, so light passing the limb curves toward Earth: by 1.25
degrees for a ray that just skims the ground, 0.14 degrees at 15 km, 0.04 at
25 km. (An observer on the ground sees the setting Sun raised by half the
first figure, the familiar 35 arcminutes of refraction at the horizon.) So
the Sun is still seen, raised, after it has geometrically set.
- **The bending spreads it out.** Lower rays bend more, so rays that came in
parallel leave fanned out downward, and the light spreads over a larger
angle: refractive dilution. Seen from a spacecraft, the Sun's image is
squeezed into a flattened ellipse, still with the same surface brightness
apart from the air's losses, so it delivers less light. From 700 km the
squeeze is 0.88 for light passing 30 km up, 0.50 at 15 km and 0.27 at 8 km,
as large an effect there as the air's own losses.
Some numbers for a ray seen from 700 km, by how high above the ground it
passes (the example camera's red, green and blue channels):
| Height | Bending | Squeeze | Red | Green | Blue |
|---|---|---|---|---|---|
| 40 km | 0.006 degrees | 0.96 | 0.97 | 0.96 | 0.92 |
| 30 km | 0.021 | 0.88 | 0.58 | 0.59 | 0.69 |
| 20 km | 0.076 | 0.66 | 0.22 | 0.21 | 0.28 |
| 15 km | 0.144 | 0.50 | 0.17 | 0.12 | 0.12 |
| 8 km | 0.367 | 0.27 | 0.10 | 0.03 | 0.01 |
Because the atmosphere is taken as a sphere of air whose density depends only
on height, a ray's whole path through it depends on one number: its **impact
parameter**, how close its straight line outside the atmosphere passes to
Earth's center. Two tables of 1024 entries, from the ray that just skims the
ground to the top of the atmosphere, hold each ray's total bending and its
transmittance per channel. They are integrals along the curved path, with
the refractive index proportional to the air's density in the same model as
the rest of the atmosphere (1.000278 at the ground, for 550 nm), computed at
setup in 2.4 ms on 24 cores.
**The target's sunlight.** Seen from the target, a ray that arrives from an
angle theta from Earth's center passes at an impact parameter that follows
from theta, and came from theta less its bending. The renderer adds up the
Sun's limb-darkened radiance over the solid angle the Sun's image covers,
each ray's from where it came from, times its transmittance: 64 rows across
the image, each an arc around Earth's center, 8 points along each. That gives
all three effects at once, per channel. It takes 17 to 25 microseconds per
target near the limb, and nothing elsewhere. Through a sunset seen from
700 km, the light fades from 99 % to 6 % of red, 3 % of blue, over the 0.02
radians (19 seconds) of orbit before the straight line to the Sun's center
meets the ground. Refraction keeps that last 6 % coming after the geometric
edge. The sunset example scene (`examples/scenes/sunset.toml`) shows Landsat 8
1.4 s before that point, lit by 7.2 % of the Sun's red light and 4.4 % of its
blue, against the twilight at the limb.
**The Sun and Moon seen through the limb.** The bodies pass bends each line
of sight that passes through the atmosphere: it finds the line's impact
parameter, turns it toward Earth by its bending about Earth's center (the
path is symmetric, so the line it leaves on is the one it came in on turned
by the bending), looks for the Moon or the Sun along that, and multiplies
what it finds by the transmittance. The Sun near the horizon comes out
flattened and red, as in photographs from the International Space Station.
Only frames where the Sun or the Moon is near the limb need it, and the code
alone, even never run, cost 0.3 ms a frame on the RTX 3060, so it is a second
version of the bodies pass that only those frames use. In them it adds
0.4 ms on the RTX 3060 and 6 ms on lavapipe.
**The eclipsed Moon.** The same rays reach into Earth's shadow. The lowest
are bent by 1.25 degrees, so they cross the shadow's axis about 290,000 km
behind Earth (Earth's radius over that angle). The Moon, at 384,000 km, is
past that point, so from anywhere in the umbra it sees a thin ring of red
sunlight around Earth's limb. A ray seen low enough comes from the far side
of Earth's center, and the target's sum follows it there. Across the umbra
at the Moon (the example camera's channels, as fractions of the Sun's light):
| From the axis | Red | Green | Blue |
|---|---|---|---|
| 0 km | 5.9e-5 | 6.1e-6 | 3.5e-7 |
| 2000 km | 1.6e-4 | 3.2e-5 | 4.7e-6 |
| 4000 km | 6.5e-4 | 3.2e-4 | 2.0e-4 |
| 4600 km, the umbra's edge | 1.8e-3 | 1.4e-3 | 1.5e-3 |
Near the axis the light comes from rays that pass below about 5 km; 4000 km
out, from rays about 5 to 19 km up. The center is darkest, because the Moon
is too far past the point where the lowest rays cross to see them gather on
the axis. Photometry of the eclipse of June 2011 (Ugolnikov et al.,
arXiv:1106.6178) found 1e-4 to 1e-3 at 677 nm from the center out to 0.6
degrees (4000 km), and about 1e-6 at 503 nm. A model of clear air there
agreed with the outer umbra and was brighter further in, where aerosols took
light out.
A sum for every pixel of the Moon, at 20 to 40 microseconds each, is too much.
The shadow is symmetric about its axis, so the light at a point depends only
on two things: its distance from Earth's center, and the angle between the
Sun and Earth's center seen from it. When the Moon comes near Earth's shadow,
the renderer puts that light in a table (`EarthShadow`), and the bodies pass
reads it for each pixel of the Moon.
- **Layout.** Two rows of 512 entries, on the near and far sides of the Moon.
The entries are spaced in the angle less the atmosphere's angular radius,
which lines the shadow's edges up in both rows.
- **Contents.** It holds the light's logarithm, which follows the umbra's
steep falloff better than the light itself does. Entries fall where the
Sun's edges cross the atmosphere's top, where the light's slope jumps. It
is within 0.5 % of the sum.
- **Cost.** Building it takes 1.6 ms on 24 cores and 6.1 ms on 4. It is kept
while the Moon stays within it, which is 870 km of the Moon's distance, 3.5
hours or more, and while the Sun's angular size stays within 1e-4 of the
same. Those frames use the second version of the pass, as for the limb:
1.80 ms against 1.45 ms for the lunar eclipse example on the RTX 3060, and
4.7 ms more on lavapipe.
`examples/scenes/lunar-eclipse.toml` shows the eclipse of 2025 March 14 at a
0.3 s exposure: copper red, and brightest along the Moon's northern edge,
which was farthest from the axis.
Tests:
- In `atmosphere/grazing.rs`: straight rays' columns of air, aerosols and
ozone against direct integrals (within 1e-5); the bending and the air along
the path against rays traced through the atmosphere step by step (within
1e-4); with no air and no bending, the target's light against the eclipse
model (within 3e-4: the eclipse model draws Earth's edge as a circle on a
flat sky, off by 1.7e-4); 64 rows against 512 (2e-5); a point-like Sun
dimmed by transmittance and dilution (within 1 %); a sunset that dims
every channel steadily, blue first, with the ozone's blue phase; Earth's
shadow at the Moon's distance against following 2 million rays back across
the sky (within 1e-3, measured 5e-4); the eclipsed Moon's light and color
against the 2011 photometry; and the shadow's table against the sum over the
Moon's depth (within 6e-3, measured 4.9e-3).
- In `tests/gpu/atmosphere.rs`, on the GPU and lavapipe: the Sun behind air
12, 20 and 30 km up, pixel by pixel down the image's middle against the
same bending and transmittance in f64 (within 7.3e-4); and the Moon behind
air 15 km up, where each pixel's light over its transmittance and its
shading is constant to 1.2e-3; and the Moon across the umbra's edge, each
pixel's light against the sum at its point (within 4.7e-3 on the GPU and
3.6e-3 on lavapipe, the table's interpolation).
- In the command-line tests: the eclipse of 2025 March 14 is red, and brighter
at the Moon's northern edge than at its center and its southern edge.
Limitations:
- **One refractive index for every wavelength.** Air's changes by about 2 %
across the visible spectrum, which separates the Sun's colors at the limb
slightly (the green flash). Not modeled.
- **The model's scale height near the ground.** The 8 km scale height bends a
ground-skimming ray by 74.8 arcminutes; the real lower atmosphere, whose
density falls more slowly, gives about 70.
- **No clouds or volcanic aerosols.** In reality clouds block most rays
below 10 km or so. At sunset the light that gets through there is already
small. In the umbra it is most of the light, so the eclipsed Moon here is as
bright as after years of clear air. Real eclipses range over a factor of 100
or more, and are darkest after large eruptions.
- **No light scattered into the shadow** by the air, which is about 1e-7 of
the Sun's.
- **Light from the eclipsed Moon onto targets** still uses Earth's hard edge,
so it is zero in the umbra.
- **A target inside the atmosphere** takes the transmittance straight toward
the Sun's center, without bending.
### Eclipses on Earth and the Moon
Code: `shaders/common.wgsl` (`sun_visible`), `shaders/bodies.wgsl`. From
lighting L1.
Earth and the Moon also shadow each other. When the Moon passes through
Earth's shadow it is a lunar eclipse; when the Moon's shadow falls on Earth,
a solar eclipse. For every pixel that sees Earth or the Moon, the shader
works out the fraction of the Sun's light that reaches that point on the
surface past the other body, with the same overlap of circles and the same
limb-darkened disk as for the target, and multiplies the point's sunlight by
it, per channel. With Earth's atmosphere, the Moon's points take the light
through the air instead, from the table in the previous section.
The sizes involved:
- At the Moon's distance, Earth's umbra reaches 4600 km from the shadow's
axis and its penumbra 8200 km. The Moon, 3474 km across, fits inside the
umbra with room to spare, so a total lunar eclipse lasts over an hour.
- The Moon's umbra only just reaches Earth. When the Moon is near its
closest, the umbra touches down as a dark spot up to about 270 km across:
a total solar eclipse. When it is farther, the umbra ends in space, and
beyond its tip the Moon covers only the middle of the Sun's disk, leaving a
ring: an annular eclipse, where about 1 % of the Sun's light still arrives,
the dim edge of the disk. The
penumbra around either is about 7000 km across.
The overlap is computed in 32-bit floats, which is enough: a lunar surface
point is placed to about 20 m out of 384,000 km, 6e-8 radians against a
solar radius of 4.6e-3 radians seen from there. One formula needed care. The
area where the two circles overlap involves angles whose cosines are close
to 1 when a sliver of the Sun is covered, and the inverse cosine of a number
near 1 loses most of its precision in 32 bits. The shader finds those angles
from their sine and cosine together instead. With the inverse cosine,
pixels at the shadow's edges came out wrong: a point the 64-bit model puts
fully in the umbra read slightly lit, and one just outside the shadow
slightly dimmed.
Tests (`tests/gpu/eclipses.rs`), on the GPU and lavapipe, compare each pixel of
Earth and the Moon against the 64-bit model at that pixel's surface point:
the Moon entirely in Earth's umbra, the Moon across the umbra's edge, a
total solar eclipse and an annular one. Pixels the model puts fully in shadow
read zero, those it puts fully outside are unchanged (both to within 1e-6,
the 32-bit rounding at the shadow's edges), and the rest
agree to within 1e-4 of the Sun's light (1.2e-6 at most on the GPU, 5e-5 on
lavapipe). The command-line tests render two real eclipses from their
epochs: the total lunar eclipse of 2025 March 14 at mid-totality, where the
whole Moon is in the umbra (red through the atmosphere, previous section), and
the total solar eclipse of 2026 August 12,
where the point of greatest eclipse in NASA's bulletin (65.2 N 25.2 W) is in
the umbra (`examples/scenes/lunar-eclipse.toml`, `solar-eclipse.toml`). In
the second, the umbra is 220 by 410 km in the image, stretched along the
direction of the Sun, which is 26 degrees high there; NASA gives the path as
294 km wide.
The light Earth and the Moon reflect onto targets (next section) includes
these shadows, so an eclipsed Moon sheds less light.
On the RTX 3060 this adds nothing measurable to the bodies pass unless an
eclipse is in view; with Earth filling the image and most of it in the
Moon's penumbra, the pass takes 0.43 ms instead of 0.20 ms, most of it the
limb-darkened sum (see "The Sun's disk"). On lavapipe, 9.5 ms instead of
3.9 ms.
Limitations:
- **Earth's atmosphere.** With it, the umbra on the Moon is lit red through
the air (previous section). Without it, the umbra is black. The observed
umbra is about 2 % larger than the solid Earth's, which is not modeled.
- **Smooth shapes.** Earth's shadow is cast by a sphere of its equatorial
radius, as for the target's eclipses; its flattening would move the
shadow's edge on the Moon by about 20 km. The Moon's mountains and valleys,
which shape the edge of a solar eclipse's path, are not modeled.
### Light from Earth and the Moon
Code: `gpu/earthshine.rs`, `shaders/earthshine.wgsl`, `shaders/mesh.wgsl`.
From lighting L1.
Earth reflects about a third of the sunlight that reaches it, and some of
that lands on the target: Earthshine. From low orbit it is large. At 500 km,
Earth fills a circle 68 degrees in radius around the point straight down,
3.9 of the sky's 12.6 steradians. A plate facing straight down over the
sunlit Earth receives up to about a quarter as much light from Earth as a
plate facing the Sun receives from the Sun. A white Earth would give 0.85
(the test below measures that case). At geostationary distance, Earth is 17
degrees across and a plate facing a full Earth gets about 0.5 % of sunlight.
The Moon does the same, Moonshine, much more weakly: a full Moon seen from
near Earth gives about a millionth of sunlight. From 100 km above the Moon,
though, a plate facing it gets about a tenth of sunlight.
This matters most where the Sun does not reach: the target's night side,
and parts in a shadow of its own. Without it those read as black, the bias in
DN; with it they show their structure, dimly.
The renderer works out, for each target and each frame, how much light
arrives from Earth and the Moon on a surface facing any direction:
1. **What the target sees of each body.** The circle on the sky that the body
fills is cut into 1024 cells of equal solid angle: 32 rings from the
center out to the edge, 32 cells around each. A line from the target
through each cell's center finds the point it meets on the body, and that
point is shaded exactly as the camera's view of the body is shaded:
reflectance from the Blue Marble image (or the stand-in spectrum), lit by
the cosine of the Sun's angle there, dark on the night side, and at
lighting L1 seen through the atmosphere. Each texture
read averages the patch of ground the cell covers, about 1000 square
kilometers straight down from 500 km and more toward Earth's edge, so a
cell sees the mean color of that patch rather than one point. A line
that meets the other body first counts nothing, so the Moon can hide part
of Earth and Earth part of the Moon.
2. **Light for every direction a surface can face.** For a surface facing
direction n, the light it receives is the sum over all cells of the cell's
brightness, times its solid angle, times the cosine between n and the
cell's direction (zero for cells behind the surface). The renderer adds
this up for a grid of directions, a cube map 16 texels across on each of
its six faces.
Drawing the target, each point reads that cube at its surface normal and
adds albedo / pi times the result to the direct sunlight. The cube is computed
at the center of each target's bounding sphere and used for the whole target;
across a spacecraft the view of Earth barely changes.
Earthshine is not dimmed by the target's eclipse factor, because the two
are different views of Earth. From anywhere in Earth's umbra only the night
side is in view, so Earthshine is zero there without being told. In the
penumbra the Sun is partly hidden but a crescent of lit ground is still in
view, and that light still arrives.
Tests (`tests/gpu/earthshine.rs`) compare a plate's brightness on the GPU and
lavapipe against the same sum done in 64-bit floating point with 90 times as
many cells: over the subsolar point at 500 km facing down, sideways and up;
toward the terminator; in Earth's shadow; at GEO; 100 km above the Moon at
full and quarter phase; with each body hiding the other; with the plate seen
from several sides, so each cube face a camera-facing surface reads is checked. Every
case where the bodies send more than a thousandth of sunlight agrees to
within 0.5 %. The largest differences come from the cube: between texels
5.6 degrees apart its interpolation rounds off the peak of a surface facing
a full Earth or Moon, by 0.4 %. A cube 32 texels across cuts that to 0.13 %
but costs four times as much, and the cells' own error on a half-lit distant
Earth (0.4 %) stays. A thin crescent of lit ground at Earth's edge, seen from
low orbit near the terminator, is off by 4 % of a small amount, 7e-5 of
sunlight, about 0.1 DN on a gray surface in the example camera. Another test
checks the sum itself against the classical formula for a distant sphere,
(2/3) albedo (R/D)^2 Phi(alpha), to within 0.3 %.
On the RTX 3060 this costs 0.04 ms of GPU time per target (0.28 ms for
eight), and 1.4 ms per target on lavapipe.
Limitations:
- **No shadows.** The target's own parts do not block light from Earth or
the Moon. A panel facing Earth does not shade the bus behind it from
Earthshine, and the inside of a dish facing Earth is lit as if open to the
whole Earth. Earth fills too much of the sky for one shadow map to stand in
for it; a ray tracer would handle this.
- **One spectrum per channel.** Light from Earth is bluer than sunlight, but
the target's albedo in each channel was worked out under sunlight. This is
the shortcut described in "Folding spectra into per-channel constants",
and its error is largest for strongly colored surfaces.
- **Lambertian bodies.** Earth and the Moon reflect as Lambertian surfaces
here, as they do in the camera's view, with the Moon's map scaled by its
phase curve as seen from the target. Clouds are Lambertian too.
## Where the Sun and Moon appear
Code: `scene.rs` (`Velocities`, `aberrate`), `ephemeris.rs` (feature
`astrokit`), `shaders/bodies.wgsl` (`seen`).
With the `astrokit` feature, the Sun's and Moon's positions come from JPL's
DE440s ephemeris, and Earth's orientation from the ITRF93 kernel, which
includes the slow wobble of Earth's pole. These are geometric positions:
where each body is at the epoch.
A camera sees something slightly different, its apparent position:
- **Light time.** Light from the Moon takes 1.3 s to reach Earth, and from
the Sun 8.3 minutes. The camera sees where each body was when the light
left it. For the Moon that is about 1.3 km of motion, a few microradians.
- **Aberration.** A moving camera sees incoming light tilted toward its
direction of motion, as rain seems to slant toward a moving car. Earth's
orbital speed (30 km/s) tilts the Sun's apparent direction by about 20
arcseconds, 100 microradians. A spacecraft's own orbital velocity adds up
to about 5 arcseconds more.
100 microradians is about one pixel of the example camera, and 18 pixels of
a narrow camera with a 1 m focal length.
The two effects combine simply. Light time moves a body back along its own
velocity by (distance / c) times that velocity; aberration moves it along
the observer's velocity by the same factor. To first order, then, a body
appears moved in direction by the observer's velocity relative to the body,
divided by c, the part of it across the line of sight. Only relative
velocities matter. For the Sun that is Earth's velocity around it plus the
spacecraft's: the 20 arcseconds. For the Moon, Earth's 30 km/s cancels,
since the Moon shares it, and what is left is the Moon's own 1 km/s around
Earth (0.7 arcseconds from Earth's center) plus the spacecraft's velocity.
For Earth itself it is the spacecraft's velocity, up to 5 arcseconds in low
orbit, about 0.2 px of the example camera. The terms left out are of the
order of the velocity squared over c squared, 1e-8 rad.
From camera tier C1 the renderer draws Earth, the Moon and the Sun at their
apparent positions. The scene carries the velocities (`Scene::velocities`:
the observer's relative to Earth, Earth's relative to the Sun and the Moon's
relative to Earth), and `Ephemeris::scene` fills in Earth's and the Moon's;
the observer's is the caller's. All zero, as in C0, the images are
geometric. Each pixel's ray is turned back, per body, by that body's
correction before it is tested against the body, so the correction follows
the ray's direction across a wide view of Earth, and the lighting, shadows
and eclipses, which happen where the bodies really are, stay geometric.
`Scene::apparent_sun` and `Scene::apparent_moon` give the same directions on
the CPU. From Earth's center they match JPL Horizons' apparent positions
(light time and stellar aberration) to under 0.002 arcseconds, against 20
arcseconds for the Sun's geometric position.
The camera's attitude is taken as the observer would measure it: a star
tracker sees the stars aberrated by the same motion, so the attitude it
reports already includes it. A simulation that passes a true, geometric
attitude gets images shifted by the aberration, as a real camera's would be
against such an attitude.
Still geometric: the targets, whose light time and aberration come from
their velocity relative to the observer, meters per second in rendezvous,
under 0.01 microradians; the rotation and lighting of a body during its
light time (the Moon turns 3 microradians in 1.3 s); and the light-time
delay between the Sun lighting a body and the camera seeing it.
## Stars
Code: `stars.rs` (`StarCatalog`, `StarPhotometry`); `gpu/stars.rs`,
`shaders/stars.wgsl`; `scripts/fetch-stars.py`.
Below lighting tier L1 the sky is black. A real camera shows stars wherever
its exposure is long enough, and they matter: star trackers find their
attitude from them, a camera measuring the Moon or a target measures it
against them, and to an algorithm looking for a target a star is a false one.
Stars are part of the scene, so they come with lighting tier L1, but only a
camera at tier C1 or above shows them. A C0 pixel takes its value from the
one point at its center, and a star is a point that almost never lies
exactly there; from C1 each pixel collects the light over its whole area,
which is what records a star.
**The catalogs.** `scripts/fetch-stars.py` merges three of ESA's catalogs
into `data/stars/stars.bin`: Hipparcos, 118,000 stars complete to about
magnitude 7.3; Tycho-2, 2.5 million stars to about 11.5 (99 % complete to
11.0); and Gaia DR3, from which it takes every star brighter than G 15 by
default, 37 million. Each star comes from the first catalog that has it, so
the brightest, which saturate Gaia's detectors, keep their Hipparcos or Tycho
measurements. Gaia measures brightness in its own bands, G and the color
BP - RP: V comes from them by Gaia's published relation (to 0.03 mag), and
B - V by a fit to the Hipparcos stars Gaia matched. Each star
keeps its position on the sky (ICRS, at epoch J2000), its proper motion and
its brightness and color. The renderer moves each star by its proper motion
to the scene's epoch when the catalog is loaded. Barnard's star, the
fastest, moves 10 arcseconds a year; most move under 0.1. Parallax, under 1
arcsecond for every star, is left out.
**Magnitudes.** A star's brightness is given as a magnitude: each step of 5
is a factor of 100, and larger is fainter. Vega is about 0, the faintest
stars the eye sees about 6, Tycho-2's limit about 11.5, 25,000 times fainter
than 0. The magnitude in the Johnson V band, a filter close to the eye's
green-yellow, fixes how much light the star sends through that band: V = 0
is 3.631e-11 W/m^2/nm averaged over it (Bessell, Castelli and Plez 1998).
The color index B - V, the difference between the star's magnitudes in blue
and in V, fixes its temperature: hot stars are blue, B - V below 0, and cool
ones red, up to about 2. The renderer takes the temperature from B - V
(Ballesteros 2012: the Sun's 0.65 gives 5,780 K), takes a blackbody spectrum
at that temperature, scales it so its light through the V band matches the
star's magnitude, and folds it through the camera's optics and channels, as
for sunlight. A star's spectrum is not a blackbody's: absorption lines and,
in cool stars, molecular bands move a few percent of the light between
channels. For the example camera, a Sun-colored star of V = 0 on the
boresight gives 56 electrons in green in the 0.1 ms exposure, against an
independent estimate of 56.4.
**Exposure.** That 56 electrons is why the example scenes show no stars: an
exposure set for a sunlit spacecraft or Earth, a fraction of a millisecond,
leaves even the brightest stars at a few hundred electrons and the rest
below one. Star trackers expose for 10 to 200 ms, and a telescope's larger
aperture collects more: a 1 m f/8 telescope, with a 12.5 cm aperture, gives
100 times what the example camera does.
**Drawing.** Each frame the renderer finds the stars in the field on the
CPU, from bands of declination sorted by right ascension, so it looks only
at the stars near the field. It places each at its apparent position, the
annual aberration of the observer's motion (about 20 arcseconds, see the
previous section) through the lens's distortion. It works out its electrons
in each channel: a point source loses light off axis only to the aperture's
foreshortening, cos(theta), not the cos^4(theta) of an extended surface,
since all of a star's light that enters lands in its spot. A star is not
rendered and then blurred: a point of light rendered into a pixel loses
where in the pixel it fell (see the blur section). Instead its electrons are
spread over the pixels around it as the Gaussian PSF integrated over each
pixel's area, which is exact for any position within a pixel. (Camera C2
draws stars into the focal plane before its convolution instead, so they
take the Airy or measured PSF too; see the PSF's wings.) The detector
adds them to the light from everything else before the full well, so a
bright star saturates like any other light. A star is hidden wherever
anything nearer is drawn, Earth, the Moon or a target: stars are at
infinity, so a star shows only where the depth image is empty. Stars giving
fewer than 0.1 electrons in every channel, far below any detector's noise, are
skipped, and the catalog is loaded only to the magnitude that can reach that.
Measured: a star moved across a pixel in tenths, behind a lens with
distortion and with Earth's 30 km/s of aberration, lands where the
projection puts its apparent direction to 0.01 px, with its electrons
matching the photometry to 0.2 %. A star behind Earth's night side or
behind a target leaves the bias alone.
Costs. A catalog is loaded only as deep as the camera's exposure can show:
for the example camera at 0.1 ms that is V 7.9, 42,000 stars, in 6 ms and 7
MB; at 50 ms, V 14.7, 21 million stars, in 2.8 s and 670 MB; for a 1 m
telescope at 0.2 s, the whole catalog, 37 million stars, in 5 s and 1.2 GB.
Each frame, finding and placing the stars takes 23 microseconds for the
telescope's 0.6 degree field (490 stars toward the Milky Way in Monoceros)
and 4.8 ms for the example camera's 12 degree field at 50 ms (52,000 stars).
Drawing them takes 0.24 ms for 40,000 stars on an RTX 3060, and 5.6 ms on
the CPU through lavapipe.
A catalog's depth matters most for a telescope's narrow field. Tycho-2 alone
averages 60 stars per square degree, so a 1 m telescope's 0.2 square degrees
would hold about a dozen; with no sky background in space such a telescope
records stars to magnitude 14 or 15 in half a second, and Gaia to G 15
averages about 900 per square degree. Deeper still (G 16, 78 million stars)
is a run of the script with `--gaia 16`, at twice the size.
## Not modeled yet
- Earthshine on the Moon's night side
- A point spread function that changes across the field, and the PSF's wings
past the reach of camera C2's grid
- Non-Lambertian materials (BRDFs), glints, and the full Moon's edge as bright
as its center
- Clouds as volumes (lighting L2): thickness, the shadows they cast on each
other, and clouds standing above the limb
- Earth's surface finer than Blue Marble's 460 m: mosaics of tens of meters
(Landsat, Sentinel-2) need tiles streamed in as a view needs them
- The atmosphere seen from inside it (a camera below 100 km)
- Motion blur, rolling shutter, stray light