Files
resolved/docs/optics.md
T
2026-10-14 21:31:22 -04:00

832 lines
41 KiB
Markdown

# Optics and radiometry guide
This explains the physics the renderer models, for a reader who knows
orbital mechanics but not optics. Each section says which code implements
it. When a feature adds a new optical effect, it adds or updates a section
here in the same change.
Numbers in the examples come from `CameraConfig::example()` unless stated
otherwise: 1920 x 1080 pixels, 5.5 um pitch, 50 mm focal length, f/4, three
color channels, 12-bit output.
## The chain from Sun to number
Every pixel value comes from the same sequence:
1. The Sun emits power across a range of wavelengths.
2. Some of it lands on a surface (Earth, the Moon, the target), which
reflects part of it in various directions.
3. The fraction reflected toward the camera enters the lens aperture. The
optics focus it onto one pixel, losing a little to absorption.
4. The pixel converts some of the arriving photons into electrons. The
fraction depends on wavelength and on the color filter over that pixel.
5. The electrons pile up during the exposure, up to a limit.
6. The readout electronics turn the electron count into an integer, the
digital number (DN).
The renderer follows this chain with physical units at every step. Image
brightness therefore comes from the scene and the camera settings, not from
a tuned scale factor. Change the exposure time, and the image changes
exactly as a real camera's would.
## Geometry: the pinhole camera
Code: `camera.rs` (`Intrinsics`), `scene.rs`.
A pinhole camera maps each direction in space to one point on the focal
plane. Put the pinhole at the origin, looking along +Z. A point at camera
coordinates (x, y, z) lands on the focal plane at (f x / z, f y / z), where
f is the focal length. Dividing by the pixel pitch p converts focal-plane
distance to pixels:
```text
u = (f / p) * x / z + cx
v = (f / p) * y / z + cy
```
`f / p` is the focal length in pixels. (cx, cy) is the principal point,
where the boresight hits the image, normally the image center. For the
example camera, f / p = 50 mm / 5.5 um = 9091 pixels.
Some quantities that follow from that:
- **IFOV, the instantaneous field of view.** The angle one pixel covers:
p / f = 110 microradians. At 10 m that is 1.1 mm per pixel; at 7000 km
it is 770 m per pixel. This sets the smallest detail the camera can
resolve.
- **FOV, the field of view.** The angle the whole image covers:
2 atan(width / 2 / (f / p)), about 12.1 x 6.8 degrees here. A longer
focal length or smaller sensor gives a narrower field of view and finer
IFOV.
Real lenses are not perfect pinholes. They bend straight lines slightly
(distortion, planned in M3) and spread each point into a small blur (the
point spread function, also M3). The pinhole model is the starting point
those effects are added to.
### Coordinate conventions
The camera frame has +Z along the boresight, +X toward increasing column u,
and +Y toward increasing row v. This is the computer-vision convention: row
0 is the top of the image, so +Y points down in the picture. Pixel (i, j)
covers u in [i, i+1) and v in [j, j+1), so its center is at (i + 0.5,
j + 0.5).
### Why the GPU sees camera-relative coordinates
GPUs compute in 32-bit floats, which carry about 7 significant digits. A
spacecraft 7000 km from Earth's center, stored in Earth-centered f32
coordinates, can only be placed to the nearest 0.5 m. A target 10 m away
would jitter by several pixels. So all positions stay in 64-bit floats on
the CPU. They are made relative to the camera there, and converted to
32-bit only after that. A target 10 m from the camera is then placed to
about a micrometer.
### Depth
When two things overlap in the image, the nearer one must win. The GPU
keeps a depth value per pixel for this. The renderer stores near / z,
which is 1 at the near clip distance and falls toward 0 with distance.
Combined with how floating-point numbers are spaced, this gives roughly the
same relative precision at every distance. That matters in a scene that
contains a 10 m target, Earth at thousands of kilometers, and the Sun at
150 million kilometers. The scheme is called reversed-Z with an infinite
far plane.
## Radiometric quantities
Code: `spectrum.rs`, `radiometry.rs`.
Light has a few distinct measures, and the camera measures a specific one.
- **Irradiance, E, in W/m^2.** Power arriving per unit area of a surface.
Sunlight at 1 AU delivers about 1361 W/m^2 to a surface facing the Sun.
It falls off with the square of the distance from the Sun. A surface
tilted by angle theta from the Sun direction receives E cos(theta).
- **Solid angle, in steradians (sr).** The 2D version of an angle: the area
a cone cuts out of a unit sphere. The full sphere is 4 pi sr. One pixel
of the example camera covers (110 urad)^2, about 1.2e-8 sr.
- **Radiance, L, in W/m^2/sr.** Power per unit area per unit solid angle,
traveling in a particular direction. It is what a camera pixel responds
to: how bright a surface looks from a given direction.
Radiance has a property that explains a lot of what a camera sees: it does
not change along a ray through empty space. Move a camera twice as far from
a wall, and each pixel sees four times as much wall area. But each square
meter now contributes a quarter of the power, so the pixel value is
unchanged. An object that covers many pixels has the same surface
brightness at any distance; it just covers fewer pixels farther away.
That stops being true once the object is smaller than a pixel. The pixel
then also sees dark space around the object, so the signal falls as one
over distance squared. That is the regime of a distant satellite seen as a
point (space situational awareness), as opposed to the resolved imagery
this renderer is mostly for.
### Reflection: albedo and Lambertian surfaces
A Lambertian surface reflects light equally in every direction. Matte
paint, the Moon (roughly) and cloud tops behave close to this. If such a
surface receives irradiance E and reflects a fraction rho (its albedo), its
radiance in every direction is:
```text
L = rho * E / pi
```
The pi comes from integrating the reflected radiance over the hemisphere
above the surface. A white Lambertian sheet facing the Sun at 1 AU, with
rho = 1, has a radiance of about 1361 / pi = 433 W/m^2/sr, summed over all
wavelengths. Earth's average albedo is about 0.3.
Real spacecraft surfaces are not Lambertian. Multilayer insulation (MLI)
and solar cells are partly mirror-like and produce bright glints at
particular angles. These are modeled by a BRDF, a function giving reflected
radiance for every pair of incoming and outgoing directions. That arrives
in M4.
## Wavelength: spectra, channels and photons
Code: `spectrum.rs` (`Spectrum`), `camera.rs` (`Channel`).
Everything above varies with wavelength. Sunlight peaks around 500 nm and
extends from the ultraviolet into the infrared. Surfaces reflect different
wavelengths differently; that is what color is. Optics transmit some
wavelengths better than others. The sensor's sensitivity also depends on
wavelength.
The renderer describes each of these as a `Spectrum`: a value as a function
of wavelength, in meters.
- **Quantum efficiency (QE).** The fraction of arriving photons that become
electrons in a pixel. A color camera puts red, green and blue filters
over different pixels, so each channel has its own QE curve. The example
camera uses idealized Gaussian curves centered at 620, 540 and 460 nm,
80 nm wide at half height, peaking at 60 %.
- **Transmission (tau).** The fraction of light the lenses pass, about 0.9
for the example camera.
Detectors count photons, not energy. A photon of wavelength lambda carries
energy h c / lambda (h is Planck's constant, c the speed of light). So one
watt of red light delivers more photons than one watt of blue. The
renderer converts power to photons per wavelength before applying QE.
### The solar spectrum
Code: `radiometry.rs` (`solar_spectrum`), generated into `solar_data.rs` by
`scripts/fetch-solar-spectrum.sh`.
The renderer uses a measured solar spectrum, the TSIS-1 Hybrid Solar
Reference Spectrum (Coddington et al. 2021). It is the current
international reference, built mainly from the TSIS-1 instruments on the
International Space Station and filled in with a high-resolution model of
the solar absorption lines. It gives the irradiance at 1 AU, outside the atmosphere,
every 1 nm from 202 to 2730 nm. Its uncertainty is 0.3 % from 460 to
2365 nm. Over that range it sums to 1325.8 W/m^2. The other 35 W/m^2 of the
1361 W/m^2 total is at wavelengths silicon detectors do not see.
The Sun is often approximated as a 5772 K blackbody: a smooth curve with
the right total power. The real spectrum has thousands of absorption lines
from the Sun's outer layers, and its shape differs from the smooth curve.
For the example camera's channels, the blackbody gives 4.6 % (red), 7.5 %
(green) and 9.7 % (blue) fewer electrons than the measured spectrum. That
is far outside what absolute radiometry can tolerate.
### Folding spectra into per-channel constants
Code: `radiometry.rs` (`Radiometry`).
Doing all of this wavelength by wavelength for every pixel would be slow.
Most factors do not depend on the pixel: the solar spectrum, the optics,
and the QE curves. The renderer multiplies and integrates those over
wavelength once, when the camera is set up. That yields a few numbers per
channel, and the GPU only multiplies by them.
The central number is the solar flux of each channel k:
```text
S_k = integral of E_sun(lambda) * tau(lambda) * QE_k(lambda) * lambda / (h c) d lambda
```
It is the rate of electrons per square meter that sunlight at 1 AU would
produce if it fell straight onto the detector through the optics. For the
example camera, S_k is about 2.4e20 electrons per second per m^2 in red and
green, and 2.0e20 in blue.
The renderer then works in channel radiance: scene radiance weighted the
same way (transmission, QE, photons per joule), in electrons per second
per m^2 per steradian. A Lambertian surface with albedo rho, lit at angle i
from its normal, at distance d from the Sun in AU, has channel radiance:
```text
L_k = rho * cos(i) * S_k / (pi * d^2)
```
The solar disk has channel radiance S_k / Omega_sun, where Omega_sun is the
Sun's solid angle at 1 AU, 6.79e-5 sr (it is 0.53 degrees across). Seen
from anywhere, the Sun's surface has that same radiance, about 2.0e7
W/m^2/sr in total. Looked at directly, it puts about 5e8 electrons into a
pixel of the example camera's green channel in 0.1 ms, 26,000 times the
full well.
A surface whose reflectance varies with wavelength is reduced to an
effective albedo per channel. That is the albedo of a flat-spectrum surface
that would give the same signal under sunlight
(`Radiometry::effective_albedo`). This step is where the error of the
shortcut comes in. Two spectra that both vary inside one channel's band
interact in ways a single number per channel cannot capture. One example is
a surface whose reflectance changes steeply across green, lit partly by
sunlight and partly by bluish light scattered from Earth's atmosphere.
Rendering more, narrower channels reduces that error, and the code keeps
the channel count generic for that reason.
## From radiance to electrons: the camera equation
Code: `radiometry.rs` (`Radiometry::electrons`); `gpu/sensor.rs`, `shaders/sensor.wgsl`.
A lens with focal length f and aperture diameter D has f-number
N = f / D. The example camera is f/4: a 50 mm lens with a 12.5 mm aperture.
For a scene radiance L, the irradiance on the focal plane at the image
center is:
```text
E_image = pi * L * tau / (4 N^2)
```
The aperture collects light over a solid angle proportional to (D / f)^2 =
1 / N^2. So opening the aperture one "stop", which divides N by the square
root of 2, doubles the light. Image irradiance is independent of distance
for the same reason radiance is.
Away from the center, image irradiance falls off as cos^4(theta), where
theta is the angle from the boresight. Two factors come from the longer
path to the off-axis pixel. One comes from the aperture appearing
foreshortened. One comes from light reaching the pixel at a slant. At the
corner of the example camera, theta is 6.9 degrees, and the falloff is
3 %. Real lenses add more falloff of their own, called vignetting.
The electrons a pixel collects in one channel are then:
```text
electrons = E_image * (pixel area) * (exposure time) * (photons per joule) * QE
```
The last two factors are wavelength-dependent. In channel radiance they are
already folded in, so the camera equation reduces to
```text
electrons = L_k * pi * (pixel area) * (exposure time) / (4 N^2) * cos^4(theta)
```
For the example camera, pi * A * t / (4 N^2) is 1.48e-16 m^2 sr s.
## From electrons to DN: the detector
Code: `camera.rs` (`Detector::dn`); `gpu/sensor.rs`, `shaders/sensor.wgsl`.
- **Full well.** The maximum number of electrons a pixel holds. Beyond it
the pixel saturates and reads its maximum value whatever the light.
Brighter scenes do not look brighter. In real sensors, excess charge can
also spill into neighboring pixels (blooming, M3).
- **Gain.** Electrons per DN. With a gain of 5, every 5 electrons add 1 to
the output.
- **Bias.** A fixed offset added to every pixel, so that noise around zero
signal does not get clipped at 0. The example uses 64 DN.
- **ADC bits.** The output resolution. 12 bits gives values 0 to 4095.
So DN = bias + min(electrons, full well) / gain, rounded and clipped to the
ADC range. The renderer does this on the GPU for every pixel and channel,
as the last step of a frame, and reads back only the resulting integers.
Noise (M3) will enter between the electrons and the gain.
Two consequences of these numbers show up in every image:
- **Saturation is not always the ADC's maximum.** In the example camera the
full well fills first: 20,000 / 5 + 64 = 4064 DN, below the 12-bit maximum
of 4095. A saturated pixel there reads 4064. Real cameras are designed
either way, and an algorithm that detects saturation by looking for 4095
would miss it in this one.
- **Black is not zero.** A pixel that receives no light reads the bias,
64 DN, so empty space in a raw frame displays as dark gray.
Raw frames are also not white-balanced. The camera reports what each
channel collected, and the channels collect different amounts from the same
gray surface. In the example camera, sunlight on a gray surface gives about
15 % fewer electrons in blue than in red, so gray things look slightly
yellow-green in a raw image. A white balance step, applied after capture,
would scale the channels to make them equal. The renderer leaves it out,
because it simulates what the sensor records.
### Worked example
Take a white Lambertian surface facing the Sun at 1 AU, seen by the example
camera's green channel. Integrating the solar spectrum through the green QE
curve, f/4 optics, 5.5 um pixels and a 0.1 ms exposure gives 11,139
electrons. Red gets 11,408 and blue 9,626. (A test in `radiometry.rs`
checks the green figure.)
- DN = 64 + 11,139 / 5, about 2292, a little over half the 12-bit range.
- The full well is 20,000 electrons, so this surface is at 56 % of
saturation.
- Earth, with albedo near 0.3, reads about 64 + 3,342 / 5, about 732 DN.
- The same white surface at the image corner, 6.9 degrees off axis, gets
cos^4 of that, 97 %: 10,819 electrons, 2228 DN.
The sensor tests check the 2292, 2228, 732, 4064 and 64 DN figures
exactly, on the GPU and on lavapipe.
Picking the exposure time is the same trade a photographer makes. A longer
exposure saturates the sunlit target. A shorter one leaves the side of the
target lit only by Earthshine lost in noise once noise is modeled (M3).
## How shapes become pixels
Two techniques turn 3D shapes into an image. The renderer uses one for the
bodies and the other for the target.
- **Mesh.** A surface approximated by many small flat triangles that share
corners (vertices). A cube needs 12 triangles. A curved surface needs
many more, and it is still faceted up close. Each vertex stores a position
and usually a normal, the direction the surface faces there. STL and glTF
files are meshes, and so is the target spacecraft.
- **Rasterization.** The GPU's built-in way of drawing a mesh. Each
triangle's corners are projected into the image with the pinhole
equations above. The GPU then finds every pixel whose center falls inside
the projected triangle, and runs a small program (a fragment shader) for
each of those pixels to compute its value. It is fast because it only
touches the pixels each triangle covers, and GPUs have dedicated hardware
for it. It can only draw triangles.
- **Ray casting.** The reverse direction: for each pixel, follow the line
of sight from the camera through the pixel center and solve for where it
first meets a surface. For a sphere or an ellipsoid that is a quadratic
equation with an exact answer, so there are no triangles and no facets.
Every pixel does the work, whether or not it ends up seeing the body.
The bodies use ray casting, because their shapes have exact equations and
their edges are what limb-based navigation measures. The target uses
rasterization, because a spacecraft model has no equation, only its mesh.
Both write into the same depth image, so they sort against each other
correctly. Ray casting a mesh is also possible, by testing rays against
its triangles. That is what M5's ray tracer does, because following rays
gives exact shadows and light bouncing between surfaces. Rasterization can
only approximate those, for example with the shadow maps lighting L1 uses
(below).
### One sample per pixel
Both techniques decide each pixel's value from one point, the pixel's
center. A real pixel collects light over its whole area, about 5.5 um
square on the detector. Where a pixel straddles an edge, say half on a
sunlit panel and half on dark space, the real pixel reads about half the
panel's value. The renderer reads either the full value or none, depending
on which side the center falls.
This shows in two ways. Edges are stair-stepped instead of graded. And
anything thinner than a pixel, like the booms of the example satellite seen
from 40 m, breaks into dashes, because only some pixel centers land on it.
For an algorithm that finds edges or measures brightness at sub-pixel
precision, the graded edge is the signal, so this matters for limb
navigation and pose estimation.
The usual fix is supersampling: shading several points inside each pixel
and averaging. The GPU's multisampling hardware does this cheaply for
rasterized edges. Sampling the bodies' limbs needs it in the ray casting
shader. The lens blur that comes with the point spread function (M3) also
spreads every edge over a few pixels.
## Earth, Moon and Sun
Code: `bodies.rs`, `gpu/bodies.rs`, `shaders/bodies.wgsl`.
The three bodies are not drawn as triangle meshes; they are ray cast. For
every pixel, the shader sends a ray from the camera through the pixel's
center and solves for where it meets each body's exact shape. The nearest hit wins. This
gives an exact edge at any zoom, at the cost of a few arithmetic operations
per pixel. A mesh would show its facets along the limb, the edge of the
disk, which is the feature horizon-based navigation measures.
- **Earth** is the WGS84 ellipsoid: 6378.137 km equatorial radius,
6356.752 km polar. The 21 km difference matters at fine pixel scales. A
camera with 5.5 urad pixels at 200,000 km sees the north limb 19 pixels
away from where a spherical Earth would put it. A test checks the limb
against the exact tangent to the ellipsoid to within a pixel.
- **The Moon** is a sphere of 1737.4 km radius. From Earth's distance it
is about 0.52 degrees across, 82 pixels in the example camera.
- **The Sun** is a sphere of 695,700 km radius: 0.53 degrees across, about
85 pixels in the example camera. Its disk has the solar radiance from the
folding section above, and it is drawn only where no nearer body covers
it. So Earth eclipses it naturally.
### Shading
Earth and the Moon are Lambertian, each with one reflectance spectrum over
the whole body, set by `Appearance`. The Moon's is a flat 0.12. Earth's
default is a stand-in shaped like cloud-free ocean seen from space: about
0.3 in the blue falling to 0.07 in the red, then back up in the near
infrared. Most of that blue is sunlight scattered by the atmosphere above
the ocean, the same scattering that makes the sky blue; the ocean itself is
dark. In the example camera it gives channel albedos of 0.09, 0.15 and 0.26
(red, green, blue), so Earth renders blue. Its overall brightness is below
the real Earth's average of about 0.3, which includes bright clouds.
Each point's radiance is rho cos(i) S_k / (pi d^2), with i the angle between
the surface normal and the Sun. The night side has i past 90 degrees and is
black. The Sun's distance d is taken from each body's center, since the
difference across a body is a few parts in 10^5.
The total light a sphere sends a far-away camera has a closed form. A
Lambertian sphere of albedo rho and radius R, at distance D, delivers an
irradiance of (2/3) rho E (R/D)^2 Phi(alpha), where alpha is the phase angle
(Sun-sphere-camera) and Phi(alpha) = (sin alpha + (pi - alpha) cos alpha) / pi.
It falls from 1 at full phase to 0.61 at 60 degrees and 0.11 at 120 degrees.
Up close it is not exact: the near side of the sphere is nearer the camera
than its center, so it counts for more. At 40,000 km from the Moon (R/D =
0.043), the real total is 3 % above the formula. The renderer's image agrees
with the exact surface integral to 0.03 % or better (`tests/analytic.rs`).
This gets the overall brightness and the phase (how much of the disk is
lit) right. It misses some things that are visible in real images:
- Earth's oceans, land and clouds differ in brightness by a factor of
several. From lighting L1, Earth's surface comes from an image (next
section); clouds come in M4.
- The full Moon looks almost uniformly bright out to its edge. A Lambertian
sphere instead darkens toward the edge, because the light there arrives
and leaves at grazing angles. Lunar soil scatters light back toward its
source much more strongly than Lambert predicts, and a dedicated lunar
reflectance model (M4) fixes that. The Moon's maria (the dark plains of
old basalt lava, the "seas" seen with the naked eye) and its bright
highlands also differ in albedo by about a factor of two (M2 albedo
map).
- Earth's atmosphere softens the limb and adds a thin bright band around it
(M4). Here Earth's edge is a sharp line.
### Earth's surface
Code: `earth.rs`, `shaders/bodies.wgsl`; data from `scripts/fetch-earth.sh`.
At lighting tier L1 (`RendererOptions::fidelity`, with the image in
`RendererOptions::earth_texture`; `--lighting L1` on the command line), Earth's
reflectance comes from an image of the whole surface instead of one spectrum. The image is NASA's
Blue Marble Next Generation: a cloud-free mosaic of the 2004 MODIS satellite
record, one image per month, so snow cover and vegetation follow the season.
The CLI picks the month from the scene's epoch.
The image is equirectangular: each column is a longitude and each row a
latitude. For every pixel that sees Earth, the shader finds the surface
point's geodetic latitude and longitude from the ellipsoid's normal there, in
Earth-fixed axes, and reads the image at that place. Earth's orientation from
the scene turns the continents with the epoch.
The full image is 500 m per pixel at the equator, resized to 660 m to fit the
largest texture lavapipe allows (16384 x 8192). Whether that is fine or coarse
depends on the view:
- From low orbit looking straight down, a camera pixel of the example camera
covers 77 m of ground, so each texture pixel spans about 8 image pixels and
looks soft.
- From geostationary orbit, a camera pixel covers about 4 km, so each image
pixel averages about 35 texture pixels (6 x 6).
For the second case the texture is stored with mipmaps: copies at half, a
quarter, an eighth of the size, and so on. The GPU reads from the copy whose
pixels best match the image pixel, so a far view shows the true average color
of a region rather than a shimmering sample of individual texture pixels.
Two details matter for brightness:
- Averaging has to happen on reflectance, not on the encoded values image files
store (sRGB, which compresses the darks). Averaging encoded black and white
gives a reflectance of 21 %, not 50 %. So the texture is decoded to linear
reflectance once at load and kept in 16-bit floating point, and both the
mipmaps and the GPU's blending between neighboring texture pixels work on
those values. A coastline seen up close blends correctly from sea to land.
- Blue Marble is surface reflectance with no atmosphere, and open ocean is
dark, a few percent. From orbit the ocean looks bright blue because the air
above it scatters sunlight toward the camera: Rayleigh scattering, which
grows as 1 / wavelength^4, which is also why the sky is blue. Until the
atmosphere is modeled (M4), a stand-in adds a reflectance of that shape,
0.07 at 450 nm falling to 0.017 at 650 nm (`Appearance::earth_haze`). It
brightens and blues the whole lit disk evenly, where the real effect grows
toward the limb, where the line of sight crosses more air.
The texture's red, green and blue are reflectances in the same three bands a
glTF base color uses (see the target model section), folded into the camera's
channels once at setup as a small matrix, so any channel layout works.
Blue Marble is a visual product built from calibrated MODIS reflectances, not a
calibrated reflectance product itself, so its values are close but not exact
reflectances. A calibrated alternative (MODIS's albedo product) is a possible
later refinement.
### Depth from the bodies
The pass also writes each hit's depth, near / z, as described in the
geometry section. The target spacecraft, drawn after the bodies, uses the
same depth buffer. So a target in front of Earth covers it, and a target
behind Earth's limb is hidden.
## The target model and its materials
Code: `model.rs`.
The target spacecraft is a triangle mesh loaded from STL or glTF. Each
piece (an STL file, or a glTF primitive) is a part with one material, and
for now a material is a Lambertian albedo spectrum, as for Earth and the
Moon.
Surface normals decide how bright each triangle is, through the cos(i)
factor. STL files store a normal per triangle, but exporters often write
zeros or wrong values, so the loader ignores them. It recomputes each
normal from the order of the triangle's corners: counterclockwise as seen
from outside. Every triangle is then lit as a flat facet, which matches the
flat panels and boxes most spacecraft are built from. glTF files that store
normals keep them.
glTF describes a material's color as red, green and blue reflectances (its
"base color"). Three numbers do not define a spectrum: very different
spectra can look the same color to the eye. The loader turns them into a
stepped spectrum, blue below 490 nm, green from 500 to 580 nm, and red above
590 nm. That is a reasonable guess for painted surfaces seen by an RGB
camera. It is a poor one for a camera whose channels differ from the eye's,
or for anything in the near infrared, where the red value simply continues.
For absolute radiometry of a real spacecraft, give the important materials
measured reflectance spectra (M4 adds those, and BRDFs).
Texture images are not used yet; each material renders with its base color
value alone. The loader removes textures before reading a glTF file, so a
model still loads when its textures are in a format or file the loader cannot
read. A model that relies on textures for its colors (one material with a
painted image) renders in that material's flat base color, often gray.
STL has no units. CAD tools usually export millimeters, so loading a CAD
model usually needs `scale: 0.001`. glTF is meters by its specification, but
models in the wild do not always follow it: of the NASA example models,
Landsat 8 and ICESat-2 are in meters, while the SSL-1300 needs a scale of 0.1
and ACE about 0.005. Checking a model's size against the real spacecraft's is
worth the minute it takes.
## Drawing the target
Code: `gpu/mesh.rs`, `shaders/mesh.wgsl`.
The target's mesh is rasterized after the bodies, into the same radiance
and depth images. Each pixel the target covers is kept only if the target
is nearer than what is already there. That one rule handles a target in
front of Earth, a target behind Earth's limb, and parts of the target
hiding other parts.
Each point on the target is shaded like the bodies: albedo times cos(i)
times the sunlight that reaches it. A panel modeled with no thickness, such
as a solar array, has one surface but two sides. Whichever side faces the
camera is the side that is lit. A closed shape never shows its inside, so
this only matters for thin parts.
### Shadows on the target
Code: `shadow.rs`, `gpu/shadow.rs`, `shaders/mesh.wgsl`. From lighting L1.
A solar array shades the bus behind it, an antenna dish shades its feed,
and a second spacecraft close by can shade the first. Rasterization draws
each triangle on its own, so by itself it cannot know that something else
stands between a point and the Sun. The renderer finds out with a shadow
map.
- **Shadow map.** Before the camera's view is drawn, the target is drawn
once more as the Sun sees it, looking straight down the Sun's rays. That
drawing keeps only depth: for each texel (a pixel of the map), how far
along the Sun's direction the first surface is. The Sun is far enough
away that its rays are parallel across a spacecraft, so the view has no
perspective, and a map 4096 texels across covers the target's bounding
sphere. For Landsat 8, whose sphere is 16.6 m across, one texel is 4 mm.
- **Lookup.** When the camera's view shades a point, it finds that point
in the map. If the map holds a surface nearer the Sun than the point, the
point is in shadow.
Each target gets its own map. Another target is drawn into that map when it
is sunward of this one and close enough across the Sun's rays to shade it.
A target off to the side is left out, since it cannot cast a shadow here.
The Sun is a disk half a degree across, so a shadow edge is not sharp. A
point near the edge sees part of the disk past the blocker, the same
penumbra as in an eclipse. The width of the penumbra is the distance from
blocker to surface times the tangent of the Sun's angular diameter, 0.93 %
at 1 AU. A solar array 2 m above the bus casts an edge 19 mm wide. One 10 m
away casts an edge 9 cm wide. The renderer reproduces this with
percentage-closer soft shadows:
1. Search the map around the point for blockers, and average their depth.
That gives the blocker-to-surface distance, and from it the penumbra
width.
2. Compare the point's depth against a 6 x 6 grid of map texels spread over
that width, and take the fraction that are not blocked as the share of
the Sun the point sees.
Both steps use fixed grids rather than random samples, so the image has no
noise and the same scene always gives the same pixels.
A surface can wrongly shadow itself, because the map only stores its depth
once per texel. Where the surface is tilted to the Sun, depth changes
across a texel, and half of each texel looks like it is behind itself. The
result is a pattern of dark stripes called shadow acne. The renderer
prevents it three ways. The stored depth is pushed away from the Sun in
proportion to the surface's tilt. The lookup is made from a point 1.5
texels off the surface, toward the Sun. And a blocker has to be more than 2
texels nearer the Sun to count.
Tests (`tests/shadows.rs`) check, on the GPU and lavapipe:
- A plate under a 2 m square: the shadow reads zero direct light, and its
edge falls where the square's edge projects, to within one texel.
- The measured penumbra width is within 2.2 % of 0.93 % of the occluder's
height, at heights of 1, 2 and 3 m.
- No pixel darkens compared with L0 on a plate tilted 80 and 87 degrees
from the Sun, or on a faceted sphere.
- A target sunward of another shades it; one to the side does not.
On the RTX 3060, shadows cost 0.14 ms of GPU time per frame for Landsat 8
(85,000 triangles) and 0.34 ms for ICESat-2 (752,000), plus about 0.15 ms
in the camera view's lookups. Maps 2048 texels across save about 0.15 ms.
Limitations:
- **Penumbra shape.** The grid averages a square of the map, so brightness
rises in a straight line across the penumbra. A real disk-shaped Sun
rises along an S-shaped curve: the full width is the same, but the middle
80 % of the change is 0.69 of the width rather than 0.8.
- **Resolution.** A detail smaller than a texel casts no reliable shadow. A
gap narrower than about 2 texels between a blocker and the surface it
shades (a few mm on Landsat 8) lets light through, because of the acne
margins above.
- **One sample per pixel.** Shadows are sampled at the pixel center, like
everything else, until M3's pixel-area sampling.
- **Bodies.** Earth and the Moon do not use shadow maps. Their shadows on
the target are the eclipse factor below, applied to the whole target.
Light from Earth and the Moon (below) is not shadowed at all.
### Eclipses
Code: `eclipse.rs`.
The Sun is a disk, not a point. So as a spacecraft passes into Earth's
shadow, the light fades over a few seconds instead of switching off:
- **Umbra.** Earth covers the whole solar disk. No direct sunlight.
- **Penumbra.** Earth covers part of the disk. 7000 km from Earth's
center, on the night side, the penumbra is about 65 km wide; a satellite
crosses it in under 10 seconds.
- **Annular.** A body small enough to sit inside the solar disk blocks a
ring-shaped bite. From orbit this happens with the Moon.
The renderer finds the visible fraction of the solar disk from the overlap
of two circles on the sky. The circles are the Sun's disk and the
occulter's, each with its apparent angular radius. It multiplies the
target's sunlight by that fraction. The model is the standard one for
satellites (Montenbruck and Gill), the same as astrokit's. A test checks it
against tracing lines from the target to thousands of points spread over
the solar disk.
The renderer counts both Earth and the Moon as occulters. The fraction is
computed once per target, at its center. The whole target
dims together, which is accurate unless the target is a sizable fraction of
the penumbra's width. Light reaching the target from Earth itself is the
next section.
### Light from Earth and the Moon
Code: `gpu/earthshine.rs`, `shaders/earthshine.wgsl`, `shaders/mesh.wgsl`.
From lighting L1.
Earth reflects about a third of the sunlight that reaches it, and some of
that lands on the target: Earthshine. From low orbit it is large. At 500 km,
Earth fills a circle 68 degrees in radius around the point straight down,
3.9 of the sky's 12.6 steradians. A plate facing straight down over the
sunlit Earth receives up to about a quarter as much light from Earth as a
plate facing the Sun receives from the Sun. A white Earth would give 0.85
(the test below measures that case). At geostationary distance, Earth is 17
degrees across and a plate facing a full Earth gets about 0.5 % of sunlight.
The Moon does the same, Moonshine, much more weakly: a full Moon seen from
near Earth gives about a millionth of sunlight. From 100 km above the Moon,
though, a plate facing it gets about a tenth of sunlight.
This matters most where the Sun does not reach: the target's night side,
and parts in a shadow of its own. Without it those read as black, the bias in
DN; with it they show their structure, dimly.
The renderer works out, for each target and each frame, how much light
arrives from Earth and the Moon on a surface facing any direction:
1. **What the target sees of each body.** The circle on the sky that the body
fills is cut into 1024 cells of equal solid angle: 32 rings from the
center out to the edge, 32 cells around each. A line from the target
through each cell's center finds the point it meets on the body, and that
point is shaded exactly as the camera's view of the body is shaded:
reflectance from the Blue Marble image (or the stand-in spectrum), lit by
the cosine of the Sun's angle there, dark on the night side. Each texture
read averages the patch of ground the cell covers, about 1000 square
kilometers straight down from 500 km and more toward Earth's edge, so a
cell sees the mean color of that patch rather than one point. A line
that meets the other body first counts nothing, so the Moon can hide part
of Earth and Earth part of the Moon.
2. **Light for every direction a surface can face.** For a surface facing
direction n, the light it receives is the sum over all cells of the cell's
brightness, times its solid angle, times the cosine between n and the
cell's direction (zero for cells behind the surface). The renderer adds
this up for a grid of directions, a cube map 16 texels across on each of
its six faces.
Drawing the target, each point reads that cube at its surface normal and
adds albedo / pi times the result to the direct sunlight. The cube is computed
at the center of each target's bounding sphere and used for the whole target;
across a spacecraft the view of Earth barely changes.
Earthshine is not dimmed by the target's eclipse factor, because the two
are different views of Earth. From anywhere in Earth's umbra only the night
side is in view, so Earthshine is zero there without being told. In the
penumbra the Sun is partly hidden but a crescent of lit ground is still in
view, and that light still arrives.
Tests (`tests/earthshine.rs`) compare a plate's brightness on the GPU and
lavapipe against the same sum done in 64-bit floating point with 90 times as
many cells: over the subsolar point at 500 km facing down, sideways and up;
toward the terminator; in Earth's shadow; at GEO; 100 km above the Moon at
full and quarter phase; with each body hiding the other; with the plate seen
from several sides, so each cube face a camera-facing surface reads is checked. Every
case where the bodies send more than a thousandth of sunlight agrees to
within 0.5 %. The largest differences come from the cube: between texels
5.6 degrees apart its interpolation rounds off the peak of a surface facing
a full Earth or Moon, by 0.4 %. A cube 32 texels across cuts that to 0.13 %
but costs four times as much, and the cells' own error on a half-lit distant
Earth (0.4 %) stays. A thin crescent of lit ground at Earth's edge, seen from
low orbit near the terminator, is off by 4 % of a small amount, 7e-5 of
sunlight, about 0.1 DN on a gray surface in the example camera. Another test
checks the sum itself against the classical formula for a distant sphere,
(2/3) albedo (R/D)^2 Phi(alpha), to within 0.3 %.
On the RTX 3060 this costs 0.04 ms of GPU time per target (0.28 ms for
eight), and 1.4 ms per target on lavapipe.
Limitations:
- **No shadows.** The target's own parts do not block light from Earth or
the Moon. A panel facing Earth does not shade the bus behind it from
Earthshine, and the inside of a dish facing Earth is lit as if open to the
whole Earth. Earth fills too much of the sky for one shadow map to stand in
for it; the ray tracer (M5) handles this.
- **One spectrum per channel.** Light from Earth is bluer than sunlight, but
the target's albedo in each channel was worked out under sunlight. This is
the shortcut described in "Folding spectra into per-channel constants",
and its error is largest for strongly colored surfaces.
- **Lambertian bodies.** Earth and the Moon reflect as Lambertian surfaces
here, as they do in the camera's view. The Moon's real reflection is
brighter toward the Sun (M4), and Earth has no clouds or atmosphere yet
beyond the stand-in haze.
- **No eclipses on the bodies.** Earth's shadow on the Moon and the Moon's
shadow on Earth are not modeled yet (M2), so a Moon in eclipse still lights
the target.
## Where the Sun and Moon appear
Code: `ephemeris.rs` (feature `astrokit`).
With the `astrokit` feature, the Sun's and Moon's positions come from JPL's
DE440s ephemeris, and Earth's orientation from the ITRF93 kernel, which
includes the slow wobble of Earth's pole. These are geometric positions:
where each body is at the epoch.
A camera sees something slightly different, its apparent position:
- **Light time.** Light from the Moon takes 1.3 s to reach Earth, and from
the Sun 8.3 minutes. The camera sees where each body was when the light
left it. For the Moon that is about 1.3 km of motion, a few microradians.
- **Aberration.** A moving camera sees incoming light tilted toward its
direction of motion, as rain seems to slant toward a moving car. Earth's
orbital speed (30 km/s) tilts the Sun's apparent direction by about 20
arcseconds, 100 microradians. A spacecraft's own orbital velocity adds up
to about 5 arcseconds more.
100 microradians is about one pixel of the example camera, and 18 pixels of
the narrow 1 m focal length camera in the limb test. So the geometric
positions are close enough for M1, but not for precise navigation against
the Sun, Moon or stars. Rendering apparent positions needs the observer's
velocity as an input, which the scene does not carry yet.
## Not modeled yet
| Effect | Where |
|---|---|
| Eclipses seen on the bodies: Earth's shadow on the Moon, the Moon's on Earth | M2 |
| Earthshine on the Moon's night side | M4 |
| The Sun as a disk that is darker toward its edge (limb darkening) | M2 |
| Pixel-area sampling (anti-aliasing): graded edges, thin parts | M3 |
| Lens distortion | M3 |
| Blur from diffraction and lens imperfections (point spread function) | M3 |
| Noise: photon shot noise, read noise, dark current, pixel-to-pixel variation | M3 |
| Saturation spilling into neighbors (blooming) | M3 |
| Non-Lambertian materials (BRDFs), glints, lunar backscatter | M4 |
| Moon albedo map | M2 |
| Clouds on Earth | M4 |
| Earth's atmosphere: scattering, the bright limb | M4 |
| Motion blur, rolling shutter, stray light | M6 |
| Apparent positions: light time and aberration | M3 |