2338 lines
127 KiB
Markdown
2338 lines
127 KiB
Markdown
# Optics and radiometry guide
|
|
|
|
This explains the physics the renderer models, for a reader who knows
|
|
orbital mechanics but not optics. Each section says which code implements
|
|
it. When a feature adds a new optical effect, it adds or updates a section
|
|
here in the same change.
|
|
|
|
Numbers in the examples come from `CameraConfig::example()` unless stated
|
|
otherwise: 1920 x 1080 pixels, 5.5 um pitch, 50 mm focal length, f/4, three
|
|
color channels, 12-bit output.
|
|
|
|
## The chain from Sun to number
|
|
|
|
Every pixel value comes from the same sequence:
|
|
|
|
1. The Sun emits power across a range of wavelengths.
|
|
2. Some of it lands on a surface (Earth, the Moon, the target), which
|
|
reflects part of it in various directions.
|
|
3. The fraction reflected toward the camera enters the lens aperture. The
|
|
optics focus it onto the detector, losing a little to absorption. Each
|
|
point of the scene lands as a small spot rather than a point.
|
|
4. The pixel converts some of the arriving photons into electrons. The
|
|
fraction depends on wavelength and on the color filter over that pixel.
|
|
5. The electrons pile up during the exposure, up to a limit.
|
|
6. The readout electronics turn the electron count into an integer, the
|
|
digital number (DN).
|
|
|
|
The renderer follows this chain with physical units at every step. Image
|
|
brightness therefore comes from the scene and the camera settings, not from
|
|
a tuned scale factor. Change the exposure time, and the image changes
|
|
exactly as a real camera's would.
|
|
|
|
## Geometry: the pinhole camera
|
|
|
|
Code: `camera.rs` (`Intrinsics`), `scene.rs`.
|
|
|
|
A pinhole camera maps each direction in space to one point on the focal
|
|
plane. Put the pinhole at the origin, looking along +Z. A point at camera
|
|
coordinates (x, y, z) lands on the focal plane at (f x / z, f y / z), where
|
|
f is the focal length. Dividing by the pixel pitch p converts focal-plane
|
|
distance to pixels:
|
|
|
|
```text
|
|
u = (f / p) * x / z + cx
|
|
v = (f / p) * y / z + cy
|
|
```
|
|
|
|
`f / p` is the focal length in pixels. (cx, cy) is the principal point,
|
|
where the boresight hits the image, normally the image center. For the
|
|
example camera, f / p = 50 mm / 5.5 um = 9091 pixels.
|
|
|
|
Some quantities that follow from that:
|
|
|
|
- **IFOV, the instantaneous field of view.** The angle one pixel covers:
|
|
p / f = 110 microradians. At 10 m that is 1.1 mm per pixel; at 7000 km
|
|
it is 770 m per pixel. This sets the smallest detail the camera can
|
|
resolve.
|
|
- **FOV, the field of view.** The angle the whole image covers:
|
|
2 atan(width / 2 / (f / p)), about 12.1 x 6.8 degrees here. A longer
|
|
focal length or smaller sensor gives a narrower field of view and finer
|
|
IFOV.
|
|
|
|
Real lenses are not perfect pinholes. They bend straight lines slightly
|
|
(distortion, next) and spread each point into a small blur (the point
|
|
spread function, below). The pinhole model is the starting point those
|
|
effects are added to.
|
|
|
|
### Lens distortion
|
|
|
|
Code: `camera.rs` (`BrownConrady`, `Intrinsics`); `gpu/warp.rs`,
|
|
`gpu/blur.rs`, `shaders/blur.wgsl`, `shaders/distortion.wgsl`.
|
|
|
|
A real lens moves each point a little away from where the pinhole puts it,
|
|
by an amount that grows toward the edge of the image. Straight lines in the
|
|
scene come out curved. When points move outward more the farther out they
|
|
are, the corners stretch (pincushion distortion); when they move inward,
|
|
the image bulges (barrel distortion). A lens element slightly off center
|
|
adds a smaller sideways shift (tangential distortion).
|
|
|
|
The standard model, Brown-Conrady, works on the pinhole's normalized
|
|
coordinates x = X / Z and y = Y / Z, with r^2 = x^2 + y^2:
|
|
|
|
```text
|
|
x_d = x (1 + k1 r^2 + k2 r^4 + k3 r^6) + 2 p1 x y + p2 (r^2 + 2 x^2)
|
|
y_d = y (1 + k1 r^2 + k2 r^4 + k3 r^6) + p1 (r^2 + 2 y^2) + 2 p2 x y
|
|
u = (f / p) x_d + cx, v = (f / p) y_d + cy
|
|
```
|
|
|
|
k1, k2 and k3 are the radial terms, positive for pincushion and negative
|
|
for barrel; p1 and p2 are the tangential terms. This is OpenCV's convention,
|
|
so the coefficients a camera calibration produces apply directly (OpenCV
|
|
lists them in the order k1, k2, p1, p2, k3). For the example camera, k1 =
|
|
0.1 alone moves the corners 1.6 px outward.
|
|
|
|
That matters for navigation, not just for looks. A limb fit, a star
|
|
identification or a target's centroid all assume where each pixel looks;
|
|
an unmodeled 1.6 px is a 0.2 mrad pointing error at the corner, which at
|
|
36,000 km is 6 km. `Intrinsics::project` and `Intrinsics::ray` include the
|
|
distortion (the inverse by Newton's method, to 1e-10 px), so navigation code
|
|
and the renderer share one model; the projection matches OpenCV's to 1e-9
|
|
px.
|
|
|
|
Rendering it, from camera tier C1 (C0 is always a pinhole): the GPU draws
|
|
the target's triangles with a linear projection, which keeps straight edges
|
|
straight, and moving their corners alone would leave long edges straight
|
|
when they should curve. So the renderer draws the whole frame without
|
|
distortion, the bodies and targets sorted together as usual, on a slightly
|
|
larger and offset area that covers everything the lens bends into view.
|
|
Then it resamples it: for each pixel of the focal plane it finds the
|
|
undistorted point the lens bends onto that pixel and reads the render
|
|
there, interpolating between the four nearest render pixels. The blur's
|
|
first pass (below) does this as it reads its input, so the distorted image
|
|
is never written out. Radiance is
|
|
the same at both ends of a ray, so moving it is all distortion does to an
|
|
extended scene. The resampling works on the finer image C1 renders (below),
|
|
before the blur, which is where the lens puts it in a real camera.
|
|
|
|
Two effects of distortion fall on the detector instead. The cos^4 falloff
|
|
uses each pixel's real line of sight, the undistorted one. And where the
|
|
lens stretches the image, the same light covers more pixels: each pixel's
|
|
signal is divided by how much the distortion stretches area there (the
|
|
determinant of its derivative), 0.6 % at the corners for k1 = 0.1.
|
|
|
|
Measured with coefficients of 0.3, -0.2, 0.1 (radial) and 0.001, -0.0008
|
|
(tangential), which move the corners about 4 px: a small Moon's light lands
|
|
centered where `Intrinsics::project` puts it to within 0.004 px across the
|
|
frame, and a long straight edge follows the predicted curve to within 0.07
|
|
px in every row and 0.02 px averaged over rows. The interpolation adds a
|
|
little blur where it falls between render pixels, at most a variance of
|
|
1/16 px^2 at 2 x 2 rendering (computed), which takes sigma 0.5 px to 0.56 px
|
|
at worst.
|
|
|
|
Distortion costs 0.25 to 0.32 ms per 1920 x 1080 frame on an RTX 3060 at
|
|
2 x 2 rendering, and 1.1 to 1.3 ms at 3840 x 2160, almost all of it the
|
|
blur's first pass reading through it, and under 0.1 ms the larger
|
|
undistorted render. A separate pass writing the distorted image out and the
|
|
blur reading it back cost 0.7 to 0.8 ms and 2.8 to 3.3 ms: at those sizes
|
|
both are limited by how fast the card moves the image through memory. On
|
|
lavapipe distortion adds 10 to 11 ms to 1920 x 1080 frames.
|
|
|
|
### Coordinate conventions
|
|
|
|
The camera frame has +Z along the boresight, +X toward increasing column u,
|
|
and +Y toward increasing row v. This is the computer-vision convention: row
|
|
0 is the top of the image, so +Y points down in the picture. Pixel (i, j)
|
|
covers u in [i, i+1) and v in [j, j+1), so its center is at (i + 0.5,
|
|
j + 0.5).
|
|
|
|
### Why the GPU sees camera-relative coordinates
|
|
|
|
GPUs compute in 32-bit floats, which carry about 7 significant digits. A
|
|
spacecraft 7000 km from Earth's center, stored in Earth-centered f32
|
|
coordinates, can only be placed to the nearest 0.5 m. A target 10 m away
|
|
would jitter by several pixels. So all positions stay in 64-bit floats on
|
|
the CPU. They are made relative to the camera there, and converted to
|
|
32-bit only after that. A target 10 m from the camera is then placed to
|
|
about a micrometer.
|
|
|
|
### Depth
|
|
|
|
When two things overlap in the image, the nearer one must win. The GPU
|
|
keeps a depth value per pixel for this. The renderer stores near / z,
|
|
which is 1 at the near clip distance and falls toward 0 with distance.
|
|
Combined with how floating-point numbers are spaced, this gives roughly the
|
|
same relative precision at every distance. That matters in a scene that
|
|
contains a 10 m target, Earth at thousands of kilometers, and the Sun at
|
|
150 million kilometers. The scheme is called reversed-Z with an infinite
|
|
far plane.
|
|
|
|
## Radiometric quantities
|
|
|
|
Code: `spectrum.rs`, `radiometry.rs`.
|
|
|
|
Light has a few distinct measures, and the camera measures a specific one.
|
|
|
|
- **Irradiance, E, in W/m^2.** Power arriving per unit area of a surface.
|
|
Sunlight at 1 AU delivers about 1361 W/m^2 to a surface facing the Sun.
|
|
It falls off with the square of the distance from the Sun. A surface
|
|
tilted by angle theta from the Sun direction receives E cos(theta).
|
|
- **Solid angle, in steradians (sr).** The 2D version of an angle: the area
|
|
a cone cuts out of a unit sphere. The full sphere is 4 pi sr. One pixel
|
|
of the example camera covers (110 urad)^2, about 1.2e-8 sr.
|
|
- **Radiance, L, in W/m^2/sr.** Power per unit area per unit solid angle,
|
|
traveling in a particular direction. It is what a camera pixel responds
|
|
to: how bright a surface looks from a given direction.
|
|
|
|
Radiance has a property that explains a lot of what a camera sees: it does
|
|
not change along a ray through empty space. Move a camera twice as far from
|
|
a wall, and each pixel sees four times as much wall area. But each square
|
|
meter now contributes a quarter of the power, so the pixel value is
|
|
unchanged. An object that covers many pixels has the same surface
|
|
brightness at any distance; it just covers fewer pixels farther away.
|
|
|
|
That stops being true once the object is smaller than a pixel. The pixel
|
|
then also sees dark space around the object, so the signal falls as one
|
|
over distance squared. That is the regime of a distant satellite seen as a
|
|
point (space situational awareness), as opposed to the resolved imagery
|
|
this renderer is mostly for.
|
|
|
|
### Reflection: albedo and Lambertian surfaces
|
|
|
|
A Lambertian surface reflects light equally in every direction. Matte
|
|
paint, the Moon (roughly) and cloud tops behave close to this. If such a
|
|
surface receives irradiance E and reflects a fraction rho (its albedo), its
|
|
radiance in every direction is:
|
|
|
|
```text
|
|
L = rho * E / pi
|
|
```
|
|
|
|
The pi comes from integrating the reflected radiance over the hemisphere
|
|
above the surface. A white Lambertian sheet facing the Sun at 1 AU, with
|
|
rho = 1, has a radiance of about 1361 / pi = 433 W/m^2/sr, summed over all
|
|
wavelengths. Earth's average albedo is about 0.3.
|
|
|
|
Real spacecraft surfaces are not Lambertian. Multilayer insulation (MLI)
|
|
and solar cells are partly mirror-like and produce bright glints at
|
|
particular angles. These are modeled by a BRDF, a function giving reflected
|
|
radiance for every pair of incoming and outgoing directions. BRDFs are not
|
|
modeled yet.
|
|
|
|
## Wavelength: spectra, channels and photons
|
|
|
|
Code: `spectrum.rs` (`Spectrum`), `camera.rs` (`Channel`).
|
|
|
|
Everything above varies with wavelength. Sunlight peaks around 500 nm and
|
|
extends from the ultraviolet into the infrared. Surfaces reflect different
|
|
wavelengths differently; that is what color is. Optics transmit some
|
|
wavelengths better than others. The sensor's sensitivity also depends on
|
|
wavelength.
|
|
|
|
The renderer describes each of these as a `Spectrum`: a value as a function
|
|
of wavelength, in meters.
|
|
|
|
- **Quantum efficiency (QE).** The fraction of arriving photons that become
|
|
electrons in a pixel. A color camera puts red, green and blue filters
|
|
over different pixels, so each channel has its own QE curve. The example
|
|
camera uses idealized Gaussian curves centered at 620, 540 and 460 nm,
|
|
80 nm wide at half height, peaking at 60 %.
|
|
- **Transmission (tau).** The fraction of light the lenses pass, about 0.9
|
|
for the example camera.
|
|
|
|
Detectors count photons, not energy. A photon of wavelength lambda carries
|
|
energy h c / lambda (h is Planck's constant, c the speed of light). So one
|
|
watt of red light delivers more photons than one watt of blue. The
|
|
renderer converts power to photons per wavelength before applying QE.
|
|
|
|
### The solar spectrum
|
|
|
|
Code: `radiometry.rs` (`solar_spectrum`), generated into `solar_data.rs` by
|
|
`scripts/fetch-solar-spectrum.sh`.
|
|
|
|
The renderer uses a measured solar spectrum, the TSIS-1 Hybrid Solar
|
|
Reference Spectrum (Coddington et al. 2021). It is the current
|
|
international reference, built mainly from the TSIS-1 instruments on the
|
|
International Space Station and filled in with a high-resolution model of
|
|
the solar absorption lines. It gives the irradiance at 1 AU, outside the atmosphere,
|
|
every 1 nm from 202 to 2730 nm. Its uncertainty is 0.3 % from 460 to
|
|
2365 nm. Over that range it sums to 1325.8 W/m^2. The other 35 W/m^2 of the
|
|
1361 W/m^2 total is at wavelengths silicon detectors do not see.
|
|
|
|
The Sun is often approximated as a 5772 K blackbody: a smooth curve with
|
|
the right total power. The real spectrum has thousands of absorption lines
|
|
from the Sun's outer layers, and its shape differs from the smooth curve.
|
|
For the example camera's channels, the blackbody gives 4.6 % (red), 7.5 %
|
|
(green) and 9.7 % (blue) fewer electrons than the measured spectrum. That
|
|
is far outside what absolute radiometry can tolerate.
|
|
|
|
### Folding spectra into per-channel constants
|
|
|
|
Code: `radiometry.rs` (`Radiometry`).
|
|
|
|
Doing all of this wavelength by wavelength for every pixel would be slow.
|
|
Most factors do not depend on the pixel: the solar spectrum, the optics,
|
|
and the QE curves. The renderer multiplies and integrates those over
|
|
wavelength once, when the camera is set up. That yields a few numbers per
|
|
channel, and the GPU only multiplies by them.
|
|
|
|
The central number is the solar flux of each channel k:
|
|
|
|
```text
|
|
S_k = integral of E_sun(lambda) * tau(lambda) * QE_k(lambda) * lambda / (h c) d lambda
|
|
```
|
|
|
|
It is the rate of electrons per square meter that sunlight at 1 AU would
|
|
produce if it fell straight onto the detector through the optics. For the
|
|
example camera, S_k is about 2.4e20 electrons per second per m^2 in red and
|
|
green, and 2.0e20 in blue.
|
|
|
|
The renderer then works in channel radiance: scene radiance weighted the
|
|
same way (transmission, QE, photons per joule), in electrons per second
|
|
per m^2 per steradian. A Lambertian surface with albedo rho, lit at angle i
|
|
from its normal, at distance d from the Sun in AU, has channel radiance:
|
|
|
|
```text
|
|
L_k = rho * cos(i) * S_k / (pi * d^2)
|
|
```
|
|
|
|
The solar disk has channel radiance S_k / Omega_sun, where Omega_sun is the
|
|
Sun's solid angle at 1 AU, 6.79e-5 sr (it is 0.53 degrees across). Seen
|
|
from anywhere, the Sun's surface has that same radiance, about 2.0e7
|
|
W/m^2/sr in total. Looked at directly, it puts about 5e8 electrons into a
|
|
pixel of the example camera's green channel in 0.1 ms, 26,000 times the
|
|
full well.
|
|
|
|
A surface whose reflectance varies with wavelength is reduced to an
|
|
effective albedo per channel. That is the albedo of a flat-spectrum surface
|
|
that would give the same signal under sunlight
|
|
(`Radiometry::effective_albedo`). This step is where the error of the
|
|
shortcut comes in. Two spectra that both vary inside one channel's band
|
|
interact in ways a single number per channel cannot capture. One example is
|
|
a surface whose reflectance changes steeply across green, lit partly by
|
|
sunlight and partly by bluish light scattered from Earth's atmosphere.
|
|
Rendering more, narrower channels reduces that error, and the code keeps
|
|
the channel count generic for that reason.
|
|
|
|
## From radiance to electrons: the camera equation
|
|
|
|
Code: `radiometry.rs` (`Radiometry::electrons`); `gpu/sensor.rs`, `shaders/sensor.wgsl`.
|
|
|
|
A lens with focal length f and aperture diameter D has f-number
|
|
N = f / D. The example camera is f/4: a 50 mm lens with a 12.5 mm aperture.
|
|
For a scene radiance L, the irradiance on the focal plane at the image
|
|
center is:
|
|
|
|
```text
|
|
E_image = pi * L * tau / (4 N^2)
|
|
```
|
|
|
|
The aperture collects light over a solid angle proportional to (D / f)^2 =
|
|
1 / N^2. So opening the aperture one "stop", which divides N by the square
|
|
root of 2, doubles the light. Image irradiance is independent of distance
|
|
for the same reason radiance is.
|
|
|
|
Away from the center, image irradiance falls off as cos^4(theta), where
|
|
theta is the angle from the boresight. Two factors come from the longer
|
|
path to the off-axis pixel. One comes from the aperture appearing
|
|
foreshortened. One comes from light reaching the pixel at a slant. At the
|
|
corner of the example camera, theta is 6.9 degrees, and the falloff is
|
|
3 %. Real lenses add more falloff of their own, called vignetting.
|
|
|
|
The electrons a pixel collects in one channel are then:
|
|
|
|
```text
|
|
electrons = E_image * (pixel area) * (exposure time) * (photons per joule) * QE
|
|
```
|
|
|
|
The last two factors are wavelength-dependent. In channel radiance they are
|
|
already folded in, so the camera equation reduces to
|
|
|
|
```text
|
|
electrons = L_k * pi * (pixel area) * (exposure time) / (4 N^2) * cos^4(theta)
|
|
```
|
|
|
|
For the example camera, pi * A * t / (4 N^2) is 1.48e-16 m^2 sr s.
|
|
|
|
## From electrons to DN: the detector
|
|
|
|
Code: `camera.rs` (`Detector::dn`, `Detector::variance`); `gpu/sensor.rs`,
|
|
`shaders/sensor.wgsl`.
|
|
|
|
- **Full well.** The maximum number of electrons a pixel holds. Beyond it
|
|
the pixel saturates and reads its maximum value whatever the light.
|
|
Brighter scenes do not look brighter. In real sensors, excess charge can
|
|
also spill into neighboring pixels (blooming, below).
|
|
- **Gain.** Electrons per DN. With a gain of 5, every 5 electrons add 1 to
|
|
the output.
|
|
- **Bias.** A fixed offset added to every pixel, so that noise around zero
|
|
signal does not get clipped at 0. The example uses 64 DN.
|
|
- **ADC bits.** The output resolution. 12 bits gives values 0 to 4095.
|
|
|
|
So DN = bias + min(electrons, full well) / gain, rounded and clipped to the
|
|
ADC range. The renderer does this on the GPU for every pixel and channel,
|
|
as the last step of a frame, and reads back only the resulting integers.
|
|
From camera tier C1, noise enters between the electrons and the gain (see
|
|
Noise below).
|
|
|
|
Two consequences of these numbers show up in every image:
|
|
|
|
- **Saturation is not always the ADC's maximum.** In the example camera the
|
|
full well fills first: 20,000 / 5 + 64 = 4064 DN, below the 12-bit maximum
|
|
of 4095. A saturated pixel there reads 4064. Real cameras are designed
|
|
either way, and an algorithm that detects saturation by looking for 4095
|
|
would miss it in this one.
|
|
- **Black is not zero.** A pixel that receives no light reads the bias,
|
|
64 DN, so empty space in a raw frame displays as dark gray.
|
|
|
|
Raw frames are also not white-balanced. The camera reports what each
|
|
channel collected, and the channels collect different amounts from the same
|
|
gray surface. In the example camera, sunlight on a gray surface gives about
|
|
15 % fewer electrons in blue than in red, so gray things look slightly
|
|
yellow-green in a raw image. A white balance step, applied after capture,
|
|
would scale the channels to make them equal. The renderer leaves it out,
|
|
because it simulates what the sensor records.
|
|
|
|
### Worked example
|
|
|
|
Take a white Lambertian surface facing the Sun at 1 AU, seen by the example
|
|
camera's green channel. Integrating the solar spectrum through the green QE
|
|
curve, f/4 optics, 5.5 um pixels and a 0.1 ms exposure gives 11,139
|
|
electrons. Red gets 11,408 and blue 9,626. (A test in `radiometry.rs`
|
|
checks the green figure.)
|
|
|
|
- DN = 64 + 11,139 / 5, about 2292, a little over half the 12-bit range.
|
|
- The full well is 20,000 electrons, so this surface is at 56 % of
|
|
saturation.
|
|
- Earth, with albedo near 0.3, reads about 64 + 3,342 / 5, about 732 DN.
|
|
- The same white surface at the image corner, 6.9 degrees off axis, gets
|
|
cos^4 of that, 97 %: 10,819 electrons, 2228 DN.
|
|
|
|
The sensor tests check the 2292, 2228, 732, 4064 and 64 DN figures
|
|
exactly, on the GPU and on lavapipe.
|
|
|
|
Picking the exposure time is the same trade a photographer makes. A longer
|
|
exposure saturates the sunlit target. A shorter one leaves the side of the
|
|
target lit only by Earthshine lost in noise.
|
|
|
|
### Noise
|
|
|
|
Code: `Detector::variance`; `sensor.wgsl` (`NOISE`).
|
|
|
|
A real image is noisy, and the noise decides what can be detected at all.
|
|
From camera tier C1 the renderer adds three kinds, following the EMVA 1288
|
|
standard's model of a camera:
|
|
|
|
- **Shot noise.** Light arrives as photons at random times, so a pixel
|
|
expecting N electrons collects a Poisson-distributed count with mean N and
|
|
standard deviation sqrt(N). It is a property of light, not of the camera:
|
|
a perfect detector still has it. A pixel expecting 10,000 electrons varies
|
|
by 100 (1 %); one expecting 100 varies by 10 (10 %).
|
|
- **Dark current.** The detector makes electrons of its own from heat,
|
|
`Detector::dark_current` per second, which add to the light's and carry
|
|
shot noise of their own. It grows with temperature, roughly doubling every
|
|
6 to 8 degrees C; the value given is for the detector's operating
|
|
temperature.
|
|
- **Read noise.** Reading a pixel out adds a few electrons of random error,
|
|
`Detector::read_noise` RMS, to every pixel in every frame, light or not.
|
|
|
|
In order: the light's electrons and the dark electrons together are drawn
|
|
as one Poisson count (a sum of Poisson counts is itself one); the count is
|
|
clipped to the full well; read noise is added; then the gain, bias and
|
|
rounding to DN as before. Below a mean of 50 electrons the count is drawn
|
|
exactly. Above it a Gaussian of the same mean and variance takes its place,
|
|
which a Poisson count's shape is already close to and which costs the same
|
|
at any level. With read noise the DN can fall below the bias, which is what
|
|
the bias is for.
|
|
|
|
Each pixel's variance in electrons^2 is then
|
|
|
|
variance = (electrons + dark current x exposure) + read noise^2 + gain^2 / 12
|
|
|
|
where the last term is the rounding to whole DN. Divided by gain^2 it is in
|
|
DN^2. Plotted against the mean signal in DN, the variance is a straight line
|
|
with slope 1 / gain and intercept (read noise^2 + dark shot noise) / gain^2:
|
|
the photon transfer curve, the standard way to measure a camera's gain and
|
|
read noise from flat fields alone.
|
|
|
|
For the example camera (5 electrons read noise, 20 electrons per second dark
|
|
current, gain 5) at 0.1 ms, the signal-to-noise ratio, signal over the
|
|
square root of the variance, is:
|
|
|
|
| What | Electrons | SNR |
|
|
|---|---|---|
|
|
| A white surface facing the Sun, green channel | 11,139 | 105 |
|
|
| Earth, albedo 0.3 | 3,342 | 58 |
|
|
| A white surface lit by 1 % of sunlight, as Earthshine can | 111 | 9.5 |
|
|
| A V = 0 star, over the 4 pixels it covers | 56 | 4.4 |
|
|
| A V = 4 star, over 4 pixels | 1.4 | 0.13 |
|
|
|
|
So at the exposures a sunlit target needs, the brightest stars are just
|
|
detectable and the rest are lost in read noise. At 50 ms, a star tracker's
|
|
exposure, a V = 6 star reaches an SNR of 7.5.
|
|
|
|
Dark current hardly matters at these exposures: 20 electrons per second is
|
|
0.002 electrons in 0.1 ms. It matters for long ones: 1 electron in 50 ms,
|
|
100 in 5 s, with 10 electrons of shot noise.
|
|
|
|
**Reproducible randomness.** The noise comes from a counter-based generator:
|
|
a hash (Jarzynski and Olano's pcg4d) of the pixel and channel, the frame
|
|
number and a 64-bit seed (`RendererOptions::noise_seed`), turned into
|
|
uniform numbers, then into Gaussians (Box-Muller) or a Poisson count (by
|
|
inverting its distribution). Nothing is carried from one pixel or frame to
|
|
the next, so every value is computed independently on the GPU. The renderer
|
|
numbers its frames from 0 (`Frame::number`), and
|
|
`Renderer::set_frame_number` renders a frame again with exactly the same
|
|
noise. Different frame numbers or seeds give independent noise. The hash is
|
|
identical on every device, but the logarithms and cosines that shape it may
|
|
differ in the last bit, so the GPU and lavapipe agree statistically rather
|
|
than pixel for pixel.
|
|
|
|
**Noise-free images.** A detector with no read noise, no dark current, shot
|
|
noise off and no fixed pattern (`Detector::noiseless`) renders C1 without
|
|
noise, exactly as before noise existed, for ground truth and ML labels. The
|
|
renderer then compiles the noise out entirely. At C0 there is never noise.
|
|
|
|
Measured, identically on the GPU and lavapipe: flat fields at six levels
|
|
from 100 to 16,000 electrons, two frames each, give half the variance of
|
|
their difference within 2 % of the formula (0.982 to 1.008 of it, against a
|
|
sampling error of 0.6 %), and the photon transfer curve's slope is 1 / gain
|
|
to 0.1 %. A dark frame of 100 electrons reads 20.005 DN above the bias, with
|
|
its variance within 0.6 %. Counts below 50 match the Poisson probabilities
|
|
count by count. Consecutive frames, different seeds and neighboring pixels
|
|
are uncorrelated (|r| at most 0.005 over 196,608 values). 400 faint stars of
|
|
300 electrons each vary as predicted to 0.4 %, an SNR of 7.35. Noise costs nothing measurable on an
|
|
RTX 3060 (0.476 ms for the sensor pass at 1920 x 1080 with or without it,
|
|
1.89 ms at 3840 x 2160); on lavapipe it adds 1.5 ms at 1920 x 1080 and
|
|
9.6 ms at 3840 x 2160.
|
|
|
|
### Fixed pattern: PRNU and DSNU
|
|
|
|
Code: `fixed_pattern.rs` (`FixedPattern`); `sensor.wgsl` (`PATTERN`).
|
|
|
|
No two pixels of a real detector are quite alike, and the differences are
|
|
the same in every frame. That sets them apart from the noise above, which is
|
|
new each frame:
|
|
|
|
- **PRNU (photo-response nonuniformity).** Each pixel converts light to
|
|
electrons a little more or less efficiently than average, typically by
|
|
0.5 to 1.5 % RMS. Under even light the image is mottled by that much.
|
|
- **DSNU (dark signal nonuniformity).** Each pixel's dark current differs
|
|
from the average, by tens of percent. A few pixels, from defects in the
|
|
silicon, make tens to hundreds of times the average: hot pixels.
|
|
|
|
From camera tier C1 each value (pixel and channel) has a response factor,
|
|
which multiplies its light's electrons (stars' included), and a dark factor,
|
|
which multiplies the dark current, before the shot noise and the full well.
|
|
The example camera has 1 % PRNU, 10 % DSNU, and 1 value in 10,000 hot at 50
|
|
times the dark current (`Detector::prnu`, `dsnu`, `hot_pixel_fraction`,
|
|
`hot_pixel_multiple`). Because the dark factor scales the dark current, the
|
|
dark pattern grows with the exposure, as a real one does: at 0.1 ms the
|
|
example camera's hot pixels make 0.1 electrons, invisible; over 5 s, 5,000.
|
|
A camera made for long exposures is calibrated: its hot pixels are found in
|
|
dark frames and mapped out. The long-exposure example scenes (the Pleiades
|
|
and the GEO telescopes) leave 1 value in a million hot instead
|
|
(`hot_pixels = [1e-6, 50]` in a scene file's `[camera]`).
|
|
|
|
The pattern comes from `Detector::pattern_seed`, apart from the noise's
|
|
seed, so one camera keeps its pattern whatever the frame or noise: the same
|
|
hash as the noise, of the seed and the value's index, gives a Gaussian
|
|
response and dark factor and decides whether the value is hot. The shader
|
|
computes it for each value as it goes, so it takes no memory and no setup.
|
|
`FixedPattern::generate` computes the same pattern on the CPU, for tests or
|
|
to correct images with. A pattern measured from a real camera can be used
|
|
instead (`FixedPattern::from_calibration`, `RendererOptions::fixed_pattern`):
|
|
a flat, a frame of even light, and a dark, a frame with none, each the
|
|
average of many so their own noise is small, bias removed; each is divided
|
|
by its mean.
|
|
|
|
This is why real images are flat-fielded and dark-subtracted before use:
|
|
divided by a flat and with a dark frame taken off, which removes the
|
|
pattern and leaves the noise. It matters for detection in particular. A hot
|
|
pixel in a long exposure is a bright, sharp point that stays put from frame
|
|
to frame, just like a faint star or a distant target; only its
|
|
single-pixel sharpness (a real point source is spread by the PSF) or a dark
|
|
frame gives it away.
|
|
|
|
Measured, on the GPU and lavapipe: with noise off, a flat field and dark
|
|
frames at two exposures read the CPU's pattern to the rounding (within
|
|
0.505 DN of the unrounded value), the hot pixels are the same ones, and the
|
|
spreads are the requested 1 % and 10 % to 2 %. With noise on, at 10,000
|
|
electrons the pattern and the noise are equal parts of the spread, and two
|
|
frames or two noise seeds correlate by the predicted 0.50, while another
|
|
pattern seed shares nothing. Calibration maps are applied to the rounding.
|
|
The pattern costs nothing measurable on an RTX 3060; on lavapipe it adds
|
|
about 2.3 ms at 1920 x 1080 and 6.3 ms at 3840 x 2160. A loaded pattern
|
|
takes 8 bytes per value, 199 MB at 3840 x 2160 in RGB.
|
|
|
|
### Blooming
|
|
|
|
Code: `Blooming`; `sensor.wgsl` (`BLOOMING`: `collect_charge`,
|
|
`bloom_columns`, `read_out`).
|
|
|
|
A saturated pixel's extra charge does not vanish. Where it goes depends on
|
|
the sensor, and matters whenever something in or near the frame is far past
|
|
the full well: the Sun at the example camera's 0.1 ms is about 26,000 full
|
|
wells at the center of its disk, and a sunlit specular part can be hundreds.
|
|
|
|
- **CCDs** move charge down columns to read it out, and the channels that
|
|
hold it are only walled off from each other across the columns. Excess
|
|
charge spills along the column, up and down, filling each pixel it
|
|
reaches before passing on: the familiar vertical streak through the Sun
|
|
or a glint, which can cross a whole frame and hide a target in it.
|
|
- **CMOS sensors** read each pixel on its own, and most have an
|
|
anti-blooming drain that carries excess charge away. What escapes it
|
|
spills into the pixels around.
|
|
|
|
From camera tier C1, `Detector::blooming` chooses: `None` (the default: the
|
|
pixel clips and the excess is lost), `Column { efficiency }` or
|
|
`Neighbors { efficiency }`, where the efficiency is the fraction of the
|
|
excess that spreads and the rest drains. Along a column, half goes up and
|
|
half down; each pixel it reaches takes what it has room for and passes the
|
|
rest on, until it runs out or leaves the frame. Into the neighbors, each of
|
|
the 8 takes an eighth, and what overfills them drains.
|
|
|
|
How far a streak runs is the excess over the full well, times the
|
|
efficiency, divided among pixels that each hold one full well. For a bright
|
|
star only a pixel or two saturate, by a little, and the streak is short and
|
|
a pixel or two wide. The Sun is different. Its disk is about 85 pixels
|
|
across in the example camera, so the streak is 85 pixels wide, and at 0.4 ms
|
|
each pixel of it holds about 100,000 full wells: a column through the middle
|
|
of the disk has some 8 million full wells of excess. With no drain
|
|
(efficiency 1), half of that runs each way, enough to fill thousands of
|
|
frames' worth of column, so every pixel in those columns saturates and the
|
|
rest runs off the edges: a white band the width of the Sun, top to bottom
|
|
(`examples/scenes/sun-glare.toml`). That is what a CCD without an
|
|
anti-blooming drain records. A drain rated for 100 to 1,000 times
|
|
overexposure, as is typical, still leaves the Sun at about 100,000 times;
|
|
only an efficiency of about 5e-5 would shorten this streak to about 200
|
|
pixels each way.
|
|
|
|
Blooming happens to the charge, so it comes after the shot noise (and the
|
|
fixed pattern) and before the read noise and the ADC. With blooming on, the
|
|
sensor pass works in three steps rather than one: it writes each value's
|
|
charge to a buffer, spreads it, and reads it out. Along columns the spread
|
|
is a walk up and down each column, one at a time; the first step marks the
|
|
columns that hold any excess, and the walk skips the rest, which in most
|
|
frames is all of them. The CPU reference in the tests does the same walk in
|
|
f64.
|
|
|
|
Measured, on the GPU and lavapipe: a single overfull pixel on an even
|
|
background fills the pixels it should along its column to the electron,
|
|
charge that leaves the frame is lost, and nothing else changes; the
|
|
neighbors share equally; below the full well, or at efficiency 0, blooming
|
|
changes no value. The Sun at 2e-5 efficiency, in a 256 x 256 frame of the
|
|
example optics, streaks 22 pixels each way from the middle of its disk,
|
|
matching the CPU column by column to 0 DN. Column blooming costs 0.23 ms per
|
|
1920 x 1080 frame on an RTX 3060 with nothing past the full well and 0.75 ms
|
|
with the Sun in the frame (1.0 and 2.2 ms at 3840 x 2160); into the
|
|
neighbors, 0.24 ms (1.0 ms). On lavapipe, about 3 ms and 10 to 12 ms at
|
|
1920 x 1080.
|
|
|
|
## How shapes become pixels
|
|
|
|
Two techniques turn 3D shapes into an image. The renderer uses one for the
|
|
bodies and the other for the target.
|
|
|
|
- **Mesh.** A surface approximated by many small flat triangles that share
|
|
corners (vertices). A cube needs 12 triangles. A curved surface needs
|
|
many more, and it is still faceted up close. Each vertex stores a position
|
|
and usually a normal, the direction the surface faces there. STL and glTF
|
|
files are meshes, and so is the target spacecraft.
|
|
- **Rasterization.** The GPU's built-in way of drawing a mesh. Each
|
|
triangle's corners are projected into the image with the pinhole
|
|
equations above. The GPU then finds every pixel whose center falls inside
|
|
the projected triangle, and runs a small program (a fragment shader) for
|
|
each of those pixels to compute its value. It is fast because it only
|
|
touches the pixels each triangle covers, and GPUs have dedicated hardware
|
|
for it. It can only draw triangles.
|
|
- **Ray casting.** The reverse direction: for each pixel, follow the line
|
|
of sight from the camera through the pixel center and solve for where it
|
|
first meets a surface. For a sphere or an ellipsoid that is a quadratic
|
|
equation with an exact answer, so there are no triangles and no facets.
|
|
Every pixel does the work, whether or not it ends up seeing the body.
|
|
|
|
The bodies use ray casting, because their shapes have exact equations and
|
|
their edges are what limb-based navigation measures. The target uses
|
|
rasterization, because a spacecraft model has no equation, only its mesh.
|
|
The meshes are drawn first, with their depth. The bodies pass then casts its
|
|
rays and compares depths at every pixel, so the two sort against each other
|
|
correctly: a target in front of Earth covers it, and a target behind the limb
|
|
is hidden. Ray casting a mesh is also possible, by testing rays against
|
|
its triangles. A ray tracer, planned, does that, because following rays
|
|
gives exact shadows and light bouncing between surfaces. Rasterization can
|
|
only approximate those, for example with the shadow maps lighting L1 uses
|
|
(below).
|
|
|
|
### Pixel area
|
|
|
|
At camera tier C0, both techniques decide each pixel's value from one
|
|
point, the pixel's center. A real pixel collects light over its whole area,
|
|
about 5.5 um square on the detector. Where a pixel straddles an edge, say
|
|
half on a sunlit panel and half on dark space, the real pixel reads about
|
|
half the panel's value. At C0 the renderer reads either the full value or
|
|
none, depending on which side the center falls.
|
|
|
|
This shows in two ways. Edges are stair-stepped instead of graded. And
|
|
anything thinner than a pixel, like the booms of the example satellite seen
|
|
from 40 m, breaks into dashes, because only some pixel centers land on it.
|
|
A regular pattern finer than the pixels, like the cells of a solar array,
|
|
beats against the pixel grid and shows as curved bands (moiré). For an
|
|
algorithm that finds edges or measures brightness at sub-pixel precision,
|
|
the graded edge is the signal, so this matters for limb navigation and
|
|
pose estimation.
|
|
|
|
Camera tier C1 samples each pixel at several points and averages them
|
|
(supersampling), in a way that costs little where nothing is crossing the
|
|
pixel:
|
|
|
|
- **Meshes** use the GPU's multisampling. The rasterizer tests each
|
|
triangle against 4 points in every pixel, and keeps a depth for each, but
|
|
runs the fragment shader once per pixel. A pixel half covered by a panel
|
|
then gets the panel's value in 2 of its 4 samples, and reads half. The
|
|
covered fraction comes in steps of a quarter, and the light is computed
|
|
at one point inside the covered part (the centroid of the covered
|
|
samples), so a pixel on an edge is still shaded as one surface.
|
|
- **Bodies** are ray cast, so the bodies pass first decides whether a body's
|
|
edge could cross the pixel: whether the line of sight through its center
|
|
passes within one pixel's angle of the edge of Earth, the Moon or the Sun.
|
|
Most pixels do not, and cast one ray as at C0. A pixel on an edge casts a
|
|
grid of 4 x 4 rays spread over its area and averages them. A body only a
|
|
few pixels across needs a finer grid, because its brightness falls off
|
|
toward its edge and a coarse grid misjudges that: a 4 x 4 grid gets a
|
|
full Moon 0.45 pixels wide 45 % wrong in total light. So the grid has at
|
|
least 8 rays across the body's radius, up to 64 x 64 for one smaller than
|
|
a pixel, which keeps the total within 1 %. Only a handful of pixels pay
|
|
for that.
|
|
- **Both together.** Where a target and a body share a pixel, each of the
|
|
grid's rays takes the mesh sample nearest to it, and keeps the target if
|
|
that sample is nearer than the body along the ray. So a target seen just
|
|
past Earth's limb takes its share of the pixel, and Earth keeps the rest.
|
|
|
|
A grid of 4 x 4 is still a coarse estimate of an area. For an edge nearly
|
|
aligned with the pixel grid, its 4 columns of rays resolve the covered
|
|
fraction in quarters, so a pixel on the limb can be off by up to an eighth
|
|
of the difference across the edge (measured 0.10 on Earth's limb from low
|
|
orbit, against up to a half at C0). Patterns that stagger the rays fix
|
|
aligned edges but do worse at other angles; the regular grid had the
|
|
smallest error on average over all angles. At C1 the point spread function
|
|
(below) then spreads every edge over a few pixels, and C1 also renders each
|
|
pixel as 2 x 2 finer ones by default, each sampled this way, which divides
|
|
these errors further.
|
|
|
|
At one rendered pixel per image pixel, C1 costs 0.4 to 0.5 ms more per
|
|
1920 x 1080 frame on an RTX 3060, and 1.2 to 1.4 ms at 3840 x 2160, measured on Landsat 8, ICESat-2 and the moonrise
|
|
scene. On the CPU through lavapipe it costs much more, 2.6 to 8 times the
|
|
mesh pass at C0 (ICESat-2's 752,000 triangles go from 74 to 580 ms per
|
|
frame), because lavapipe appears to run the fragment shader for every
|
|
sample rather than every pixel. The 4 samples of radiance and depth also
|
|
take 80 bytes per pixel of GPU memory: 166 MB at 1920 x 1080 and 660 MB at
|
|
3840 x 2160.
|
|
|
|
## Blur: the point spread function
|
|
|
|
Code: `camera.rs` (`Psf`); `gpu/blur.rs`, `shaders/blur.wgsl`; from camera
|
|
C2, `psf.rs`, `gpu/convolve.rs`, `gpu/fft.rs`, `shaders/convolve.wgsl`,
|
|
`shaders/fft.wgsl`.
|
|
|
|
No camera focuses a point to a point. Light from one point of the scene
|
|
lands on the detector as a small spot, and the shape of that spot is the
|
|
point spread function (PSF). Several things widen it:
|
|
|
|
- **Diffraction.** Light passing through a round aperture spreads into a
|
|
pattern of rings (the Airy pattern) whose size depends only on the
|
|
wavelength and the f-number. At f/4 and 550 nm the first dark ring is
|
|
2.7 um from the center, half a pixel of the example camera.
|
|
- **Aberrations and defocus.** Real lenses bring different parts of the
|
|
aperture, different wavelengths and different parts of the field to
|
|
slightly different points.
|
|
- **The detector.** Electrons freed deep in a pixel can drift into its
|
|
neighbors before they are collected (charge diffusion).
|
|
- **Jitter.** The camera shakes a little during the exposure.
|
|
|
|
Each of these blurs the image of the one before it, and many such blurs
|
|
together come out close to a Gaussian, a bell curve in 2D. So a common model
|
|
of a camera's PSF is a Gaussian with one width, its standard deviation
|
|
sigma, measured in pixels. A spot with sigma 0.5 px is 1.2 px wide at half
|
|
its peak, and a point centered on a pixel puts about 47 % of its light in
|
|
that pixel and the rest in its neighbors; at 0.7 px, 28 %. Longer
|
|
wavelengths diffract more, so sigma is set per channel and usually grows
|
|
toward the red.
|
|
|
|
How wide the spot should be depends on the camera. Published measurements,
|
|
converted to a Gaussian's sigma with the pixel's own area taken out:
|
|
|
|
- Earth-observation imagers are built sharp: about 0.2 px for Landsat 8's
|
|
OLI, the sharpest, and 0.35 to 0.55 px for WorldView-3, Pléiades Neo and
|
|
Sentinel-2's specification. The Lunar Reconnaissance Orbiter's cameras are
|
|
specified in the same range.
|
|
- Navigation and small-satellite cameras are softer: about 0.6 to 0.9 px
|
|
for DART's DRACO camera in flight, and 0.55 to 0.7 px for Satellogic's
|
|
imagers.
|
|
- Star trackers are defocused so each star covers several pixels. The one
|
|
measured in flight here (on Lunar Flashlight) has a sharp core and wide
|
|
wings that no single Gaussian matches: its spot is as wide at half its
|
|
peak as a 0.4 px Gaussian, and spreads its light as widely as a 1.25 px
|
|
one.
|
|
|
|
The imagers' numbers are for the whole system, including the atmosphere
|
|
and image processing, so the optics alone are somewhat sharper. The example
|
|
camera uses 0.5 px in every channel, a sharp camera, not derived from its
|
|
optics; diffraction alone at f/4 would give about 0.17 px.
|
|
|
|
Without blur, a star or a distant spacecraft lands in one pixel or in none,
|
|
and every edge is as sharp as the pixel grid allows. Algorithms that find
|
|
the center of a star or a target to a fraction of a pixel (centroiding)
|
|
need the spot spread over more than one pixel, so they cannot be tested on
|
|
an unblurred image.
|
|
|
|
The blur is applied to radiance on the focal plane, before light becomes
|
|
electrons. That order matters for anything bright enough to saturate. The
|
|
Sun's image is tens of thousands of times brighter than a full well.
|
|
Blurred first, its spot spreads that light into the neighboring pixels,
|
|
which also fill, so the saturated region is wider than the Sun's disk, as
|
|
in a real camera. Clipped first and then blurred, the saturated region
|
|
would shrink back inside the disk and its edge would fade over a pixel or
|
|
two, as if the Sun were barely brighter than the full well. The blur's
|
|
weights sum to 1, so it moves light without adding or removing any.
|
|
|
|
### Blurring before the pixels average
|
|
|
|
The blur has to act on the light where it lands, before each pixel averages
|
|
it. Blurring an image of pixel averages afterward is not the same once the
|
|
spot is about as small as a pixel: a point of light that fell anywhere in a
|
|
pixel has become that pixel's value, and where in the pixel it fell is
|
|
gone. Its blurred spot should lean toward wherever it was, and no blur of
|
|
the averages can tell which way. Measured with sigma 0.4 px, points of
|
|
light blurred after averaging put up to 51 % of their light in the wrong
|
|
pixel, and edges were off by up to 12 % of their step.
|
|
|
|
So from camera tier C1 the renderer draws the scene finer than the pixels,
|
|
n x n rendered pixels to each image pixel, blurs that by a Gaussian PSF set
|
|
in `CameraConfig::psf`, and only then averages each n x n block into one
|
|
pixel. Each rendered pixel still collects light over its own area the way
|
|
the previous section describes, so the finer image is itself made of
|
|
small-pixel averages, and the same error remains at a smaller scale:
|
|
|
|
| Rendered pixels per pixel | Points, worst pixel | Edges, worst pixel |
|
|
|---|---|---|
|
|
| 1 x 1 | 51 % of the light | 12 % of the step |
|
|
| 2 x 2 | 19 % | 2.2 % |
|
|
| 4 x 4 | 4.5 % | 0.5 % |
|
|
|
|
(sigma 0.4 px, points at 16 and edges at 129 positions within a pixel; the
|
|
worst edges lie on a pixel boundary.) C1 renders 2 x 2 by default;
|
|
`RendererOptions::supersampling` sets 1 to 4. Point sources need better
|
|
than that, so stars are drawn with the blurred spot computed exactly over
|
|
each pixel instead (see Stars).
|
|
|
|
A 2D Gaussian is the product of a 1D Gaussian across and one down, and
|
|
averaging an n x n block is likewise an average across and one down. So the
|
|
blur runs as two passes, one along rows and one along columns, each taking
|
|
a weighted sum of the rendered pixels out to 4 sigma on either side
|
|
(beyond which a Gaussian holds less than 1e-4 of its light) and averaging
|
|
n of them into one pixel at once. Light from just outside the frame also
|
|
blurs into it, so the renderer draws a margin of that width around the
|
|
image, and the blur reads it and drops it. The weights are the Gaussian at
|
|
each rendered pixel's offset, which keeps a spot's width (its variance)
|
|
within 1 % of sigma squared once sigma is 0.65 rendered pixels or more:
|
|
0.33 px in image pixels at 2 x 2.
|
|
|
|
Rendering finer costs about n^2 times the drawing, and the blur grows with
|
|
it. To keep that down, the meshes are covered at 2 points per rendered pixel
|
|
rather than 4 when rendering finer: at 2 x 2 that still puts samples at 4
|
|
distinct offsets across and down each image pixel, as 4 samples did at
|
|
1 x 1, so mesh edges still grade in quarters or better. (A device that
|
|
cannot render 2 samples, such as lavapipe, uses 4.) GPU time per frame at
|
|
C1, from the first pass to the last, on an RTX 3060 (Landsat 8, ICESat-2
|
|
and the moonrise scene, at lighting L1, sigma 0.5 px):
|
|
|
|
| Rendered pixels per pixel | 1920 x 1080 | 3840 x 2160 |
|
|
|---|---|---|
|
|
| 1 x 1 | 1.9 to 2.5 ms | 6.1 to 6.9 ms |
|
|
| 2 x 2 (default) | 3.7 to 4.4 ms | 12.6 to 14.6 ms |
|
|
| 3 x 3 | 6.7 to 7.9 ms | 34 to 38 ms |
|
|
| 4 x 4 | 10.9 to 13.0 ms | 59 to 66 ms |
|
|
|
|
Of the 3.7 to 4.4 ms at 2 x 2, the bodies take 1.3 to 1.4 ms, the meshes
|
|
0.7 to 1.1 ms and the blur 1.0 ms. With 2 samples instead of 4 the meshes
|
|
and bodies took 0.3 to 0.6 ms less in all; with 1 sample, 0.7 to 1.0 ms
|
|
less, but a mesh edge then grades only in halves at 2 x 2, coarser than at
|
|
1 x 1. The rendered images take 60 bytes per rendered pixel (100 at 1 x 1,
|
|
with 4 samples), and the blur 16 bytes per image pixel per rendered row:
|
|
about 0.27, 0.60, 1.25 and 2.15 GB at 1920 x 1080 for 1 x 1 to 4 x 4, and
|
|
four times that at 3840 x 2160. The renderer reports running out of memory
|
|
as an error. On the CPU through lavapipe, 2 x 2 adds 9 to 13 ms to
|
|
1920 x 1080 frames of 116 to 680 ms, and 3 x 3 at 3840 x 2160 runs out of
|
|
memory.
|
|
|
|
### Diffraction and the PSF's wings (camera C2)
|
|
|
|
A Gaussian falls to nothing within a few sigma. A real camera's spot does
|
|
not: diffraction's rings fade only as the cube of the distance, and dust and
|
|
rough lens surfaces scatter a little light much farther. Next to most of a
|
|
scene that faint halo, the PSF's wings, is invisible. Next to the Sun it is
|
|
not. In the sun-glare example scene each pixel of the Sun's disk collects
|
|
about 5e8 electrons in its 0.4 ms exposure, 26,000 full wells, over about
|
|
5,700 pixels. Diffraction at f/4 spreads a fraction of about
|
|
λN / (π^3 r^3) of a point's light into each pixel r pixels away (λN, the
|
|
wavelength times the f-number, is 0.39 px for green light in the example
|
|
camera), so 500 px from the Sun each pixel still gets about 300 electrons
|
|
from it, and 1000 px away about 40: a glow over the whole frame that raises
|
|
the background, can hide a faint target, and pulls a limb fit toward the
|
|
bright side.
|
|
|
|
Camera tier C2 models this. `Psf::Airy { obstruction }` is diffraction by
|
|
the camera's aperture: the Airy pattern, set by the f-number, with an
|
|
optional central obstruction such as a reflecting telescope's secondary
|
|
mirror (which moves light from the core into the rings). The pattern scales
|
|
with the wavelength, so a channel's spot is the pattern averaged over the
|
|
channel's band, each wavelength weighted by the photons sunlight gives it
|
|
there; across an 80 nm band the outer rings blur into a smooth halo.
|
|
`Psf::Measured { image, pixel_scale }` takes a spot measured in a lab or in
|
|
flight, the same over the whole field, one image per channel or one for
|
|
all. Measured: a 2 nm wide channel's first dark ring falls within 1 % of
|
|
1.22 λN, where the textbook puts it.
|
|
|
|
A kernel reaching hundreds of pixels is far too slow to apply pixel by
|
|
pixel. C2 instead uses the convolution theorem: transform the image into
|
|
its frequencies with a fast Fourier transform (FFT), multiply by the
|
|
kernel's frequencies, computed once at setup, and transform back. The cost
|
|
no longer depends on the kernel's size, only on the image's. The transform
|
|
runs on the GPU in compute shaders, sized to powers of two up to 4096 on a
|
|
side, with two channels sharing one complex transform, so three channels
|
|
take four transforms per frame. C2 applies every PSF this way, Gaussians
|
|
too, and draws stars into the focal plane before the transform, so they get
|
|
the same wings; each star's light is shared among its four nearest rendered
|
|
pixels, which keeps its center exact but can widen its spot by up to 0.25 px
|
|
RMS at 2 x 2.
|
|
|
|
As at C1, the transform acts on the scene rendered finer than the pixels,
|
|
before they average it. The kernel is the PSF integrated over a rendered
|
|
pixel twice, once for the pixel the light leaves and once for the one it
|
|
lands in, so a Gaussian at C2 differs slightly from C1's, which samples the
|
|
Gaussian at the rendered pixels' centers (by up to 0.34 % of the brightest
|
|
pixel, measured on the Moon). The transform treats the image as repeating,
|
|
so the renderer draws a margin around it as wide as the kernel reaches, and
|
|
the grid holds the image and that margin. The reach is therefore set by the
|
|
grid: at least `RendererOptions::psf_reach` (16 px by default), and as far
|
|
as the smallest grid that holds it allows. Light the kernel would carry
|
|
farther is added back to the spot instead, so none is lost, but the glow
|
|
stops at the reach, and with the Sun in the frame its edge shows in a
|
|
brightened preview.
|
|
|
|
| Image, rendered pixels per pixel | Grid | Reach | RTX 3060 | lavapipe |
|
|
|---|---|---|---|---|
|
|
| 1920 x 1080, 2 x 2 | 4096 x 4096 | 64 px | 30 ms | 560 ms |
|
|
| 1920 x 1080, 1 x 1 | 2048 x 2048 | 64 px | 5.8 ms | 213 ms |
|
|
| 1920 x 1080, 1 x 1, reach 1000 | 4096 x 4096 | 1088 px | 29 ms | 556 ms |
|
|
| 3840 x 2160, 1 x 1 | 4096 x 4096 | 128 px | 30 ms | 623 ms |
|
|
|
|
(The convolution pass alone, in the sun-glare scene at lighting L1. The
|
|
whole frame took 35 ms at 1920 x 1080 and 2 x 2, against 5.9 ms at C1.) A
|
|
grid of 4096 x 4096 takes 640 MB on the device. 3840 x 2160 at 2 x 2 would
|
|
need a grid 8192 across and is refused; 1 x 1 fits.
|
|
|
|
The transforms work in 32-bit floating point, which keeps about seven
|
|
significant digits, and their rounding error spreads over the whole grid in
|
|
proportion to its brightest values. With the Sun in the frame that leaves a
|
|
floor of about 10 electrons RMS, up to 165, across the frame (measured in
|
|
the sun-glare scene with a noiseless detector, where C1 reads exactly 0),
|
|
above the example detector's 5 electrons of read noise. A sunlit target at
|
|
half the full well leaves a floor near 1e-3 electrons, far below any noise.
|
|
Computing the brightest sources' wings separately, outside the transform,
|
|
would remove the Sun's floor; that is not done yet.
|
|
|
|
## Earth, Moon and Sun
|
|
|
|
Code: `bodies.rs`, `gpu/bodies.rs`, `shaders/bodies.wgsl`.
|
|
|
|
The three bodies are not drawn as triangle meshes; they are ray cast. For
|
|
every pixel, the shader sends a ray from the camera through the pixel's
|
|
center and solves for where it meets each body's exact shape. The nearest hit wins. This
|
|
gives an exact edge at any zoom, at the cost of a few arithmetic operations
|
|
per pixel. A mesh would show its facets along the limb, the edge of the
|
|
disk, which is the feature horizon-based navigation measures.
|
|
|
|
- **Earth** is the WGS84 ellipsoid: 6378.137 km equatorial radius,
|
|
6356.752 km polar. The 21 km difference matters at fine pixel scales. A
|
|
camera with 5.5 urad pixels at 200,000 km sees the north limb 19 pixels
|
|
away from where a spherical Earth would put it. A test checks the limb
|
|
against the exact tangent to the ellipsoid to within a pixel.
|
|
- **The Moon** is a sphere of 1737.4 km radius. From Earth's distance it
|
|
is about 0.52 degrees across, 82 pixels in the example camera.
|
|
- **The Sun** is a sphere of 695,700 km radius: 0.53 degrees across, about
|
|
85 pixels in the example camera. Averaged over its disk it has the solar
|
|
radiance from the folding section above; from lighting L1 its center is
|
|
brighter than its edge (see "The Sun's disk" below). It is drawn only
|
|
where no nearer body covers it, so Earth eclipses it naturally.
|
|
|
|
### Shading
|
|
|
|
Earth and the Moon are Lambertian, at lighting L0 each with one reflectance
|
|
spectrum over the whole body, set by `Appearance`. The Moon's is a flat 0.12
|
|
(from L1 it comes from a map; see "The Moon's surface"). Earth's
|
|
default is a stand-in shaped like cloud-free ocean seen from space: about
|
|
0.3 in the blue falling to 0.07 in the red, then back up in the near
|
|
infrared. Most of that blue is sunlight scattered by the atmosphere above
|
|
the ocean, the same scattering that makes the sky blue; the ocean itself is
|
|
dark. In the example camera it gives channel albedos of 0.09, 0.15 and 0.26
|
|
(red, green, blue), so Earth renders blue. Its overall brightness is below
|
|
the real Earth's average of about 0.3, which includes bright clouds.
|
|
|
|
Each point's radiance is rho cos(i) S_k / (pi d^2), with i the angle between
|
|
the surface normal and the Sun. The night side has i past 90 degrees and is
|
|
black. The Sun's distance d is taken from each body's center, since the
|
|
difference across a body is a few parts in 10^5.
|
|
|
|
The total light a sphere sends a far-away camera has a closed form. A
|
|
Lambertian sphere of albedo rho and radius R, at distance D, delivers an
|
|
irradiance of (2/3) rho E (R/D)^2 Phi(alpha), where alpha is the phase angle
|
|
(Sun-sphere-camera) and Phi(alpha) = (sin alpha + (pi - alpha) cos alpha) / pi.
|
|
It falls from 1 at full phase to 0.61 at 60 degrees and 0.11 at 120 degrees.
|
|
Up close it is not exact: the near side of the sphere is nearer the camera
|
|
than its center, so it counts for more. At 40,000 km from the Moon (R/D =
|
|
0.043), the real total is 3 % above the formula. The renderer's image agrees
|
|
with the exact surface integral to 0.03 % or better (`tests/gpu/analytic.rs`).
|
|
|
|
This gets the overall brightness and the phase (how much of the disk is
|
|
lit) right. It misses some things that are visible in real images:
|
|
|
|
- Earth's oceans, land and clouds differ in brightness by a factor of
|
|
several. From lighting L1, Earth's surface comes from an image and its
|
|
clouds from another (next sections).
|
|
- The full Moon looks almost uniformly bright out to its edge. A Lambertian
|
|
sphere instead darkens toward the edge, because the light there arrives
|
|
and leaves at grazing angles. Lunar soil scatters light back toward its
|
|
source much more strongly than Lambert predicts, which is also why it
|
|
brightens so steeply toward full Moon. At lighting L1 the Moon's total
|
|
brightness is scaled to its measured phase curve ("The Moon's surface"),
|
|
but its edge still darkens; a dedicated lunar reflectance model would fix
|
|
that. The Moon's maria (the dark plains of
|
|
old basalt lava, the "seas" seen with the naked eye) and its bright
|
|
highlands also differ in albedo by about a factor of two, which from
|
|
lighting L1 the Moon's map shows.
|
|
- Earth's atmosphere softens the limb and adds a thin glowing band around it.
|
|
At lighting L0 Earth's edge is a sharp line; from L1 the atmosphere is drawn
|
|
("Earth's atmosphere").
|
|
|
|
### Earth's surface
|
|
|
|
Code: `earth.rs`, `shaders/bodies.wgsl`; data from `scripts/fetch-earth.sh`.
|
|
|
|
At lighting tier L1 (`RendererOptions::fidelity`, with the image in
|
|
`RendererOptions::earth_texture`; `--lighting L1` on the command line), Earth's
|
|
reflectance comes from an image of the whole surface instead of one spectrum. The image is NASA's
|
|
Blue Marble Next Generation: a cloud-free mosaic of the 2004 MODIS satellite
|
|
record, one image per month, so snow cover and vegetation follow the season.
|
|
The CLI picks the month from the scene's epoch.
|
|
|
|
The image is equirectangular: each column is a longitude and each row a
|
|
latitude. For every pixel that sees Earth, the shader finds the surface
|
|
point's geodetic latitude and longitude from the ellipsoid's normal there, in
|
|
Earth-fixed axes, and reads the image at that place. Earth's orientation from
|
|
the scene turns the continents with the epoch.
|
|
|
|
The image comes in two sizes:
|
|
|
|
- The global map, always there: NASA's 21600 x 10800 image (1 arcminute,
|
|
1.85 km per pixel at the equator), resized to 16384 x 8192 (2.45 km), the
|
|
largest texture lavapipe allows.
|
|
- A sharp map, optional: NASA's 86400 x 43200 image (15 arcseconds, 460 m),
|
|
made once per month by `scripts/fetch-earth.sh --sharp` (below).
|
|
|
|
Whether a map is fine or coarse depends on the view:
|
|
|
|
- From low orbit looking straight down, a camera pixel of the example camera
|
|
covers 77 m of ground. A texel of the global map spans 32 image pixels and
|
|
looks blurred; one of the sharp map, 6.
|
|
- From geostationary orbit, a camera pixel covers about 4 km, so each image
|
|
pixel averages about 2 x 2 texels of the global map.
|
|
|
|
For the second case the texture is stored with mipmaps: copies at half, a
|
|
quarter, an eighth of the size, and so on. The GPU reads from the copy whose
|
|
pixels best match the image pixel, so a far view shows the true average color
|
|
of a region rather than a shimmering sample of individual texture pixels.
|
|
|
|
The sharp map is too large for one texture on either device (lavapipe's limit
|
|
is 16384 on a side) and for one allocation on lavapipe (2 GB), and 30 GB as
|
|
16-bit floating point. So it is cut into 8 x 4 square tiles of 10800 texels,
|
|
each stored with a border of 4 texels copied from its neighbors, so that
|
|
blending near a tile's edge reads the right texels without a second tile, and
|
|
compressed to BC6H: blocks of 4 x 4 texels in 16 bytes, which the GPU decodes
|
|
as it reads them. BC6H keeps linear values with about the precision of 16-bit
|
|
floating point; on Blue Marble's July map around Sinai, 99 % of pixels are
|
|
within 1 to 2.5 % of the uncompressed value and the mean within 0.1 %. Three
|
|
mipmap levels are stored (460 m, 930 m, 1.85 km), 4.9 GB in all, in four
|
|
texture arrays of 8 tiles. A pixel reads the sharp map where it covers less
|
|
ground than the coarsest level's texels, the global map beyond, and the two
|
|
blended across half a level so the change does not show.
|
|
|
|
Its cost on the RTX 3060: 0.1 to 0.2 ms more in the bodies pass where it is
|
|
read, and 1.6 s more setup to upload it. On lavapipe, which decodes BC6H in
|
|
software, about 100 ms more in the bodies pass where it is read and 5 s more
|
|
setup, with 8 GB of memory at its peak. From geostationary orbit it is not
|
|
read, and costs nothing per frame.
|
|
|
|
The GPU and lavapipe choose mipmap levels and blend texels slightly
|
|
differently, so a textured Earth differs between them by about 1 DN on
|
|
average at lighting L1 and camera C1 (0.8 DN with the global map alone,
|
|
1.3 DN with the sharp map; 2.7 DN before the atmosphere, which lowers the
|
|
contrast). The tests of the maps compare each device with itself.
|
|
|
|
Two details matter for brightness:
|
|
|
|
- Averaging has to happen on reflectance, not on the encoded values image files
|
|
store (sRGB, which compresses the darks). Averaging encoded black and white
|
|
gives a reflectance of 21 %, not 50 %. So the texture is decoded to linear
|
|
reflectance once at load and kept in 16-bit floating point, and both the
|
|
mipmaps and the GPU's blending between neighboring texture pixels work on
|
|
those values. A coastline seen up close blends correctly from sea to land.
|
|
- Blue Marble is surface reflectance with no atmosphere, and open ocean is
|
|
dark, a few percent. From orbit the ocean looks bright blue because the air
|
|
above it scatters sunlight toward the camera. The renderer adds that air
|
|
itself, next.
|
|
|
|
The texture's red, green and blue are reflectances in the same three bands a
|
|
glTF base color uses (see the target model section), folded into the camera's
|
|
channels once at setup as a small matrix, so any channel layout works.
|
|
|
|
Blue Marble is a visual product built from calibrated MODIS reflectances, not a
|
|
calibrated reflectance product itself, so its values are close but not exact
|
|
reflectances. A calibrated alternative (MODIS's albedo product) is a possible
|
|
later refinement.
|
|
|
|
### Earth's atmosphere
|
|
|
|
Code: `atmosphere.rs`, `shaders/atmosphere.wgsl`, `shaders/bodies.wgsl`.
|
|
|
|
At lighting L1 Earth is seen through its atmosphere
|
|
(`Appearance::earth_atmosphere`, on by default). Air does three things to
|
|
light passing through it:
|
|
|
|
- **Molecules scatter it**, much more at short wavelengths: Rayleigh
|
|
scattering grows as 1 / wavelength^4, so blue light at 450 nm scatters 3.6
|
|
times as much as red at 620 nm. Straight up from the ground, air scatters
|
|
about 10 % of green light (an optical depth of 0.11 at 550 nm) and 21 % of
|
|
blue (0.24 at 450 nm). That is why the sky is blue and the setting Sun red.
|
|
- **Aerosols** (dust, sea salt, smoke droplets) scatter colors far more
|
|
evenly than air does, so they make haze white rather than blue, and mostly
|
|
forward, within a few tens of degrees of the light's direction. They sit
|
|
low: half of them in the lowest 0.8 km. Straight up they scatter or absorb
|
|
about as much light as the air does, an optical depth of 0.12 at 550 nm on
|
|
average over Earth, and far more over deserts and fires.
|
|
- **Ozone absorbs** light in a broad band centered near 600 nm (the Chappuis
|
|
band). From space it matters mainly at the limb, where the line of sight
|
|
crosses the ozone layer edge on, and tints the glow there.
|
|
|
|
What leaves the ground is reduced by the light scattered or absorbed out of
|
|
it on the way to the camera: the **transmittance**, between 0 and 1. And the
|
|
air along the line of sight sends sunlight it scatters toward the camera: the
|
|
**path radiance**, which adds to whatever is behind it. Both grow with the
|
|
amount of air the line of sight crosses, so the ground looks hazier and bluer
|
|
toward Earth's edge, where the line of sight is long and slanting. Past the
|
|
edge the line of sight crosses only air, which glows: the **limb**. There the
|
|
air is thin enough that the glow is in proportion to the density at the
|
|
lowest point of the line of sight, so it fades by a factor of e (2.7) for
|
|
every 8 km of height.
|
|
|
|
The ground itself is lit by the sunlight that reaches it, which the air has
|
|
also dimmed (more at a low Sun, and more in blue, so ground near sunset is lit
|
|
redder), and by **skylight**: the light the whole sky scatters down, about a
|
|
tenth of the Sun's in blue with the Sun overhead and less in red. Past the
|
|
terminator the sky still lights the ground for a while: twilight.
|
|
|
|
Light can also scatter more than once, from air to air or off the ground and
|
|
back. With the Sun high that adds a few tens of percent to the sky's
|
|
brightness.
|
|
|
|
Adding up the scattered light along every line of sight for every pixel would
|
|
be far too slow. Instead the renderer precomputes tables once, at setup, with
|
|
the method of Bruneton and Neyret (2008), in the form of Bruneton's 2017
|
|
implementation and with its atmosphere: what a point at a given height sees
|
|
looking in a given direction, with the Sun at a given angle, is a function of
|
|
four numbers (height, view angle from the zenith, Sun angle from the zenith,
|
|
and the angle between the view and the Sun), so it fits a table. Three tables:
|
|
|
|
- the scattered light (4D, stored as a 3D texture of 256 x 128 x 16 texels,
|
|
Rayleigh and aerosols separately);
|
|
- the transmittance from any point to the top of the atmosphere (256 x 64),
|
|
from which the transmittance between any two points follows;
|
|
- the skylight on the ground (64 x 16).
|
|
|
|
Each pixel that sees Earth then reads them about a dozen times: the
|
|
transmittance to the ground and the light scattered along the way (the light
|
|
scattered along the whole line of sight, less what lies beyond the ground),
|
|
and the sunlight and skylight on the ground there.
|
|
|
|
Light scattered more than once is approximated as Hillaire (2020) does:
|
|
assume the light arriving at a point after one scattering comes equally from
|
|
every direction and is the same at nearby points. Then all the higher orders
|
|
together follow from a small table per height and Sun angle, instead of being
|
|
computed order by order.
|
|
|
|
The air is Bruneton's, with more aerosols: Rayleigh scattering falling with
|
|
an 8 km scale height; aerosols with a 1.2 km scale height, an optical depth of
|
|
0.12 at 550 nm (the global annual mean of Kinne et al.'s 2013 climatology)
|
|
falling as wavelength^-1.3, 90 % of it scattering, with Bruneton's forward
|
|
phase function (asymmetry 0.8); and 300 Dobson units of ozone between 10 and
|
|
40 km. Bruneton's aerosol depth of 0.0053 is air cleaner than almost anywhere
|
|
on Earth: with it the ground looked bluer and sharper toward the limb than it
|
|
should. The top of the atmosphere is set
|
|
100 km up. Earth is the sphere of its equatorial radius, in axes stretched
|
|
along the polar axis, where its ellipsoid is that sphere (heights are off by
|
|
at most 0.3 % near the poles).
|
|
|
|
The tables are computed every 20 nm and folded into the camera's channels
|
|
with sunlight and each channel's QE, like every other spectrum
|
|
("Folding spectra into per-channel constants"). The scattered light folds
|
|
exactly. Transmittance does not: a channel whose band spans very different
|
|
transmittances gets their average weighted by sunlight, which is wrong when
|
|
the light passing through is a different color from sunlight. It matters most
|
|
for blue light along long, grazing paths. Setup takes 0.25 s on the 24-core
|
|
desktop CPU these numbers come from, and 1.1 s on four of its cores.
|
|
|
|
Tests: the tables against direct integrals, in `atmosphere.rs` (the amount of
|
|
air from the ground straight up within 1e-4 of the scale height, Rayleigh
|
|
scattered light within 3.4e-4, aerosols within 0.31 %); the shaders against
|
|
the tables, on the GPU and lavapipe (`tests/gpu/atmosphere.rs`): the ground
|
|
seen at 60 degrees from its zenith and the limb at 40 and 50 km, within
|
|
2.5e-4. The limb fades by 3.38 over those 10 km, against 3.49 for the density
|
|
alone; the air between takes the rest.
|
|
|
|
Checked against a real satellite: GOES-16's full disk of 2024-07-01 at
|
|
15:00 UTC, in its blue (470 nm) and red (640 nm) bands, over 640,000 cloud-free
|
|
ocean pixels (NOAA's cloud mask), away from the Sun's glint. Over ocean,
|
|
which is dark, most of the light the satellite measures comes from the air,
|
|
and GOES sees the sea from straight above out to its limb: the same slanting
|
|
paths through the air as a camera in low orbit looking toward the horizon.
|
|
The tables, evaluated at each pixel's angles with Blue Marble's ocean color
|
|
and channels shaped like GOES's bands, against the measured reflectance
|
|
(the median in each band of viewing angle from the zenith):
|
|
|
|
| Viewing angle | GOES blue | Model | Clean air | GOES red | Model | Clean air |
|
|
|---|---|---|---|---|---|---|
|
|
| 20-40 degrees | 0.097 | 0.090 | 0.077 | 0.029 | 0.030 | 0.023 |
|
|
| 55-65 | 0.146 | 0.145 | 0.123 | 0.044 | 0.051 | 0.037 |
|
|
| 70-75 | 0.205 | 0.191 | 0.169 | 0.076 | 0.071 | 0.056 |
|
|
| 78-81 | 0.249 | 0.239 | 0.219 | 0.093 | 0.095 | 0.078 |
|
|
| 81-84 | 0.267 | 0.266 | 0.246 | 0.104 | 0.110 | 0.092 |
|
|
|
|
The model is within 7 % in blue (low: Blue Marble's ocean is darker in blue
|
|
than real open ocean) and 15 % in red out to 84 degrees, and the haze grows
|
|
toward the limb as measured. With Bruneton's clean air ("Clean air") it is 8
|
|
to 29 % too dark. NOAA's own retrieval put the aerosols' optical depth over
|
|
these pixels at 0.13 (median), close to the 0.12 the model assumes.
|
|
|
|
On the RTX 3060 the atmosphere adds 0.8 to 1.0 ms to the bodies pass at
|
|
1920 x 1080 with camera C1's 2 x 2 supersampling, and 14 to 27 ms on lavapipe.
|
|
|
|
Limitations:
|
|
|
|
- **The camera must be above the atmosphere**, over 100 km up; below that a
|
|
frame is refused.
|
|
- **Sunlight reaching the target**, and the Sun and Moon seen past the limb,
|
|
pass through the atmosphere as "Sunset through the atmosphere" describes;
|
|
targets, stars and Earth's own surface seen past the limb do not.
|
|
- **Clouds** are a separate layer, the next section.
|
|
- **The Moon's shadow** in a solar eclipse dims the air along the line of
|
|
sight as it dims the ground point behind it.
|
|
- **Light from Earth onto targets** (Earthshine) includes the atmosphere
|
|
over the ground, but not the glow beyond Earth's edge.
|
|
|
|
### Clouds
|
|
|
|
Code: `clouds.rs`, `shaders/clouds.wgsl`, `shaders/cloudy.wgsl`,
|
|
`shaders/earthshine.wgsl`. From lighting L1, with the atmosphere.
|
|
|
|
Blue Marble's surface is a cloud-free mosaic, but at any moment about two
|
|
thirds of Earth is under cloud, and clouds are the brightest thing on it.
|
|
Without them Earth looks too dark and too clear, and Earthshine on a target is
|
|
too low.
|
|
|
|
The renderer draws clouds as a thin layer 5 km above the ground. Real cloud
|
|
tops run from about 1 to 15 km; one height stands for them all. The layer's map
|
|
is the cloud layer of NASA's original Blue Marble (2002): Terra MODIS clouds
|
|
of 2001 July 29, with thermal infrared over the poles, 1 km per pixel, resized
|
|
to 16384 x 8192 like the Earth maps (`scripts/fetch-clouds.sh`). The same
|
|
clouds show whatever the date.
|
|
|
|
Each pixel's value is read as the cloud's diffuse reflectance there: what it
|
|
reflects of light arriving from every direction alike, from 0 for clear sky to 1.
|
|
Clouds absorb almost no visible light, so what a cloud does not reflect it passes
|
|
on. A thick cloud is bright and hides the ground. A thin one is dim and lets the
|
|
ground show through. The value gives the cloud's optical thickness tau (how many
|
|
times light would be scattered crossing it straight) by the two-stream result of
|
|
Lacis and Hansen (1974):
|
|
|
|
R = 0.13 tau / (1 + 0.13 tau)
|
|
|
|
The 0.13 is sqrt(3) / 2 times (1 - g), with g = 0.85 the cloud droplets' strong
|
|
forward scattering. So tau = 5 reflects 39 %, tau = 20 reflects 72 %, and tau =
|
|
50 reflects 87 %. Over Earth's area the map's mean reflectance is 0.21, and 62 %
|
|
of the area has more than 0.05. The map stores reflectance, so its mip levels
|
|
average reflectance, which keeps a far view of Earth as bright as a near one.
|
|
|
|
A cloud reflects more of a beam the more slanting it is: light entering at a low
|
|
angle scatters out of the top before it gets deep. A Monte Carlo of a cloud slab
|
|
(no absorption, the droplets' scattering as a Henyey-Greenstein function of g =
|
|
0.85, black below, 200,000 photons a case) gives:
|
|
|
|
| tau | Sun overhead | 60 degrees from it | 78 degrees | Diffuse (above) |
|
|
|---|---|---|---|---|
|
|
| 1 | 0.043 | 0.165 | 0.418 | 0.115 |
|
|
| 3 | 0.142 | 0.359 | 0.568 | 0.281 |
|
|
| 10 | 0.422 | 0.603 | 0.732 | 0.565 |
|
|
| 30 | 0.713 | 0.806 | 0.868 | 0.796 |
|
|
| 100 | 0.897 | 0.929 | 0.952 | 0.929 |
|
|
|
|
So the sunlight a cloud reflects, and the shadow it casts, come from tau and the
|
|
Sun's angle by the delta-Eddington method (Joseph, Wiscombe and Weinman 1976),
|
|
in closed form. It takes the forward peak of the droplets' scattering, a
|
|
fraction g^2 of it, as no scattering at all, and the rest as two streams of
|
|
light, up and down. It is within 0.08 of every case of the Monte Carlo, worst for
|
|
thin cloud under a low Sun, where the diffuse value is off by up to 0.30.
|
|
Averaged over the Sun's angles it is within 0.02 of the Monte Carlo's average,
|
|
as the diffuse value is. Sky light, which arrives from every direction, takes
|
|
the diffuse value.
|
|
|
|
For each pixel that sees the ground, the shader:
|
|
|
|
1. finds where the line of sight crosses the layer, and reads the cloud there.
|
|
In an oblique view that is not the cloud straight above the ground point,
|
|
so clouds show parallax against the ground;
|
|
2. finds where the sunlight reaching the ground point crossed the layer, and
|
|
dims the ground's sunlight by what that cloud passes on at the Sun's angle,
|
|
and its sky light by what it passes on of diffuse light. That is the cloud's
|
|
shadow, offset from the cloud away from the Sun: 5 km times the cotangent of
|
|
the Sun's elevation, 29 km with the Sun 10 degrees up;
|
|
3. adds the cloud's own light: the sunlight at 5 km, which passes through less
|
|
air than at the ground, times the cloud's reflectance at the Sun's angle, and
|
|
the sky light there times its diffuse reflectance;
|
|
4. adds what is below the cloud, the ground and the air between it and the
|
|
layer, times what the cloud passes on at the line of sight's angle. Then it
|
|
applies the air above the layer: its glow added, and its transmittance on
|
|
what is under it.
|
|
|
|
So high clouds are less hazed than the ground, the way they look from orbit.
|
|
The ray's entry into the atmosphere is looked up once and shared by the cloud
|
|
and the ground below it.
|
|
|
|
The Earthshine pass samples the same layer, so a target over cloud receives
|
|
more light. A plate 700 km up facing straight down, the Sun overhead, under
|
|
clouds of reflectance 0.5 everywhere, receives 2.0, 1.9 and 1.75 times its
|
|
cloud-free Earthshine in red, green and blue.
|
|
|
|
How bright Earth is as a whole tests the reading of the map's values as
|
|
reflectance. With clouds, rendered at phase angles from 0 to 180 degrees and
|
|
integrated over them, Earth reflects 0.26 to 0.28 of the sunlight in green
|
|
(the Bond albedo), from four sides in July. Without clouds it reflects 0.13 to
|
|
0.16. Measured, Earth's Bond albedo is about 0.30 (CERES, over the whole solar
|
|
spectrum). With the diffuse reflectance alone in place of delta-Eddington it was
|
|
0.27 to 0.29. The geometric albedo, the full disk's brightness seen from the
|
|
Sun's direction, is 0.17 to 0.19 in green here, against 0.43 for the real
|
|
Earth, because clouds and the ocean send back toward the Sun more light than a
|
|
Lambertian surface does.
|
|
|
|
Tests (`tests/gpu/clouds.rs`), on the GPU and lavapipe:
|
|
|
|
- A uniform layer over gray ground, for four reflectances from clear to opaque
|
|
and the Sun 30 and 70 degrees from the zenith, against the atmosphere's tables
|
|
and delta-Eddington in f64 at the cloud's height and the ground's (within
|
|
3.3e-4).
|
|
- A clear map against no map. Lavapipe agrees to 2.4e-7. The RTX 3060 differs
|
|
by up to 6.7e-3 in 0.3 % of pixels, 1.8e-6 on average: its texture filtering
|
|
steps by 1/256 between texels, and a coordinate that differs in its last bit
|
|
between the two pipelines can land on the next step.
|
|
- One opaque cloud with the Sun 10 degrees up, seen from 1000 km straight
|
|
above: the cloud and its shadow 28.7 km to the west land within 0.41 and
|
|
0.12 px of where they project, 261 px apart.
|
|
- The Earthshine above, against the ratio for Earth seen straight down (within
|
|
10 %; 3 to 8 % less, most in blue, as the slant views around the nadir see the
|
|
clouds pass on less of what is below them, and more of their light is the
|
|
air's).
|
|
|
|
Unit tests in `clouds.rs` check the reflectance formulas, delta-Eddington against
|
|
the Monte Carlo above, and the mip averaging.
|
|
The command-line tests check that a missing map names `scripts/fetch-clouds.sh`,
|
|
and that `[appearance] clouds = false` renders without one.
|
|
|
|
Cost: the map takes 360 MB of GPU memory. It is compiled into the bodies pass
|
|
only once set, so frames without clouds cost what they did. With Earth filling
|
|
1920 x 1080 at camera C1, the bodies pass on the RTX 3060 goes from 2.4 to 3.1 ms
|
|
over mostly clear Egypt, and from 2.5 to 3.7 ms over the cloudy Pacific,
|
|
7.2 ms a frame end to end. On lavapipe, the Pacific view goes from 49 to 64 ms.
|
|
Delta-Eddington's division and three exponentials per pixel add 0.03 ms to that
|
|
on the RTX 3060 and nothing measurable on lavapipe. Earthshine: unchanged
|
|
(0.045 ms).
|
|
|
|
Limitations:
|
|
|
|
- **One height**, and no thickness. Clouds stand neither above the limb nor
|
|
above one another, and cast no shadows on each other. Volumetric clouds are
|
|
planned for lighting L2.
|
|
- **Gray, and not absorbing.** The light a cloud reflects leaves it evenly in
|
|
every direction (Lambertian), though how much it reflects follows the Sun's
|
|
angle. Real clouds send more back toward the Sun and forward along the light,
|
|
and show a glory and bright rims at the terminator.
|
|
- **No light between the cloud and the ground**: the ground's light reflected
|
|
back down by the cloud's base is left out. It matters over snow and ice.
|
|
- **The sky light under a cloud** is dimmed by the cloud toward the Sun, not by
|
|
the clouds over the whole sky.
|
|
- **At lighting L1, one day's clouds**, whatever the scene's date. Lighting L2
|
|
draws the scene's day's (next section).
|
|
- **The map** is 2.45 km per pixel. Small cumulus are blurred into a gray
|
|
veil. The composite also has straight seams where its pieces meet, at
|
|
high latitudes, which show as clouds that end on a straight line (east of
|
|
Greenland in the solar eclipse example).
|
|
- **Without the atmosphere** (`Appearance::earth_atmosphere` off), no clouds.
|
|
|
|
### Clouds of the day (lighting L2)
|
|
|
|
Code: `scripts/fetch-dated-clouds.py`, the command line's choice of map. Lighting
|
|
L2.
|
|
|
|
At lighting L1 the clouds are always those of 2001 July 29. Lighting L2 draws the
|
|
clouds of the scene's day, from MODIS on Terra and Aqua, with the same layer and
|
|
shading as above. Only the map differs.
|
|
|
|
**Making a day's map.** NASA's Global Imagery Browse Services (GIBS) serve the
|
|
MODIS cloud products as images, one global composite per day. The cloud optical
|
|
thickness comes as colors that stand for values, in steps of about 4 % from 0.01
|
|
to 150 (GIBS's color map). `scripts/fetch-dated-clouds.py FIRST [LAST]` fetches
|
|
each day's layers as two hemispheres of 8192 x 8192 pixels (GIBS times out on the
|
|
whole map at once) and turns the colors back into numbers. All 252 colors in the
|
|
images are in the color map, so none were blended in the resampling. For each
|
|
pixel it takes the first optical thickness of:
|
|
|
|
1. Terra's standard retrieval;
|
|
2. Aqua's;
|
|
3. Terra's partly cloudy retrieval (PCL);
|
|
4. Aqua's PCL.
|
|
|
|
That thickness becomes reflectance by the formula above. A pixel with none of them
|
|
but a cloud fraction from either satellite is clear, because MODIS retrieves
|
|
thickness wherever its cloud mask finds cloud in daylight. A pixel with no data at
|
|
all, between passes or in polar night, takes the static map's value. The map is
|
|
written at the static map's size, 16384 x 8192, as `data/clouds/YYYY-MM-DD.png`.
|
|
Each day takes about two minutes and 0.3 GB of downloads, and keeps 20 MB.
|
|
|
|
For 2024 July 15 that gives:
|
|
|
|
- thickness over 56 % of Earth's area;
|
|
- clear over 43 %, including the dark Antarctic winter, where only the infrared
|
|
cloud fraction has data;
|
|
- no data over 1.3 %.
|
|
|
|
The mean reflectance is 0.247, against 0.207 for the static map. Fewer pixels are
|
|
cloudy, 55 % against 65 % north of 60 S, but those that are are brighter.
|
|
|
|
**Checking it.** Earth rendered at phase angles from 0 to 180 degrees, as for the
|
|
static map, reflects 0.30 to 0.32 of the sunlight in green (Bond albedo) with the
|
|
day's clouds, against 0.26 to 0.28 with the static map and about 0.30 measured.
|
|
Seen from 80,000 km over Africa, the day's clouds lie where MODIS's true-color
|
|
image of that day shows them: the storms over the North Atlantic, the band over
|
|
the Sahel, and the clouds over Europe. The Saharan dust in that image is aerosol,
|
|
not cloud, and is not drawn.
|
|
|
|
**Choosing the day.** The command line takes the day of the scene's epoch in UTC,
|
|
or `[appearance] cloud_date` in a scene without an epoch. The July example scenes
|
|
give 2024-07-15. A day whose map is missing is an error that names the command to
|
|
fetch it, and a scene with no day at all is an error too. Lighting L2 needs a
|
|
cloud map and Earth's atmosphere, and the renderer refuses it without them.
|
|
`Renderer::set_cloud_map` swaps one map for another between frames, for a run that
|
|
crosses days. It takes about 175 ms on the RTX 3060 and 240 ms on lavapipe, mostly
|
|
building the mip chain and uploading it, and the frames after it are those of a
|
|
renderer made with the new map. Rendering costs what it does at L1.
|
|
|
|
Tests: the day's choice from `cloud_date`, from the epoch on either side of UTC
|
|
midnight, and from a TT epoch half a minute after midnight, which is still the day
|
|
before in UTC; the errors for a missing map and for a scene with no day; L2's
|
|
refusal without clouds or without the atmosphere; and a swap that draws what a
|
|
fresh renderer draws, to the bit, on the GPU and lavapipe.
|
|
|
|
Limitations:
|
|
|
|
- **Seams.** Terra crosses the equator at 10:30 local time and Aqua at 13:30, so
|
|
where one satellite's data gives way to the other's, clouds that moved in three
|
|
hours end on a line. So do the edges of a single satellite's passes, made at
|
|
different times of the day.
|
|
- **One map per day.** The clouds of a whole UTC day are drawn at any time of it,
|
|
though each place is seen by MODIS only around late morning and early afternoon.
|
|
- **Thickness at 1 km, map at 2.45 km.** Broken clouds below that blur, as in the
|
|
static map.
|
|
- **Clear where MODIS saw no cloud,** including where its retrieval failed for
|
|
other reasons, such as sun glint; there the map has no cloud that may have been
|
|
there.
|
|
- **Polar night** takes the static map, which is dark there anyway.
|
|
|
|
### The Moon's surface
|
|
|
|
Code: `moon.rs`, `shaders/bodies.wgsl`; data from `scripts/fetch-moon.py`. From
|
|
lighting L1, with the map in `RendererOptions::moon_texture`.
|
|
|
|
At lighting L1 the Moon's reflectance comes from a map instead of one spectrum.
|
|
The map is the LRO Camera's Wide Angle Camera mosaic at 643 nm (red), built from
|
|
more than 100,000 images taken under every kind of lighting. The camera team
|
|
used a model of how each spot on the Moon reflects light at different angles
|
|
(Hapke's, fitted to the images; Sato et al. 2014) to work out what every spot
|
|
would look like under one standard geometry: the Sun 60 degrees from overhead,
|
|
seen straight down. The map's values are that reflectance, I/F, the ratio of
|
|
the surface's brightness to that of a perfect white surface facing the Sun.
|
|
Over 98 % of the Moon they lie between 0.019 in dark maria and 0.067 on the
|
|
brightest highlands and crater rays, and they average 0.042.
|
|
|
|
The map is 400 m per pixel and covers 70 degrees south to 70 degrees north;
|
|
the polar caps beyond are filled with the average, so they render as smooth
|
|
gray (6 % of the surface, seen only near the limb from Earth). It is resampled
|
|
to 16384 x 8192, 666 m per pixel, the largest texture lavapipe takes, and read
|
|
exactly as Earth's image is: by latitude and longitude in the Moon's own axes,
|
|
with mipmaps, and without a seam at 180 degrees.
|
|
|
|
The Moon's axes come from `Scene::moon_attitude`. The Moon keeps one face
|
|
toward Earth, but rocks back and forth by up to 8 degrees (libration), so
|
|
which features are near the limb changes from month to month. With an epoch,
|
|
the renderer uses the IAU's model of the Moon's rotation; it puts the point
|
|
facing Earth within 0.008 degrees (230 m) of JPL Horizons' high-precision
|
|
frame, under a third of a map pixel. Without one, `Scene::moon_facing_earth`
|
|
turns the Moon's prime meridian toward Earth and is off by the libration, up
|
|
to about 10 degrees.
|
|
|
|
Two choices turn the map into something the renderer can shade:
|
|
|
|
- **Brightness.** The renderer shades the Moon as Lambertian, starting
|
|
from the albedo that gives the map's reflectance at the map's geometry:
|
|
I/F / cos(60 degrees), on average 0.084. On its own that makes the Moon
|
|
far too dark at full: the real Moon is not Lambertian, and it brightens
|
|
steeply toward full Moon, as rough soil hides its own shadows when lit
|
|
from behind the viewer. A Lambert sphere with these albedos is about a
|
|
third as bright as the real full Moon, and slightly too bright past
|
|
quarter. So each point's albedo is multiplied by a phase factor, from the
|
|
phase angle between the Sun and the viewer seen from that point. The
|
|
factor is the whole Moon's measured brightness at that phase (Allen's
|
|
lunar phase curve, 0.026 g + 4e-9 g^4 magnitudes below full, g in
|
|
degrees) over a Lambert sphere's, scaled so that the map's full Moon has
|
|
the measured geometric albedo (the full Moon's brightness against a white
|
|
Lambertian disk of the same size), 0.121 in the V band. It is 2.8 at 4
|
|
degrees, 1.16 at 60, 0.89 at 90 and 0.75 at 120, and holds its 150 degree
|
|
value beyond. At 60 degrees it would be 1 if the map and the measured
|
|
curve agreed exactly; the 16 % between them is the Lambert shape's error
|
|
at that phase together with the two calibrations'.
|
|
|
|
Rendered whole in the example camera's green channel, the Moon is then
|
|
within 5 % of the measured curve from 2 to 30 degrees of phase, and
|
|
within 11 % out to 120 degrees. At the same phase, the side of the Moon
|
|
holding Oceanus Procellarum, the largest of the maria, is darker than the
|
|
other: 0.89 of the curve at 90 degrees against 1.06. The real Moon differs
|
|
between waxing and waning in the same way, which the curve averages out.
|
|
At 150 degrees it is 9-15 % bright.
|
|
|
|
Across the disk the light still falls off as Lambert's does, darkening
|
|
toward the edge, while the real full Moon looks nearly flat out to its
|
|
limb. A lunar reflectance model would fix that.
|
|
- **Color.** The map is one band. Its color comes from the Moon's average
|
|
spectrum across the WAC's seven bands, 321 to 689 nm, from the same mosaics
|
|
averaged over the whole Moon by area: relative to 643 nm, 0.42 at 321 nm, 0.61
|
|
at 415 nm, 0.87 at 566 nm and 1.08 at 689 nm. The Moon is distinctly red,
|
|
which is why it looks yellowish gray to the eye. Each pixel's
|
|
643 nm value scales that spectrum, which is folded into the camera's
|
|
channels once at setup: in the example camera the Moon's albedo in red,
|
|
green and blue is 0.96, 0.83 and 0.69 times its 643 nm value over cos 60.
|
|
Beyond 689 nm the spectrum continues the slope from 643 to 689 nm to 900 nm
|
|
(1.45) and holds there, roughly following real lunar soils, which flatten
|
|
into an absorption by iron-bearing minerals near 1000 nm; below 321 nm it
|
|
holds the 321 nm value. Real maria rich in titanium are up to about 10 %
|
|
bluer than this, and highlands slightly redder; the seven-band mosaic
|
|
could give each pixel its own color later. The real Moon also reddens
|
|
slightly with phase, which the phase factor leaves out.
|
|
|
|
Light the Moon sheds on targets (Moonshine) reads the same map, with the
|
|
phase factor for the angle between the Sun and the target.
|
|
|
|
Tests (`tests/gpu/moon_texture.rs`), with small synthetic maps, on the GPU and
|
|
lavapipe: the pixel under the camera reads the texel at its latitude and
|
|
longitude, and moves when the Moon's attitude does; the color is the spectrum's;
|
|
a uniform map draws exactly the Moon a flat albedo draws times the phase
|
|
factor, at every pixel; there
|
|
is no seam at 180 degrees; a far view averages a fine checker; Moonshine comes
|
|
from the part of the map that faces the target.
|
|
|
|
Cost: the map adds about 0.04 ms to the bodies pass on the RTX 3060, for working
|
|
out where to read it at every pixel (the read itself happens only where the
|
|
Moon is). On lavapipe the pass got faster, from 2.7 to 2.0 ms with the Moon
|
|
filling the view, because Earth's image is now also read only where Earth is.
|
|
Loading the map takes 0.5 s and uploading it 0.17 s; it occupies 360 MB on the
|
|
GPU.
|
|
|
|
### The Sun's disk
|
|
|
|
Code: `sun.rs`, `shaders/common.wgsl`, `shaders/bodies.wgsl`. From lighting
|
|
L1.
|
|
|
|
The Sun's disk is brighter at its center than at its edge: limb darkening.
|
|
Light leaving the Sun comes from a layer a few hundred kilometers deep in
|
|
which the gas gets hotter with depth. Looking at the center of the disk, the
|
|
line of sight goes straight down into hot gas; near the edge it runs through
|
|
the layer at a slant and sees higher, cooler gas, which shines less.
|
|
|
|
Where the line of sight meets the Sun's surface, call mu the cosine of the
|
|
angle between it and the surface's normal: 1 at the center of the disk, 0 at
|
|
the edge, 0.5 at 87 % of the way out. Neckel and Labs (1994) measured the
|
|
brightness across the disk at 30 wavelengths from 303 to 1099 nm, and
|
|
Hestroffer and Magnan (1998) showed that at each wavelength it is close to
|
|
mu^alpha times the center's, to about 1 %. Alpha is 0.94 at 303 nm, 0.48 at
|
|
580 nm and 0.26 at 1099 nm: blue light darkens much more toward the edge
|
|
than red. The renderer uses their values of alpha, interpolated between those
|
|
wavelengths and held constant outside them.
|
|
|
|
The total light from the disk does not change; that is the measured solar
|
|
spectrum. Only its spread does. A disk of brightness mu^alpha averages 2 /
|
|
(alpha + 2) of its center, so the center is (alpha + 2) / 2 times the mean.
|
|
Folded through the example camera's channels, the center is 1.22, 1.26 and
|
|
1.31 times the mean in red, green and blue, and 87 % of the way to the edge
|
|
(mu = 0.5) the disk is 0.70 of its center's brightness in green. The
|
|
folding is exact: for each channel the renderer weights every wavelength's
|
|
profile by that channel's solar flux and tabulates the result, so any channel
|
|
layout works.
|
|
|
|
It matters in two places:
|
|
|
|
- **The Sun in an image.** A camera that looks at the Sun almost always
|
|
saturates, but not with a dark filter, a Sun sensor, or once the lens's
|
|
blur, once modeled, spreads the Sun's light into the pixels around it.
|
|
- **Eclipses.** When Earth or the Moon covers part of the Sun, what counts is
|
|
how much light it hides, not how much of the disk's area. Near the umbra
|
|
only a sliver of the dim edge is left, so less light gets through than the
|
|
area says: 7000 km behind Earth, 25 km inside the penumbra's dark side,
|
|
4.5 % in green against 6.1 %. Near the outside of the penumbra the covered
|
|
part is dim edge, so more gets through: 88.2 % against 86.0 %. In an
|
|
annular solar eclipse the Moon covers the bright center and leaves the dim
|
|
ring: 1 % of the light reaches the ground, not the 2.4 % of the disk's area.
|
|
|
|
The renderer computes the light hidden by an occulter in two parts. Circles
|
|
around the Sun's center that the occulter covers completely count through a
|
|
table of the light inside each radius. Circles it covers in part count by
|
|
adding up, over eight radii, each circle's brightness times the length of its
|
|
arc inside the occulter, with the radii placed by a Gauss-Legendre rule; eight radii are within 2.4e-5
|
|
of what 32 give. The same
|
|
calculation runs on the CPU for each target's eclipse and on the GPU for
|
|
every pixel of Earth and the Moon in an eclipse, per channel.
|
|
|
|
Tests: every pixel of the Sun seen through a 250 mm lens (420 pixels across)
|
|
reads its mean radiance times the profile at its mu, to 6e-5, and the disk's
|
|
light summed over pixels is the channel's solar flux to 7e-5. The light
|
|
hidden by an occulter matches a fine sum over the disk to within 1e-4, and a
|
|
uniform disk gives the area overlap. In eclipses the tests of the sections below use
|
|
the limb-darkened light.
|
|
|
|
Cost: nothing measurable on the RTX 3060 unless an eclipse is in view. With
|
|
Earth filling the image and most of it in the Moon's penumbra, the bodies
|
|
pass takes 0.43 ms instead of 0.21 ms; lavapipe 9.5 ms instead of 5.0 ms.
|
|
|
|
Limitations:
|
|
|
|
- **The very edge.** Hestroffer and Magnan fitted the profiles over the
|
|
inner 90 % of the radius, where measurements are reliable. Right at the
|
|
edge mu^alpha falls to zero, faster than the measured Sun does, over the
|
|
outermost fraction of a percent of the radius.
|
|
- **Wavelengths outside 303 to 1099 nm** use the end values. The example
|
|
camera's channels are inside that range.
|
|
- **A quiet Sun.** Sunspots and bright faculae change the Sun's output by
|
|
about 0.1 % over the solar cycle and are not modeled.
|
|
- **Shadows on the target** (above) still treat the Sun as a square of
|
|
uniform brightness in their penumbra.
|
|
|
|
### Depth from the bodies
|
|
|
|
The pass also writes each hit's depth, near / z, as described in the
|
|
geometry section. The target spacecraft, drawn after the bodies, uses the
|
|
same depth buffer. So a target in front of Earth covers it, and a target
|
|
behind Earth's limb is hidden.
|
|
|
|
## The target model and its materials
|
|
|
|
Code: `model.rs`.
|
|
|
|
The target spacecraft is a triangle mesh loaded from STL or glTF. Each
|
|
piece (an STL file, or a glTF primitive) is a part with one material, and
|
|
for now a material is a Lambertian albedo spectrum, as for Earth and the
|
|
Moon.
|
|
|
|
Surface normals decide how bright each triangle is, through the cos(i)
|
|
factor. STL files store a normal per triangle, but exporters often write
|
|
zeros or wrong values, so the loader ignores them. It recomputes each
|
|
normal from the order of the triangle's corners: counterclockwise as seen
|
|
from outside. Every triangle is then lit as a flat facet, which matches the
|
|
flat panels and boxes most spacecraft are built from. glTF files that store
|
|
normals keep them.
|
|
|
|
glTF describes a material's color as red, green and blue reflectances (its
|
|
"base color"). Three numbers do not define a spectrum: very different
|
|
spectra can look the same color to the eye. The loader turns them into a
|
|
stepped spectrum, blue below 490 nm, green from 500 to 580 nm, and red above
|
|
590 nm. That is a reasonable guess for painted surfaces seen by an RGB
|
|
camera. It is a poor one for a camera whose channels differ from the eye's,
|
|
or for anything in the near infrared, where the red value simply continues.
|
|
For absolute radiometry of a real spacecraft, give the important materials
|
|
measured reflectance spectra (not supported yet).
|
|
|
|
Texture images are not used yet; each material renders with its base color
|
|
value alone. The loader removes textures before reading a glTF file, so a
|
|
model still loads when its textures are in a format or file the loader cannot
|
|
read. A model that relies on textures for its colors (one material with a
|
|
painted image) renders in that material's flat base color, often gray.
|
|
|
|
STL has no units. CAD tools usually export millimeters, so loading a CAD
|
|
model usually needs `scale: 0.001`. glTF is meters by its specification, but
|
|
models in the wild do not always follow it: of the NASA example models,
|
|
Landsat 8 and ICESat-2 are in meters, while the SSL-1300 needs a scale of 0.1
|
|
and ACE about 0.005. Checking a model's size against the real spacecraft's is
|
|
worth the minute it takes.
|
|
|
|
## Drawing the target
|
|
|
|
Code: `gpu/mesh.rs`, `shaders/mesh.wgsl`.
|
|
|
|
The target's mesh is rasterized after the bodies, into the same radiance
|
|
and depth images. Each pixel the target covers is kept only if the target
|
|
is nearer than what is already there. That one rule handles a target in
|
|
front of Earth, a target behind Earth's limb, and parts of the target
|
|
hiding other parts.
|
|
|
|
Each point on the target is shaded like the bodies: albedo times cos(i)
|
|
times the sunlight that reaches it. A panel modeled with no thickness, such
|
|
as a solar array, has one surface but two sides. Whichever side faces the
|
|
camera is the side that is lit. A closed shape never shows its inside, so
|
|
this only matters for thin parts.
|
|
|
|
### Shadows on the target
|
|
|
|
Code: `shadow.rs`, `gpu/shadow.rs`, `shaders/mesh.wgsl`. From lighting L1.
|
|
|
|
A solar array shades the bus behind it, an antenna dish shades its feed,
|
|
and a second spacecraft close by can shade the first. Rasterization draws
|
|
each triangle on its own, so by itself it cannot know that something else
|
|
stands between a point and the Sun. The renderer finds out with a shadow
|
|
map.
|
|
|
|
- **Shadow map.** Before the camera's view is drawn, the target is drawn
|
|
once more as the Sun sees it, looking straight down the Sun's rays. That
|
|
drawing keeps only depth: for each texel (a pixel of the map), how far
|
|
along the Sun's direction the first surface is. The Sun is far enough
|
|
away that its rays are parallel across a spacecraft, so the view has no
|
|
perspective, and a map 4096 texels across covers the target's bounding
|
|
sphere. For Landsat 8, whose sphere is 16.6 m across, one texel is 4 mm.
|
|
- **Lookup.** When the camera's view shades a point, it finds that point
|
|
in the map. If the map holds a surface nearer the Sun than the point, the
|
|
point is in shadow.
|
|
|
|
Each target gets its own map. Another target is drawn into that map when it
|
|
is sunward of this one and close enough across the Sun's rays to shade it.
|
|
A target off to the side is left out, since it cannot cast a shadow here.
|
|
|
|
The Sun is a disk half a degree across, so a shadow edge is not sharp. A
|
|
point near the edge sees part of the disk past the blocker, the same
|
|
penumbra as in an eclipse. The width of the penumbra is the distance from
|
|
blocker to surface times the tangent of the Sun's angular diameter, 0.93 %
|
|
at 1 AU. A solar array 2 m above the bus casts an edge 19 mm wide. One 10 m
|
|
away casts an edge 9 cm wide. The renderer reproduces this with
|
|
percentage-closer soft shadows:
|
|
|
|
1. Search the map around the point for blockers, and average their depth.
|
|
That gives the blocker-to-surface distance, and from it the penumbra
|
|
width.
|
|
2. Compare the point's depth against a 6 x 6 grid of map texels spread over
|
|
that width, and take the fraction that are not blocked as the share of
|
|
the Sun the point sees.
|
|
|
|
Both steps use fixed grids rather than random samples, so the image has no
|
|
noise and the same scene always gives the same pixels.
|
|
|
|
A surface can wrongly shadow itself, because the map only stores its depth
|
|
once per texel. Where the surface is tilted to the Sun, depth changes
|
|
across a texel, and half of each texel looks like it is behind itself. The
|
|
result is a pattern of dark stripes called shadow acne. The renderer
|
|
prevents it three ways. The stored depth is pushed away from the Sun in
|
|
proportion to the surface's tilt. The lookup is made from a point 1.5
|
|
texels off the surface, toward the Sun. And a blocker has to be more than 2
|
|
texels nearer the Sun to count.
|
|
|
|
Tests (`tests/gpu/shadows.rs`) check, on the GPU and lavapipe:
|
|
|
|
- A plate under a 2 m square: the shadow reads zero direct light, and its
|
|
edge falls where the square's edge projects, to within one texel.
|
|
- The measured penumbra width is within 2.2 % of 0.93 % of the occluder's
|
|
height, at heights of 1, 2 and 3 m.
|
|
- No pixel darkens compared with L0 on a plate tilted 80 and 87 degrees
|
|
from the Sun, or on a faceted sphere.
|
|
- A target sunward of another shades it; one to the side does not.
|
|
|
|
On the RTX 3060, shadows cost 0.14 ms of GPU time per frame for Landsat 8
|
|
(85,000 triangles) and 0.34 ms for ICESat-2 (752,000), plus about 0.15 ms
|
|
in the camera view's lookups. Maps 2048 texels across save about 0.15 ms.
|
|
|
|
Limitations:
|
|
|
|
- **Penumbra shape.** The grid averages a square of the map, so brightness
|
|
rises in a straight line across the penumbra. A real disk-shaped Sun
|
|
rises along an S-shaped curve: the full width is the same, but the middle
|
|
80 % of the change is 0.69 of the width rather than 0.8. Limb darkening
|
|
(from L1) would steepen it a little more, and is not modeled here either.
|
|
- **Resolution.** A detail smaller than a texel casts no reliable shadow. A
|
|
gap narrower than about 2 texels between a blocker and the surface it
|
|
shades (a few mm on Landsat 8) lets light through, because of the acne
|
|
margins above.
|
|
- **One shadow lookup per pixel.** Shadows are looked up once per pixel, even
|
|
at camera C1, so the edge of a shadow inside a pixel is not graded by its
|
|
4 samples; the penumbra's own width usually grades it.
|
|
- **Bodies.** Earth and the Moon do not use shadow maps. Their shadows on
|
|
the target are the eclipse factor below, applied to the whole target.
|
|
Light from Earth and the Moon (below) is not shadowed at all.
|
|
|
|
### Eclipses
|
|
|
|
Code: `eclipse.rs`.
|
|
|
|
The Sun is a disk, not a point. So as a spacecraft passes into Earth's
|
|
shadow, the light fades over a few seconds instead of switching off:
|
|
|
|
- **Umbra.** Earth covers the whole solar disk. No direct sunlight.
|
|
- **Penumbra.** Earth covers part of the disk. 7000 km from Earth's
|
|
center, on the night side, the penumbra is about 65 km wide; a satellite
|
|
crosses it in under 10 seconds.
|
|
- **Annular.** A body small enough to sit inside the solar disk blocks a
|
|
ring-shaped bite. From orbit this happens with the Moon.
|
|
|
|
The renderer finds the visible fraction of the solar disk from the overlap
|
|
of two circles on the sky. The circles are the Sun's disk and the
|
|
occulter's, each with its apparent angular radius. It multiplies the
|
|
target's sunlight by that fraction. The model is the standard one for
|
|
satellites (Montenbruck and Gill), the same as astrokit's. A test checks it
|
|
against tracing lines from the target to thousands of points spread over
|
|
the solar disk.
|
|
|
|
From lighting L1 the fraction is of the Sun's light rather than its area,
|
|
per channel, because the disk is brighter at its center (see "The Sun's
|
|
disk"); that makes a penumbra darker toward the umbra and brighter toward
|
|
the outside than the area alone says.
|
|
|
|
The renderer counts both Earth and the Moon as occulters. The fraction is
|
|
computed once per target, at its center. The whole target
|
|
dims together, which is accurate unless the target is a sizable fraction of
|
|
the penumbra's width. With Earth's atmosphere (lighting L1), Earth's part is
|
|
the light that comes through the air past its edge, the next section. Light
|
|
reaching the target from Earth itself is the section after the next two.
|
|
|
|
### Sunset through the atmosphere
|
|
|
|
Code: `atmosphere/grazing.rs`, `eclipse.rs` (`sun_visible_through_air`,
|
|
`moon_near_shadow`), `shaders/bending.wgsl`.
|
|
|
|
From low orbit, the Sun sets behind Earth's atmosphere before it sets behind
|
|
the ground. For the last 20 seconds or so the light reaching a spacecraft has
|
|
crossed hundreds of kilometers of air edge on, and three things happen to it:
|
|
|
|
- **The air takes light out of it**, as anywhere else, but crossing about 70
|
|
times as much air as a path straight up from the same height. Blue goes first, scattered away by air
|
|
molecules. Then, through the ozone layer 20 to 30 km up, ozone's absorption
|
|
band near 600 nm takes more of the red and green than the air takes of the
|
|
blue, so the light is bluish for a few seconds: the turquoise fringe at the
|
|
edge of Earth's shadow in photographs of lunar eclipses. Lower down, below
|
|
about 15 km, the air's scattering wins again and what is left is orange and
|
|
red, until aerosols near the ground take the rest.
|
|
- **The air bends it.** Air's refractive index is a little over 1 and grows
|
|
with its density, so light passing the limb curves toward Earth: by 1.25
|
|
degrees for a ray that just skims the ground, 0.14 degrees at 15 km, 0.04 at
|
|
25 km. (An observer on the ground sees the setting Sun raised by half the
|
|
first figure, the familiar 35 arcminutes of refraction at the horizon.) So
|
|
the Sun is still seen, raised, after it has geometrically set.
|
|
- **The bending spreads it out.** Lower rays bend more, so rays that came in
|
|
parallel leave fanned out downward, and the light spreads over a larger
|
|
angle: refractive dilution. Seen from a spacecraft, the Sun's image is
|
|
squeezed into a flattened ellipse, still with the same surface brightness
|
|
apart from the air's losses, so it delivers less light. From 700 km the
|
|
squeeze is 0.88 for light passing 30 km up, 0.50 at 15 km and 0.27 at 8 km,
|
|
as large an effect there as the air's own losses.
|
|
|
|
Some numbers for a ray seen from 700 km, by how high above the ground it
|
|
passes (the example camera's red, green and blue channels):
|
|
|
|
| Height | Bending | Squeeze | Red | Green | Blue |
|
|
|---|---|---|---|---|---|
|
|
| 40 km | 0.006 degrees | 0.96 | 0.97 | 0.96 | 0.92 |
|
|
| 30 km | 0.021 | 0.88 | 0.58 | 0.59 | 0.69 |
|
|
| 20 km | 0.076 | 0.66 | 0.22 | 0.21 | 0.28 |
|
|
| 15 km | 0.144 | 0.50 | 0.17 | 0.12 | 0.12 |
|
|
| 8 km | 0.367 | 0.27 | 0.10 | 0.03 | 0.01 |
|
|
|
|
Because the atmosphere is taken as a sphere of air whose density depends only
|
|
on height, a ray's whole path through it depends on one number: its **impact
|
|
parameter**, how close its straight line outside the atmosphere passes to
|
|
Earth's center. Two tables of 1024 entries, from the ray that just skims the
|
|
ground to the top of the atmosphere, hold each ray's total bending and its
|
|
transmittance per channel. They are integrals along the curved path, with
|
|
the refractive index proportional to the air's density in the same model as
|
|
the rest of the atmosphere (1.000278 at the ground, for 550 nm), computed at
|
|
setup in 2.4 ms on 24 cores.
|
|
|
|
**The target's sunlight.** Seen from the target, a ray that arrives from an
|
|
angle theta from Earth's center passes at an impact parameter that follows
|
|
from theta, and came from theta less its bending. The renderer adds up the
|
|
Sun's limb-darkened radiance over the solid angle the Sun's image covers,
|
|
each ray's from where it came from, times its transmittance: 64 rows across
|
|
the image, each an arc around Earth's center, 8 points along each. That gives
|
|
all three effects at once, per channel. It takes 17 to 25 microseconds per
|
|
target near the limb, and nothing elsewhere. Through a sunset seen from
|
|
700 km, the light fades from 99 % to 6 % of red, 3 % of blue, over the 0.02
|
|
radians (19 seconds) of orbit before the straight line to the Sun's center
|
|
meets the ground. Refraction keeps that last 6 % coming after the geometric
|
|
edge. The sunset example scene (`examples/scenes/sunset.toml`) shows Landsat 8
|
|
1.4 s before that point, lit by 7.2 % of the Sun's red light and 4.4 % of its
|
|
blue, against the twilight at the limb.
|
|
|
|
**The Sun and Moon seen through the limb.** The bodies pass bends each line
|
|
of sight that passes through the atmosphere: it finds the line's impact
|
|
parameter, turns it toward Earth by its bending about Earth's center (the
|
|
path is symmetric, so the line it leaves on is the one it came in on turned
|
|
by the bending), looks for the Moon or the Sun along that, and multiplies
|
|
what it finds by the transmittance. The Sun near the horizon comes out
|
|
flattened and red, as in photographs from the International Space Station.
|
|
Only frames where the Sun or the Moon is near the limb need it, and the code
|
|
alone, even never run, cost 0.3 ms a frame on the RTX 3060, so it is a second
|
|
version of the bodies pass that only those frames use. In them it adds
|
|
0.4 ms on the RTX 3060 and 6 ms on lavapipe.
|
|
|
|
**The eclipsed Moon.** The same rays reach into Earth's shadow. The lowest
|
|
are bent by 1.25 degrees, so they cross the shadow's axis about 290,000 km
|
|
behind Earth (Earth's radius over that angle). The Moon, at 384,000 km, is
|
|
past that point, so from anywhere in the umbra it sees a thin ring of red
|
|
sunlight around Earth's limb. A ray seen low enough comes from the far side
|
|
of Earth's center, and the target's sum follows it there. Across the umbra
|
|
at the Moon (the example camera's channels, as fractions of the Sun's light):
|
|
|
|
| From the axis | Red | Green | Blue |
|
|
|---|---|---|---|
|
|
| 0 km | 5.9e-5 | 6.1e-6 | 3.5e-7 |
|
|
| 2000 km | 1.6e-4 | 3.2e-5 | 4.7e-6 |
|
|
| 4000 km | 6.5e-4 | 3.2e-4 | 2.0e-4 |
|
|
| 4600 km, the umbra's edge | 1.8e-3 | 1.4e-3 | 1.5e-3 |
|
|
|
|
Near the axis the light comes from rays that pass below about 5 km; 4000 km
|
|
out, from rays about 5 to 19 km up. The center is darkest, because the Moon
|
|
is too far past the point where the lowest rays cross to see them gather on
|
|
the axis. Photometry of the eclipse of June 2011 (Ugolnikov et al.,
|
|
arXiv:1106.6178) found 1e-4 to 1e-3 at 677 nm from the center out to 0.6
|
|
degrees (4000 km), and about 1e-6 at 503 nm. A model of clear air there
|
|
agreed with the outer umbra and was brighter further in, where aerosols took
|
|
light out.
|
|
|
|
A sum for every pixel of the Moon, at 20 to 40 microseconds each, is too much.
|
|
The shadow is symmetric about its axis, so the light at a point depends only
|
|
on two things: its distance from Earth's center, and the angle between the
|
|
Sun and Earth's center seen from it. When the Moon comes near Earth's shadow,
|
|
the renderer puts that light in a table (`EarthShadow`), and the bodies pass
|
|
reads it for each pixel of the Moon.
|
|
|
|
- **Layout.** Two rows of 512 entries, on the near and far sides of the Moon.
|
|
The entries are spaced in the angle less the atmosphere's angular radius,
|
|
which lines the shadow's edges up in both rows.
|
|
- **Contents.** It holds the light's logarithm, which follows the umbra's
|
|
steep falloff better than the light itself does. Entries fall where the
|
|
Sun's edges cross the atmosphere's top, where the light's slope jumps. It
|
|
is within 0.5 % of the sum.
|
|
- **Cost.** Building it takes 1.6 ms on 24 cores and 6.1 ms on 4. It is kept
|
|
while the Moon stays within it, which is 870 km of the Moon's distance, 3.5
|
|
hours or more, and while the Sun's angular size stays within 1e-4 of the
|
|
same. Those frames use the second version of the pass, as for the limb:
|
|
1.80 ms against 1.45 ms for the lunar eclipse example on the RTX 3060, and
|
|
4.7 ms more on lavapipe.
|
|
|
|
`examples/scenes/lunar-eclipse.toml` shows the eclipse of 2025 March 14 at a
|
|
0.3 s exposure: copper red, and brightest along the Moon's northern edge,
|
|
which was farthest from the axis.
|
|
|
|
Tests:
|
|
|
|
- In `atmosphere/grazing.rs`: straight rays' columns of air, aerosols and
|
|
ozone against direct integrals (within 1e-5); the bending and the air along
|
|
the path against rays traced through the atmosphere step by step (within
|
|
1e-4); with no air and no bending, the target's light against the eclipse
|
|
model (within 3e-4: the eclipse model draws Earth's edge as a circle on a
|
|
flat sky, off by 1.7e-4); 64 rows against 512 (2e-5); a point-like Sun
|
|
dimmed by transmittance and dilution (within 1 %); a sunset that dims
|
|
every channel steadily, blue first, with the ozone's blue phase; Earth's
|
|
shadow at the Moon's distance against following 2 million rays back across
|
|
the sky (within 1e-3, measured 5e-4); the eclipsed Moon's light and color
|
|
against the 2011 photometry; and the shadow's table against the sum over the
|
|
Moon's depth (within 6e-3, measured 4.9e-3).
|
|
- In `tests/gpu/atmosphere.rs`, on the GPU and lavapipe: the Sun behind air
|
|
12, 20 and 30 km up, pixel by pixel down the image's middle against the
|
|
same bending and transmittance in f64 (within 7.3e-4); and the Moon behind
|
|
air 15 km up, where each pixel's light over its transmittance and its
|
|
shading is constant to 1.2e-3; and the Moon across the umbra's edge, each
|
|
pixel's light against the sum at its point (within 4.7e-3 on the GPU and
|
|
3.6e-3 on lavapipe, the table's interpolation).
|
|
- In the command-line tests: the eclipse of 2025 March 14 is red, and brighter
|
|
at the Moon's northern edge than at its center and its southern edge.
|
|
|
|
Limitations:
|
|
|
|
- **One refractive index for every wavelength.** Air's changes by about 2 %
|
|
across the visible spectrum, which separates the Sun's colors at the limb
|
|
slightly (the green flash). Not modeled.
|
|
- **The model's scale height near the ground.** The 8 km scale height bends a
|
|
ground-skimming ray by 74.8 arcminutes; the real lower atmosphere, whose
|
|
density falls more slowly, gives about 70.
|
|
- **No clouds or volcanic aerosols.** In reality clouds block most rays
|
|
below 10 km or so. At sunset the light that gets through there is already
|
|
small. In the umbra it is most of the light, so the eclipsed Moon here is as
|
|
bright as after years of clear air. Real eclipses range over a factor of 100
|
|
or more, and are darkest after large eruptions.
|
|
- **No light scattered into the shadow** by the air, which is about 1e-7 of
|
|
the Sun's.
|
|
- **Light from the eclipsed Moon onto targets** still uses Earth's hard edge,
|
|
so it is zero in the umbra.
|
|
- **A target inside the atmosphere** takes the transmittance straight toward
|
|
the Sun's center, without bending.
|
|
|
|
### Eclipses on Earth and the Moon
|
|
|
|
Code: `shaders/common.wgsl` (`sun_visible`), `shaders/bodies.wgsl`. From
|
|
lighting L1.
|
|
|
|
Earth and the Moon also shadow each other. When the Moon passes through
|
|
Earth's shadow it is a lunar eclipse; when the Moon's shadow falls on Earth,
|
|
a solar eclipse. For every pixel that sees Earth or the Moon, the shader
|
|
works out the fraction of the Sun's light that reaches that point on the
|
|
surface past the other body, with the same overlap of circles and the same
|
|
limb-darkened disk as for the target, and multiplies the point's sunlight by
|
|
it, per channel. With Earth's atmosphere, the Moon's points take the light
|
|
through the air instead, from the table in the previous section.
|
|
|
|
The sizes involved:
|
|
|
|
- At the Moon's distance, Earth's umbra reaches 4600 km from the shadow's
|
|
axis and its penumbra 8200 km. The Moon, 3474 km across, fits inside the
|
|
umbra with room to spare, so a total lunar eclipse lasts over an hour.
|
|
- The Moon's umbra only just reaches Earth. When the Moon is near its
|
|
closest, the umbra touches down as a dark spot up to about 270 km across:
|
|
a total solar eclipse. When it is farther, the umbra ends in space, and
|
|
beyond its tip the Moon covers only the middle of the Sun's disk, leaving a
|
|
ring: an annular eclipse, where about 1 % of the Sun's light still arrives,
|
|
the dim edge of the disk. The
|
|
penumbra around either is about 7000 km across.
|
|
|
|
The overlap is computed in 32-bit floats, which is enough: a lunar surface
|
|
point is placed to about 20 m out of 384,000 km, 6e-8 radians against a
|
|
solar radius of 4.6e-3 radians seen from there. One formula needed care. The
|
|
area where the two circles overlap involves angles whose cosines are close
|
|
to 1 when a sliver of the Sun is covered, and the inverse cosine of a number
|
|
near 1 loses most of its precision in 32 bits. The shader finds those angles
|
|
from their sine and cosine together instead. With the inverse cosine,
|
|
pixels at the shadow's edges came out wrong: a point the 64-bit model puts
|
|
fully in the umbra read slightly lit, and one just outside the shadow
|
|
slightly dimmed.
|
|
|
|
Tests (`tests/gpu/eclipses.rs`), on the GPU and lavapipe, compare each pixel of
|
|
Earth and the Moon against the 64-bit model at that pixel's surface point:
|
|
the Moon entirely in Earth's umbra, the Moon across the umbra's edge, a
|
|
total solar eclipse and an annular one. Pixels the model puts fully in shadow
|
|
read zero, those it puts fully outside are unchanged (both to within 1e-6,
|
|
the 32-bit rounding at the shadow's edges), and the rest
|
|
agree to within 1e-4 of the Sun's light (1.2e-6 at most on the GPU, 5e-5 on
|
|
lavapipe). The command-line tests render two real eclipses from their
|
|
epochs: the total lunar eclipse of 2025 March 14 at mid-totality, where the
|
|
whole Moon is in the umbra (red through the atmosphere, previous section), and
|
|
the total solar eclipse of 2026 August 12,
|
|
where the point of greatest eclipse in NASA's bulletin (65.2 N 25.2 W) is in
|
|
the umbra (`examples/scenes/lunar-eclipse.toml`, `solar-eclipse.toml`). In
|
|
the second, the umbra is 220 by 410 km in the image, stretched along the
|
|
direction of the Sun, which is 26 degrees high there; NASA gives the path as
|
|
294 km wide.
|
|
|
|
The light Earth and the Moon reflect onto targets (next section) includes
|
|
these shadows, so an eclipsed Moon sheds less light.
|
|
|
|
On the RTX 3060 this adds nothing measurable to the bodies pass unless an
|
|
eclipse is in view; with Earth filling the image and most of it in the
|
|
Moon's penumbra, the pass takes 0.43 ms instead of 0.20 ms, most of it the
|
|
limb-darkened sum (see "The Sun's disk"). On lavapipe, 9.5 ms instead of
|
|
3.9 ms.
|
|
|
|
Limitations:
|
|
|
|
- **Earth's atmosphere.** With it, the umbra on the Moon is lit red through
|
|
the air (previous section). Without it, the umbra is black. The observed
|
|
umbra is about 2 % larger than the solid Earth's, which is not modeled.
|
|
- **Smooth shapes.** Earth's shadow is cast by a sphere of its equatorial
|
|
radius, as for the target's eclipses; its flattening would move the
|
|
shadow's edge on the Moon by about 20 km. The Moon's mountains and valleys,
|
|
which shape the edge of a solar eclipse's path, are not modeled.
|
|
|
|
### Light from Earth and the Moon
|
|
|
|
Code: `gpu/earthshine.rs`, `shaders/earthshine.wgsl`, `shaders/mesh.wgsl`.
|
|
From lighting L1.
|
|
|
|
Earth reflects about a third of the sunlight that reaches it, and some of
|
|
that lands on the target: Earthshine. From low orbit it is large. At 500 km,
|
|
Earth fills a circle 68 degrees in radius around the point straight down,
|
|
3.9 of the sky's 12.6 steradians. A plate facing straight down over the
|
|
sunlit Earth receives up to about a quarter as much light from Earth as a
|
|
plate facing the Sun receives from the Sun. A white Earth would give 0.85
|
|
(the test below measures that case). At geostationary distance, Earth is 17
|
|
degrees across and a plate facing a full Earth gets about 0.5 % of sunlight.
|
|
The Moon does the same, Moonshine, much more weakly: a full Moon seen from
|
|
near Earth gives about a millionth of sunlight. From 100 km above the Moon,
|
|
though, a plate facing it gets about a tenth of sunlight.
|
|
|
|
This matters most where the Sun does not reach: the target's night side,
|
|
and parts in a shadow of its own. Without it those read as black, the bias in
|
|
DN; with it they show their structure, dimly.
|
|
|
|
The renderer works out, for each target and each frame, how much light
|
|
arrives from Earth and the Moon on a surface facing any direction:
|
|
|
|
1. **What the target sees of each body.** The circle on the sky that the body
|
|
fills is cut into 1024 cells of equal solid angle: 32 rings from the
|
|
center out to the edge, 32 cells around each. A line from the target
|
|
through each cell's center finds the point it meets on the body, and that
|
|
point is shaded exactly as the camera's view of the body is shaded:
|
|
reflectance from the Blue Marble image (or the stand-in spectrum), lit by
|
|
the cosine of the Sun's angle there, dark on the night side, and at
|
|
lighting L1 seen through the atmosphere. Each texture
|
|
read averages the patch of ground the cell covers, about 1000 square
|
|
kilometers straight down from 500 km and more toward Earth's edge, so a
|
|
cell sees the mean color of that patch rather than one point. A line
|
|
that meets the other body first counts nothing, so the Moon can hide part
|
|
of Earth and Earth part of the Moon.
|
|
2. **Light for every direction a surface can face.** For a surface facing
|
|
direction n, the light it receives is the sum over all cells of the cell's
|
|
brightness, times its solid angle, times the cosine between n and the
|
|
cell's direction (zero for cells behind the surface). The renderer adds
|
|
this up for a grid of directions, a cube map 16 texels across on each of
|
|
its six faces.
|
|
|
|
Drawing the target, each point reads that cube at its surface normal and
|
|
adds albedo / pi times the result to the direct sunlight. The cube is computed
|
|
at the center of each target's bounding sphere and used for the whole target;
|
|
across a spacecraft the view of Earth barely changes.
|
|
|
|
Earthshine is not dimmed by the target's eclipse factor, because the two
|
|
are different views of Earth. From anywhere in Earth's umbra only the night
|
|
side is in view, so Earthshine is zero there without being told. In the
|
|
penumbra the Sun is partly hidden but a crescent of lit ground is still in
|
|
view, and that light still arrives.
|
|
|
|
Tests (`tests/gpu/earthshine.rs`) compare a plate's brightness on the GPU and
|
|
lavapipe against the same sum done in 64-bit floating point with 90 times as
|
|
many cells: over the subsolar point at 500 km facing down, sideways and up;
|
|
toward the terminator; in Earth's shadow; at GEO; 100 km above the Moon at
|
|
full and quarter phase; with each body hiding the other; with the plate seen
|
|
from several sides, so each cube face a camera-facing surface reads is checked. Every
|
|
case where the bodies send more than a thousandth of sunlight agrees to
|
|
within 0.5 %. The largest differences come from the cube: between texels
|
|
5.6 degrees apart its interpolation rounds off the peak of a surface facing
|
|
a full Earth or Moon, by 0.4 %. A cube 32 texels across cuts that to 0.13 %
|
|
but costs four times as much, and the cells' own error on a half-lit distant
|
|
Earth (0.4 %) stays. A thin crescent of lit ground at Earth's edge, seen from
|
|
low orbit near the terminator, is off by 4 % of a small amount, 7e-5 of
|
|
sunlight, about 0.1 DN on a gray surface in the example camera. Another test
|
|
checks the sum itself against the classical formula for a distant sphere,
|
|
(2/3) albedo (R/D)^2 Phi(alpha), to within 0.3 %.
|
|
|
|
On the RTX 3060 this costs 0.04 ms of GPU time per target (0.28 ms for
|
|
eight), and 1.4 ms per target on lavapipe.
|
|
|
|
Limitations:
|
|
|
|
- **No shadows.** The target's own parts do not block light from Earth or
|
|
the Moon. A panel facing Earth does not shade the bus behind it from
|
|
Earthshine, and the inside of a dish facing Earth is lit as if open to the
|
|
whole Earth. Earth fills too much of the sky for one shadow map to stand in
|
|
for it; a ray tracer would handle this.
|
|
- **One spectrum per channel.** Light from Earth is bluer than sunlight, but
|
|
the target's albedo in each channel was worked out under sunlight. This is
|
|
the shortcut described in "Folding spectra into per-channel constants",
|
|
and its error is largest for strongly colored surfaces.
|
|
- **Lambertian bodies.** Earth and the Moon reflect as Lambertian surfaces
|
|
here, as they do in the camera's view, with the Moon's map scaled by its
|
|
phase curve as seen from the target. Clouds are Lambertian too.
|
|
|
|
## Where the Sun and Moon appear
|
|
|
|
Code: `scene.rs` (`Velocities`, `aberrate`), `ephemeris.rs` (feature
|
|
`astrokit`), `shaders/bodies.wgsl` (`seen`).
|
|
|
|
With the `astrokit` feature, the Sun's and Moon's positions come from JPL's
|
|
DE440s ephemeris, and Earth's orientation from the ITRF93 kernel, which
|
|
includes the slow wobble of Earth's pole. These are geometric positions:
|
|
where each body is at the epoch.
|
|
|
|
A camera sees something slightly different, its apparent position:
|
|
|
|
- **Light time.** Light from the Moon takes 1.3 s to reach Earth, and from
|
|
the Sun 8.3 minutes. The camera sees where each body was when the light
|
|
left it. For the Moon that is about 1.3 km of motion, a few microradians.
|
|
- **Aberration.** A moving camera sees incoming light tilted toward its
|
|
direction of motion, as rain seems to slant toward a moving car. Earth's
|
|
orbital speed (30 km/s) tilts the Sun's apparent direction by about 20
|
|
arcseconds, 100 microradians. A spacecraft's own orbital velocity adds up
|
|
to about 5 arcseconds more.
|
|
|
|
100 microradians is about one pixel of the example camera, and 18 pixels of
|
|
a narrow camera with a 1 m focal length.
|
|
|
|
The two effects combine simply. Light time moves a body back along its own
|
|
velocity by (distance / c) times that velocity; aberration moves it along
|
|
the observer's velocity by the same factor. To first order, then, a body
|
|
appears moved in direction by the observer's velocity relative to the body,
|
|
divided by c, the part of it across the line of sight. Only relative
|
|
velocities matter. For the Sun that is Earth's velocity around it plus the
|
|
spacecraft's: the 20 arcseconds. For the Moon, Earth's 30 km/s cancels,
|
|
since the Moon shares it, and what is left is the Moon's own 1 km/s around
|
|
Earth (0.7 arcseconds from Earth's center) plus the spacecraft's velocity.
|
|
For Earth itself it is the spacecraft's velocity, up to 5 arcseconds in low
|
|
orbit, about 0.2 px of the example camera. The terms left out are of the
|
|
order of the velocity squared over c squared, 1e-8 rad.
|
|
|
|
From camera tier C1 the renderer draws Earth, the Moon and the Sun at their
|
|
apparent positions. The scene carries the velocities (`Scene::velocities`:
|
|
the observer's relative to Earth, Earth's relative to the Sun and the Moon's
|
|
relative to Earth), and `Ephemeris::scene` fills in Earth's and the Moon's;
|
|
the observer's is the caller's. All zero, as in C0, the images are
|
|
geometric. Each pixel's ray is turned back, per body, by that body's
|
|
correction before it is tested against the body, so the correction follows
|
|
the ray's direction across a wide view of Earth, and the lighting, shadows
|
|
and eclipses, which happen where the bodies really are, stay geometric.
|
|
`Scene::apparent_sun` and `Scene::apparent_moon` give the same directions on
|
|
the CPU. From Earth's center they match JPL Horizons' apparent positions
|
|
(light time and stellar aberration) to under 0.002 arcseconds, against 20
|
|
arcseconds for the Sun's geometric position.
|
|
|
|
The camera's attitude is taken as the observer would measure it: a star
|
|
tracker sees the stars aberrated by the same motion, so the attitude it
|
|
reports already includes it. A simulation that passes a true, geometric
|
|
attitude gets images shifted by the aberration, as a real camera's would be
|
|
against such an attitude.
|
|
|
|
Still geometric: the targets, whose light time and aberration come from
|
|
their velocity relative to the observer, meters per second in rendezvous,
|
|
under 0.01 microradians; the rotation and lighting of a body during its
|
|
light time (the Moon turns 3 microradians in 1.3 s); and the light-time
|
|
delay between the Sun lighting a body and the camera seeing it.
|
|
|
|
## Stars
|
|
|
|
Code: `stars.rs` (`StarCatalog`, `StarPhotometry`); `gpu/stars.rs`,
|
|
`shaders/stars.wgsl`; `scripts/fetch-stars.py`.
|
|
|
|
Below lighting tier L1 the sky is black. A real camera shows stars wherever
|
|
its exposure is long enough, and they matter: star trackers find their
|
|
attitude from them, a camera measuring the Moon or a target measures it
|
|
against them, and to an algorithm looking for a target a star is a false one.
|
|
|
|
Stars are part of the scene, so they come with lighting tier L1, but only a
|
|
camera at tier C1 or above shows them. A C0 pixel takes its value from the
|
|
one point at its center, and a star is a point that almost never lies
|
|
exactly there; from C1 each pixel collects the light over its whole area,
|
|
which is what records a star.
|
|
|
|
**The catalogs.** `scripts/fetch-stars.py` merges three of ESA's catalogs
|
|
into `data/stars/stars.bin`: Hipparcos, 118,000 stars complete to about
|
|
magnitude 7.3; Tycho-2, 2.5 million stars to about 11.5 (99 % complete to
|
|
11.0); and Gaia DR3, from which it takes every star brighter than G 15 by
|
|
default, 37 million. Each star comes from the first catalog that has it, so
|
|
the brightest, which saturate Gaia's detectors, keep their Hipparcos or Tycho
|
|
measurements. Gaia measures brightness in its own bands, G and the color
|
|
BP - RP: V comes from them by Gaia's published relation (to 0.03 mag), and
|
|
B - V by a fit to the Hipparcos stars Gaia matched. Each star
|
|
keeps its position on the sky (ICRS, at epoch J2000), its proper motion and
|
|
its brightness and color. The renderer moves each star by its proper motion
|
|
to the scene's epoch when the catalog is loaded. Barnard's star, the
|
|
fastest, moves 10 arcseconds a year; most move under 0.1. Parallax, under 1
|
|
arcsecond for every star, is left out.
|
|
|
|
**Magnitudes.** A star's brightness is given as a magnitude: each step of 5
|
|
is a factor of 100, and larger is fainter. Vega is about 0, the faintest
|
|
stars the eye sees about 6, Tycho-2's limit about 11.5, 25,000 times fainter
|
|
than 0. The magnitude in the Johnson V band, a filter close to the eye's
|
|
green-yellow, fixes how much light the star sends through that band: V = 0
|
|
is 3.631e-11 W/m^2/nm averaged over it (Bessell, Castelli and Plez 1998).
|
|
The color index B - V, the difference between the star's magnitudes in blue
|
|
and in V, fixes its temperature: hot stars are blue, B - V below 0, and cool
|
|
ones red, up to about 2. The renderer takes the temperature from B - V
|
|
(Ballesteros 2012: the Sun's 0.65 gives 5,780 K), takes a blackbody spectrum
|
|
at that temperature, scales it so its light through the V band matches the
|
|
star's magnitude, and folds it through the camera's optics and channels, as
|
|
for sunlight. A star's spectrum is not a blackbody's: absorption lines and,
|
|
in cool stars, molecular bands move a few percent of the light between
|
|
channels. For the example camera, a Sun-colored star of V = 0 on the
|
|
boresight gives 56 electrons in green in the 0.1 ms exposure, against an
|
|
independent estimate of 56.4.
|
|
|
|
**Exposure.** That 56 electrons is why the example scenes show no stars: an
|
|
exposure set for a sunlit spacecraft or Earth, a fraction of a millisecond,
|
|
leaves even the brightest stars at a few hundred electrons and the rest
|
|
below one. Star trackers expose for 10 to 200 ms, and a telescope's larger
|
|
aperture collects more: a 1 m f/8 telescope, with a 12.5 cm aperture, gives
|
|
100 times what the example camera does.
|
|
|
|
**Drawing.** Each frame the renderer finds the stars in the field on the
|
|
CPU, from bands of declination sorted by right ascension, so it looks only
|
|
at the stars near the field. It places each at its apparent position, the
|
|
annual aberration of the observer's motion (about 20 arcseconds, see the
|
|
previous section) through the lens's distortion. It works out its electrons
|
|
in each channel: a point source loses light off axis only to the aperture's
|
|
foreshortening, cos(theta), not the cos^4(theta) of an extended surface,
|
|
since all of a star's light that enters lands in its spot. A star is not
|
|
rendered and then blurred: a point of light rendered into a pixel loses
|
|
where in the pixel it fell (see the blur section). Instead its electrons are
|
|
spread over the pixels around it as the Gaussian PSF integrated over each
|
|
pixel's area, which is exact for any position within a pixel. (Camera C2
|
|
draws stars into the focal plane before its convolution instead, so they
|
|
take the Airy or measured PSF too; see the PSF's wings.) The detector
|
|
adds them to the light from everything else before the full well, so a
|
|
bright star saturates like any other light. A star is hidden wherever
|
|
anything nearer is drawn, Earth, the Moon or a target: stars are at
|
|
infinity, so a star shows only where the depth image is empty. Stars giving
|
|
fewer than 0.1 electrons in every channel, far below any detector's noise, are
|
|
skipped, and the catalog is loaded only to the magnitude that can reach that.
|
|
|
|
Measured: a star moved across a pixel in tenths, behind a lens with
|
|
distortion and with Earth's 30 km/s of aberration, lands where the
|
|
projection puts its apparent direction to 0.01 px, with its electrons
|
|
matching the photometry to 0.2 %. A star behind Earth's night side or
|
|
behind a target leaves the bias alone.
|
|
|
|
Costs. A catalog is loaded only as deep as the camera's exposure can show:
|
|
for the example camera at 0.1 ms that is V 7.9, 42,000 stars, in 6 ms and 7
|
|
MB; at 50 ms, V 14.7, 21 million stars, in 2.8 s and 670 MB; for a 1 m
|
|
telescope at 0.2 s, the whole catalog, 37 million stars, in 5 s and 1.2 GB.
|
|
Each frame, finding and placing the stars takes 23 microseconds for the
|
|
telescope's 0.6 degree field (490 stars toward the Milky Way in Monoceros)
|
|
and 4.8 ms for the example camera's 12 degree field at 50 ms (52,000 stars).
|
|
Drawing them takes 0.24 ms for 40,000 stars on an RTX 3060, and 5.6 ms on
|
|
the CPU through lavapipe.
|
|
|
|
A catalog's depth matters most for a telescope's narrow field. Tycho-2 alone
|
|
averages 60 stars per square degree, so a 1 m telescope's 0.2 square degrees
|
|
would hold about a dozen; with no sky background in space such a telescope
|
|
records stars to magnitude 14 or 15 in half a second, and Gaia to G 15
|
|
averages about 900 per square degree. Deeper still (G 16, 78 million stars)
|
|
is a run of the script with `--gaia 16`, at twice the size.
|
|
|
|
## Not modeled yet
|
|
|
|
- Earthshine on the Moon's night side
|
|
- A point spread function that changes across the field, and the PSF's wings
|
|
past the reach of camera C2's grid
|
|
- Non-Lambertian materials (BRDFs), glints, and the full Moon's edge as bright
|
|
as its center
|
|
- Clouds as volumes (lighting L2): thickness, the shadows they cast on each
|
|
other, and clouds standing above the limb
|
|
- Earth's surface finer than Blue Marble's 460 m: mosaics of tens of meters
|
|
(Landsat, Sentinel-2) need tiles streamed in as a view needs them
|
|
- The atmosphere seen from inside it (a camera below 100 km)
|
|
- Motion blur, rolling shutter, stray light
|