832 lines
41 KiB
Markdown
832 lines
41 KiB
Markdown
# Optics and radiometry guide
|
|
|
|
This explains the physics the renderer models, for a reader who knows
|
|
orbital mechanics but not optics. Each section says which code implements
|
|
it. When a feature adds a new optical effect, it adds or updates a section
|
|
here in the same change.
|
|
|
|
Numbers in the examples come from `CameraConfig::example()` unless stated
|
|
otherwise: 1920 x 1080 pixels, 5.5 um pitch, 50 mm focal length, f/4, three
|
|
color channels, 12-bit output.
|
|
|
|
## The chain from Sun to number
|
|
|
|
Every pixel value comes from the same sequence:
|
|
|
|
1. The Sun emits power across a range of wavelengths.
|
|
2. Some of it lands on a surface (Earth, the Moon, the target), which
|
|
reflects part of it in various directions.
|
|
3. The fraction reflected toward the camera enters the lens aperture. The
|
|
optics focus it onto one pixel, losing a little to absorption.
|
|
4. The pixel converts some of the arriving photons into electrons. The
|
|
fraction depends on wavelength and on the color filter over that pixel.
|
|
5. The electrons pile up during the exposure, up to a limit.
|
|
6. The readout electronics turn the electron count into an integer, the
|
|
digital number (DN).
|
|
|
|
The renderer follows this chain with physical units at every step. Image
|
|
brightness therefore comes from the scene and the camera settings, not from
|
|
a tuned scale factor. Change the exposure time, and the image changes
|
|
exactly as a real camera's would.
|
|
|
|
## Geometry: the pinhole camera
|
|
|
|
Code: `camera.rs` (`Intrinsics`), `scene.rs`.
|
|
|
|
A pinhole camera maps each direction in space to one point on the focal
|
|
plane. Put the pinhole at the origin, looking along +Z. A point at camera
|
|
coordinates (x, y, z) lands on the focal plane at (f x / z, f y / z), where
|
|
f is the focal length. Dividing by the pixel pitch p converts focal-plane
|
|
distance to pixels:
|
|
|
|
```text
|
|
u = (f / p) * x / z + cx
|
|
v = (f / p) * y / z + cy
|
|
```
|
|
|
|
`f / p` is the focal length in pixels. (cx, cy) is the principal point,
|
|
where the boresight hits the image, normally the image center. For the
|
|
example camera, f / p = 50 mm / 5.5 um = 9091 pixels.
|
|
|
|
Some quantities that follow from that:
|
|
|
|
- **IFOV, the instantaneous field of view.** The angle one pixel covers:
|
|
p / f = 110 microradians. At 10 m that is 1.1 mm per pixel; at 7000 km
|
|
it is 770 m per pixel. This sets the smallest detail the camera can
|
|
resolve.
|
|
- **FOV, the field of view.** The angle the whole image covers:
|
|
2 atan(width / 2 / (f / p)), about 12.1 x 6.8 degrees here. A longer
|
|
focal length or smaller sensor gives a narrower field of view and finer
|
|
IFOV.
|
|
|
|
Real lenses are not perfect pinholes. They bend straight lines slightly
|
|
(distortion, planned in M3) and spread each point into a small blur (the
|
|
point spread function, also M3). The pinhole model is the starting point
|
|
those effects are added to.
|
|
|
|
### Coordinate conventions
|
|
|
|
The camera frame has +Z along the boresight, +X toward increasing column u,
|
|
and +Y toward increasing row v. This is the computer-vision convention: row
|
|
0 is the top of the image, so +Y points down in the picture. Pixel (i, j)
|
|
covers u in [i, i+1) and v in [j, j+1), so its center is at (i + 0.5,
|
|
j + 0.5).
|
|
|
|
### Why the GPU sees camera-relative coordinates
|
|
|
|
GPUs compute in 32-bit floats, which carry about 7 significant digits. A
|
|
spacecraft 7000 km from Earth's center, stored in Earth-centered f32
|
|
coordinates, can only be placed to the nearest 0.5 m. A target 10 m away
|
|
would jitter by several pixels. So all positions stay in 64-bit floats on
|
|
the CPU. They are made relative to the camera there, and converted to
|
|
32-bit only after that. A target 10 m from the camera is then placed to
|
|
about a micrometer.
|
|
|
|
### Depth
|
|
|
|
When two things overlap in the image, the nearer one must win. The GPU
|
|
keeps a depth value per pixel for this. The renderer stores near / z,
|
|
which is 1 at the near clip distance and falls toward 0 with distance.
|
|
Combined with how floating-point numbers are spaced, this gives roughly the
|
|
same relative precision at every distance. That matters in a scene that
|
|
contains a 10 m target, Earth at thousands of kilometers, and the Sun at
|
|
150 million kilometers. The scheme is called reversed-Z with an infinite
|
|
far plane.
|
|
|
|
## Radiometric quantities
|
|
|
|
Code: `spectrum.rs`, `radiometry.rs`.
|
|
|
|
Light has a few distinct measures, and the camera measures a specific one.
|
|
|
|
- **Irradiance, E, in W/m^2.** Power arriving per unit area of a surface.
|
|
Sunlight at 1 AU delivers about 1361 W/m^2 to a surface facing the Sun.
|
|
It falls off with the square of the distance from the Sun. A surface
|
|
tilted by angle theta from the Sun direction receives E cos(theta).
|
|
- **Solid angle, in steradians (sr).** The 2D version of an angle: the area
|
|
a cone cuts out of a unit sphere. The full sphere is 4 pi sr. One pixel
|
|
of the example camera covers (110 urad)^2, about 1.2e-8 sr.
|
|
- **Radiance, L, in W/m^2/sr.** Power per unit area per unit solid angle,
|
|
traveling in a particular direction. It is what a camera pixel responds
|
|
to: how bright a surface looks from a given direction.
|
|
|
|
Radiance has a property that explains a lot of what a camera sees: it does
|
|
not change along a ray through empty space. Move a camera twice as far from
|
|
a wall, and each pixel sees four times as much wall area. But each square
|
|
meter now contributes a quarter of the power, so the pixel value is
|
|
unchanged. An object that covers many pixels has the same surface
|
|
brightness at any distance; it just covers fewer pixels farther away.
|
|
|
|
That stops being true once the object is smaller than a pixel. The pixel
|
|
then also sees dark space around the object, so the signal falls as one
|
|
over distance squared. That is the regime of a distant satellite seen as a
|
|
point (space situational awareness), as opposed to the resolved imagery
|
|
this renderer is mostly for.
|
|
|
|
### Reflection: albedo and Lambertian surfaces
|
|
|
|
A Lambertian surface reflects light equally in every direction. Matte
|
|
paint, the Moon (roughly) and cloud tops behave close to this. If such a
|
|
surface receives irradiance E and reflects a fraction rho (its albedo), its
|
|
radiance in every direction is:
|
|
|
|
```text
|
|
L = rho * E / pi
|
|
```
|
|
|
|
The pi comes from integrating the reflected radiance over the hemisphere
|
|
above the surface. A white Lambertian sheet facing the Sun at 1 AU, with
|
|
rho = 1, has a radiance of about 1361 / pi = 433 W/m^2/sr, summed over all
|
|
wavelengths. Earth's average albedo is about 0.3.
|
|
|
|
Real spacecraft surfaces are not Lambertian. Multilayer insulation (MLI)
|
|
and solar cells are partly mirror-like and produce bright glints at
|
|
particular angles. These are modeled by a BRDF, a function giving reflected
|
|
radiance for every pair of incoming and outgoing directions. That arrives
|
|
in M4.
|
|
|
|
## Wavelength: spectra, channels and photons
|
|
|
|
Code: `spectrum.rs` (`Spectrum`), `camera.rs` (`Channel`).
|
|
|
|
Everything above varies with wavelength. Sunlight peaks around 500 nm and
|
|
extends from the ultraviolet into the infrared. Surfaces reflect different
|
|
wavelengths differently; that is what color is. Optics transmit some
|
|
wavelengths better than others. The sensor's sensitivity also depends on
|
|
wavelength.
|
|
|
|
The renderer describes each of these as a `Spectrum`: a value as a function
|
|
of wavelength, in meters.
|
|
|
|
- **Quantum efficiency (QE).** The fraction of arriving photons that become
|
|
electrons in a pixel. A color camera puts red, green and blue filters
|
|
over different pixels, so each channel has its own QE curve. The example
|
|
camera uses idealized Gaussian curves centered at 620, 540 and 460 nm,
|
|
80 nm wide at half height, peaking at 60 %.
|
|
- **Transmission (tau).** The fraction of light the lenses pass, about 0.9
|
|
for the example camera.
|
|
|
|
Detectors count photons, not energy. A photon of wavelength lambda carries
|
|
energy h c / lambda (h is Planck's constant, c the speed of light). So one
|
|
watt of red light delivers more photons than one watt of blue. The
|
|
renderer converts power to photons per wavelength before applying QE.
|
|
|
|
### The solar spectrum
|
|
|
|
Code: `radiometry.rs` (`solar_spectrum`), generated into `solar_data.rs` by
|
|
`scripts/fetch-solar-spectrum.sh`.
|
|
|
|
The renderer uses a measured solar spectrum, the TSIS-1 Hybrid Solar
|
|
Reference Spectrum (Coddington et al. 2021). It is the current
|
|
international reference, built mainly from the TSIS-1 instruments on the
|
|
International Space Station and filled in with a high-resolution model of
|
|
the solar absorption lines. It gives the irradiance at 1 AU, outside the atmosphere,
|
|
every 1 nm from 202 to 2730 nm. Its uncertainty is 0.3 % from 460 to
|
|
2365 nm. Over that range it sums to 1325.8 W/m^2. The other 35 W/m^2 of the
|
|
1361 W/m^2 total is at wavelengths silicon detectors do not see.
|
|
|
|
The Sun is often approximated as a 5772 K blackbody: a smooth curve with
|
|
the right total power. The real spectrum has thousands of absorption lines
|
|
from the Sun's outer layers, and its shape differs from the smooth curve.
|
|
For the example camera's channels, the blackbody gives 4.6 % (red), 7.5 %
|
|
(green) and 9.7 % (blue) fewer electrons than the measured spectrum. That
|
|
is far outside what absolute radiometry can tolerate.
|
|
|
|
### Folding spectra into per-channel constants
|
|
|
|
Code: `radiometry.rs` (`Radiometry`).
|
|
|
|
Doing all of this wavelength by wavelength for every pixel would be slow.
|
|
Most factors do not depend on the pixel: the solar spectrum, the optics,
|
|
and the QE curves. The renderer multiplies and integrates those over
|
|
wavelength once, when the camera is set up. That yields a few numbers per
|
|
channel, and the GPU only multiplies by them.
|
|
|
|
The central number is the solar flux of each channel k:
|
|
|
|
```text
|
|
S_k = integral of E_sun(lambda) * tau(lambda) * QE_k(lambda) * lambda / (h c) d lambda
|
|
```
|
|
|
|
It is the rate of electrons per square meter that sunlight at 1 AU would
|
|
produce if it fell straight onto the detector through the optics. For the
|
|
example camera, S_k is about 2.4e20 electrons per second per m^2 in red and
|
|
green, and 2.0e20 in blue.
|
|
|
|
The renderer then works in channel radiance: scene radiance weighted the
|
|
same way (transmission, QE, photons per joule), in electrons per second
|
|
per m^2 per steradian. A Lambertian surface with albedo rho, lit at angle i
|
|
from its normal, at distance d from the Sun in AU, has channel radiance:
|
|
|
|
```text
|
|
L_k = rho * cos(i) * S_k / (pi * d^2)
|
|
```
|
|
|
|
The solar disk has channel radiance S_k / Omega_sun, where Omega_sun is the
|
|
Sun's solid angle at 1 AU, 6.79e-5 sr (it is 0.53 degrees across). Seen
|
|
from anywhere, the Sun's surface has that same radiance, about 2.0e7
|
|
W/m^2/sr in total. Looked at directly, it puts about 5e8 electrons into a
|
|
pixel of the example camera's green channel in 0.1 ms, 26,000 times the
|
|
full well.
|
|
|
|
A surface whose reflectance varies with wavelength is reduced to an
|
|
effective albedo per channel. That is the albedo of a flat-spectrum surface
|
|
that would give the same signal under sunlight
|
|
(`Radiometry::effective_albedo`). This step is where the error of the
|
|
shortcut comes in. Two spectra that both vary inside one channel's band
|
|
interact in ways a single number per channel cannot capture. One example is
|
|
a surface whose reflectance changes steeply across green, lit partly by
|
|
sunlight and partly by bluish light scattered from Earth's atmosphere.
|
|
Rendering more, narrower channels reduces that error, and the code keeps
|
|
the channel count generic for that reason.
|
|
|
|
## From radiance to electrons: the camera equation
|
|
|
|
Code: `radiometry.rs` (`Radiometry::electrons`); `gpu/sensor.rs`, `shaders/sensor.wgsl`.
|
|
|
|
A lens with focal length f and aperture diameter D has f-number
|
|
N = f / D. The example camera is f/4: a 50 mm lens with a 12.5 mm aperture.
|
|
For a scene radiance L, the irradiance on the focal plane at the image
|
|
center is:
|
|
|
|
```text
|
|
E_image = pi * L * tau / (4 N^2)
|
|
```
|
|
|
|
The aperture collects light over a solid angle proportional to (D / f)^2 =
|
|
1 / N^2. So opening the aperture one "stop", which divides N by the square
|
|
root of 2, doubles the light. Image irradiance is independent of distance
|
|
for the same reason radiance is.
|
|
|
|
Away from the center, image irradiance falls off as cos^4(theta), where
|
|
theta is the angle from the boresight. Two factors come from the longer
|
|
path to the off-axis pixel. One comes from the aperture appearing
|
|
foreshortened. One comes from light reaching the pixel at a slant. At the
|
|
corner of the example camera, theta is 6.9 degrees, and the falloff is
|
|
3 %. Real lenses add more falloff of their own, called vignetting.
|
|
|
|
The electrons a pixel collects in one channel are then:
|
|
|
|
```text
|
|
electrons = E_image * (pixel area) * (exposure time) * (photons per joule) * QE
|
|
```
|
|
|
|
The last two factors are wavelength-dependent. In channel radiance they are
|
|
already folded in, so the camera equation reduces to
|
|
|
|
```text
|
|
electrons = L_k * pi * (pixel area) * (exposure time) / (4 N^2) * cos^4(theta)
|
|
```
|
|
|
|
For the example camera, pi * A * t / (4 N^2) is 1.48e-16 m^2 sr s.
|
|
|
|
## From electrons to DN: the detector
|
|
|
|
Code: `camera.rs` (`Detector::dn`); `gpu/sensor.rs`, `shaders/sensor.wgsl`.
|
|
|
|
- **Full well.** The maximum number of electrons a pixel holds. Beyond it
|
|
the pixel saturates and reads its maximum value whatever the light.
|
|
Brighter scenes do not look brighter. In real sensors, excess charge can
|
|
also spill into neighboring pixels (blooming, M3).
|
|
- **Gain.** Electrons per DN. With a gain of 5, every 5 electrons add 1 to
|
|
the output.
|
|
- **Bias.** A fixed offset added to every pixel, so that noise around zero
|
|
signal does not get clipped at 0. The example uses 64 DN.
|
|
- **ADC bits.** The output resolution. 12 bits gives values 0 to 4095.
|
|
|
|
So DN = bias + min(electrons, full well) / gain, rounded and clipped to the
|
|
ADC range. The renderer does this on the GPU for every pixel and channel,
|
|
as the last step of a frame, and reads back only the resulting integers.
|
|
Noise (M3) will enter between the electrons and the gain.
|
|
|
|
Two consequences of these numbers show up in every image:
|
|
|
|
- **Saturation is not always the ADC's maximum.** In the example camera the
|
|
full well fills first: 20,000 / 5 + 64 = 4064 DN, below the 12-bit maximum
|
|
of 4095. A saturated pixel there reads 4064. Real cameras are designed
|
|
either way, and an algorithm that detects saturation by looking for 4095
|
|
would miss it in this one.
|
|
- **Black is not zero.** A pixel that receives no light reads the bias,
|
|
64 DN, so empty space in a raw frame displays as dark gray.
|
|
|
|
Raw frames are also not white-balanced. The camera reports what each
|
|
channel collected, and the channels collect different amounts from the same
|
|
gray surface. In the example camera, sunlight on a gray surface gives about
|
|
15 % fewer electrons in blue than in red, so gray things look slightly
|
|
yellow-green in a raw image. A white balance step, applied after capture,
|
|
would scale the channels to make them equal. The renderer leaves it out,
|
|
because it simulates what the sensor records.
|
|
|
|
### Worked example
|
|
|
|
Take a white Lambertian surface facing the Sun at 1 AU, seen by the example
|
|
camera's green channel. Integrating the solar spectrum through the green QE
|
|
curve, f/4 optics, 5.5 um pixels and a 0.1 ms exposure gives 11,139
|
|
electrons. Red gets 11,408 and blue 9,626. (A test in `radiometry.rs`
|
|
checks the green figure.)
|
|
|
|
- DN = 64 + 11,139 / 5, about 2292, a little over half the 12-bit range.
|
|
- The full well is 20,000 electrons, so this surface is at 56 % of
|
|
saturation.
|
|
- Earth, with albedo near 0.3, reads about 64 + 3,342 / 5, about 732 DN.
|
|
- The same white surface at the image corner, 6.9 degrees off axis, gets
|
|
cos^4 of that, 97 %: 10,819 electrons, 2228 DN.
|
|
|
|
The sensor tests check the 2292, 2228, 732, 4064 and 64 DN figures
|
|
exactly, on the GPU and on lavapipe.
|
|
|
|
Picking the exposure time is the same trade a photographer makes. A longer
|
|
exposure saturates the sunlit target. A shorter one leaves the side of the
|
|
target lit only by Earthshine lost in noise once noise is modeled (M3).
|
|
|
|
## How shapes become pixels
|
|
|
|
Two techniques turn 3D shapes into an image. The renderer uses one for the
|
|
bodies and the other for the target.
|
|
|
|
- **Mesh.** A surface approximated by many small flat triangles that share
|
|
corners (vertices). A cube needs 12 triangles. A curved surface needs
|
|
many more, and it is still faceted up close. Each vertex stores a position
|
|
and usually a normal, the direction the surface faces there. STL and glTF
|
|
files are meshes, and so is the target spacecraft.
|
|
- **Rasterization.** The GPU's built-in way of drawing a mesh. Each
|
|
triangle's corners are projected into the image with the pinhole
|
|
equations above. The GPU then finds every pixel whose center falls inside
|
|
the projected triangle, and runs a small program (a fragment shader) for
|
|
each of those pixels to compute its value. It is fast because it only
|
|
touches the pixels each triangle covers, and GPUs have dedicated hardware
|
|
for it. It can only draw triangles.
|
|
- **Ray casting.** The reverse direction: for each pixel, follow the line
|
|
of sight from the camera through the pixel center and solve for where it
|
|
first meets a surface. For a sphere or an ellipsoid that is a quadratic
|
|
equation with an exact answer, so there are no triangles and no facets.
|
|
Every pixel does the work, whether or not it ends up seeing the body.
|
|
|
|
The bodies use ray casting, because their shapes have exact equations and
|
|
their edges are what limb-based navigation measures. The target uses
|
|
rasterization, because a spacecraft model has no equation, only its mesh.
|
|
Both write into the same depth image, so they sort against each other
|
|
correctly. Ray casting a mesh is also possible, by testing rays against
|
|
its triangles. That is what M5's ray tracer does, because following rays
|
|
gives exact shadows and light bouncing between surfaces. Rasterization can
|
|
only approximate those, for example with the shadow maps lighting L1 uses
|
|
(below).
|
|
|
|
### One sample per pixel
|
|
|
|
Both techniques decide each pixel's value from one point, the pixel's
|
|
center. A real pixel collects light over its whole area, about 5.5 um
|
|
square on the detector. Where a pixel straddles an edge, say half on a
|
|
sunlit panel and half on dark space, the real pixel reads about half the
|
|
panel's value. The renderer reads either the full value or none, depending
|
|
on which side the center falls.
|
|
|
|
This shows in two ways. Edges are stair-stepped instead of graded. And
|
|
anything thinner than a pixel, like the booms of the example satellite seen
|
|
from 40 m, breaks into dashes, because only some pixel centers land on it.
|
|
For an algorithm that finds edges or measures brightness at sub-pixel
|
|
precision, the graded edge is the signal, so this matters for limb
|
|
navigation and pose estimation.
|
|
|
|
The usual fix is supersampling: shading several points inside each pixel
|
|
and averaging. The GPU's multisampling hardware does this cheaply for
|
|
rasterized edges. Sampling the bodies' limbs needs it in the ray casting
|
|
shader. The lens blur that comes with the point spread function (M3) also
|
|
spreads every edge over a few pixels.
|
|
|
|
## Earth, Moon and Sun
|
|
|
|
Code: `bodies.rs`, `gpu/bodies.rs`, `shaders/bodies.wgsl`.
|
|
|
|
The three bodies are not drawn as triangle meshes; they are ray cast. For
|
|
every pixel, the shader sends a ray from the camera through the pixel's
|
|
center and solves for where it meets each body's exact shape. The nearest hit wins. This
|
|
gives an exact edge at any zoom, at the cost of a few arithmetic operations
|
|
per pixel. A mesh would show its facets along the limb, the edge of the
|
|
disk, which is the feature horizon-based navigation measures.
|
|
|
|
- **Earth** is the WGS84 ellipsoid: 6378.137 km equatorial radius,
|
|
6356.752 km polar. The 21 km difference matters at fine pixel scales. A
|
|
camera with 5.5 urad pixels at 200,000 km sees the north limb 19 pixels
|
|
away from where a spherical Earth would put it. A test checks the limb
|
|
against the exact tangent to the ellipsoid to within a pixel.
|
|
- **The Moon** is a sphere of 1737.4 km radius. From Earth's distance it
|
|
is about 0.52 degrees across, 82 pixels in the example camera.
|
|
- **The Sun** is a sphere of 695,700 km radius: 0.53 degrees across, about
|
|
85 pixels in the example camera. Its disk has the solar radiance from the
|
|
folding section above, and it is drawn only where no nearer body covers
|
|
it. So Earth eclipses it naturally.
|
|
|
|
### Shading
|
|
|
|
Earth and the Moon are Lambertian, each with one reflectance spectrum over
|
|
the whole body, set by `Appearance`. The Moon's is a flat 0.12. Earth's
|
|
default is a stand-in shaped like cloud-free ocean seen from space: about
|
|
0.3 in the blue falling to 0.07 in the red, then back up in the near
|
|
infrared. Most of that blue is sunlight scattered by the atmosphere above
|
|
the ocean, the same scattering that makes the sky blue; the ocean itself is
|
|
dark. In the example camera it gives channel albedos of 0.09, 0.15 and 0.26
|
|
(red, green, blue), so Earth renders blue. Its overall brightness is below
|
|
the real Earth's average of about 0.3, which includes bright clouds.
|
|
|
|
Each point's radiance is rho cos(i) S_k / (pi d^2), with i the angle between
|
|
the surface normal and the Sun. The night side has i past 90 degrees and is
|
|
black. The Sun's distance d is taken from each body's center, since the
|
|
difference across a body is a few parts in 10^5.
|
|
|
|
The total light a sphere sends a far-away camera has a closed form. A
|
|
Lambertian sphere of albedo rho and radius R, at distance D, delivers an
|
|
irradiance of (2/3) rho E (R/D)^2 Phi(alpha), where alpha is the phase angle
|
|
(Sun-sphere-camera) and Phi(alpha) = (sin alpha + (pi - alpha) cos alpha) / pi.
|
|
It falls from 1 at full phase to 0.61 at 60 degrees and 0.11 at 120 degrees.
|
|
Up close it is not exact: the near side of the sphere is nearer the camera
|
|
than its center, so it counts for more. At 40,000 km from the Moon (R/D =
|
|
0.043), the real total is 3 % above the formula. The renderer's image agrees
|
|
with the exact surface integral to 0.03 % or better (`tests/analytic.rs`).
|
|
|
|
This gets the overall brightness and the phase (how much of the disk is
|
|
lit) right. It misses some things that are visible in real images:
|
|
|
|
- Earth's oceans, land and clouds differ in brightness by a factor of
|
|
several. From lighting L1, Earth's surface comes from an image (next
|
|
section); clouds come in M4.
|
|
- The full Moon looks almost uniformly bright out to its edge. A Lambertian
|
|
sphere instead darkens toward the edge, because the light there arrives
|
|
and leaves at grazing angles. Lunar soil scatters light back toward its
|
|
source much more strongly than Lambert predicts, and a dedicated lunar
|
|
reflectance model (M4) fixes that. The Moon's maria (the dark plains of
|
|
old basalt lava, the "seas" seen with the naked eye) and its bright
|
|
highlands also differ in albedo by about a factor of two (M2 albedo
|
|
map).
|
|
- Earth's atmosphere softens the limb and adds a thin bright band around it
|
|
(M4). Here Earth's edge is a sharp line.
|
|
|
|
### Earth's surface
|
|
|
|
Code: `earth.rs`, `shaders/bodies.wgsl`; data from `scripts/fetch-earth.sh`.
|
|
|
|
At lighting tier L1 (`RendererOptions::fidelity`, with the image in
|
|
`RendererOptions::earth_texture`; `--lighting L1` on the command line), Earth's
|
|
reflectance comes from an image of the whole surface instead of one spectrum. The image is NASA's
|
|
Blue Marble Next Generation: a cloud-free mosaic of the 2004 MODIS satellite
|
|
record, one image per month, so snow cover and vegetation follow the season.
|
|
The CLI picks the month from the scene's epoch.
|
|
|
|
The image is equirectangular: each column is a longitude and each row a
|
|
latitude. For every pixel that sees Earth, the shader finds the surface
|
|
point's geodetic latitude and longitude from the ellipsoid's normal there, in
|
|
Earth-fixed axes, and reads the image at that place. Earth's orientation from
|
|
the scene turns the continents with the epoch.
|
|
|
|
The full image is 500 m per pixel at the equator, resized to 660 m to fit the
|
|
largest texture lavapipe allows (16384 x 8192). Whether that is fine or coarse
|
|
depends on the view:
|
|
|
|
- From low orbit looking straight down, a camera pixel of the example camera
|
|
covers 77 m of ground, so each texture pixel spans about 8 image pixels and
|
|
looks soft.
|
|
- From geostationary orbit, a camera pixel covers about 4 km, so each image
|
|
pixel averages about 35 texture pixels (6 x 6).
|
|
|
|
For the second case the texture is stored with mipmaps: copies at half, a
|
|
quarter, an eighth of the size, and so on. The GPU reads from the copy whose
|
|
pixels best match the image pixel, so a far view shows the true average color
|
|
of a region rather than a shimmering sample of individual texture pixels.
|
|
|
|
Two details matter for brightness:
|
|
|
|
- Averaging has to happen on reflectance, not on the encoded values image files
|
|
store (sRGB, which compresses the darks). Averaging encoded black and white
|
|
gives a reflectance of 21 %, not 50 %. So the texture is decoded to linear
|
|
reflectance once at load and kept in 16-bit floating point, and both the
|
|
mipmaps and the GPU's blending between neighboring texture pixels work on
|
|
those values. A coastline seen up close blends correctly from sea to land.
|
|
- Blue Marble is surface reflectance with no atmosphere, and open ocean is
|
|
dark, a few percent. From orbit the ocean looks bright blue because the air
|
|
above it scatters sunlight toward the camera: Rayleigh scattering, which
|
|
grows as 1 / wavelength^4, which is also why the sky is blue. Until the
|
|
atmosphere is modeled (M4), a stand-in adds a reflectance of that shape,
|
|
0.07 at 450 nm falling to 0.017 at 650 nm (`Appearance::earth_haze`). It
|
|
brightens and blues the whole lit disk evenly, where the real effect grows
|
|
toward the limb, where the line of sight crosses more air.
|
|
|
|
The texture's red, green and blue are reflectances in the same three bands a
|
|
glTF base color uses (see the target model section), folded into the camera's
|
|
channels once at setup as a small matrix, so any channel layout works.
|
|
|
|
Blue Marble is a visual product built from calibrated MODIS reflectances, not a
|
|
calibrated reflectance product itself, so its values are close but not exact
|
|
reflectances. A calibrated alternative (MODIS's albedo product) is a possible
|
|
later refinement.
|
|
|
|
### Depth from the bodies
|
|
|
|
The pass also writes each hit's depth, near / z, as described in the
|
|
geometry section. The target spacecraft, drawn after the bodies, uses the
|
|
same depth buffer. So a target in front of Earth covers it, and a target
|
|
behind Earth's limb is hidden.
|
|
|
|
## The target model and its materials
|
|
|
|
Code: `model.rs`.
|
|
|
|
The target spacecraft is a triangle mesh loaded from STL or glTF. Each
|
|
piece (an STL file, or a glTF primitive) is a part with one material, and
|
|
for now a material is a Lambertian albedo spectrum, as for Earth and the
|
|
Moon.
|
|
|
|
Surface normals decide how bright each triangle is, through the cos(i)
|
|
factor. STL files store a normal per triangle, but exporters often write
|
|
zeros or wrong values, so the loader ignores them. It recomputes each
|
|
normal from the order of the triangle's corners: counterclockwise as seen
|
|
from outside. Every triangle is then lit as a flat facet, which matches the
|
|
flat panels and boxes most spacecraft are built from. glTF files that store
|
|
normals keep them.
|
|
|
|
glTF describes a material's color as red, green and blue reflectances (its
|
|
"base color"). Three numbers do not define a spectrum: very different
|
|
spectra can look the same color to the eye. The loader turns them into a
|
|
stepped spectrum, blue below 490 nm, green from 500 to 580 nm, and red above
|
|
590 nm. That is a reasonable guess for painted surfaces seen by an RGB
|
|
camera. It is a poor one for a camera whose channels differ from the eye's,
|
|
or for anything in the near infrared, where the red value simply continues.
|
|
For absolute radiometry of a real spacecraft, give the important materials
|
|
measured reflectance spectra (M4 adds those, and BRDFs).
|
|
|
|
Texture images are not used yet; each material renders with its base color
|
|
value alone. The loader removes textures before reading a glTF file, so a
|
|
model still loads when its textures are in a format or file the loader cannot
|
|
read. A model that relies on textures for its colors (one material with a
|
|
painted image) renders in that material's flat base color, often gray.
|
|
|
|
STL has no units. CAD tools usually export millimeters, so loading a CAD
|
|
model usually needs `scale: 0.001`. glTF is meters by its specification, but
|
|
models in the wild do not always follow it: of the NASA example models,
|
|
Landsat 8 and ICESat-2 are in meters, while the SSL-1300 needs a scale of 0.1
|
|
and ACE about 0.005. Checking a model's size against the real spacecraft's is
|
|
worth the minute it takes.
|
|
|
|
## Drawing the target
|
|
|
|
Code: `gpu/mesh.rs`, `shaders/mesh.wgsl`.
|
|
|
|
The target's mesh is rasterized after the bodies, into the same radiance
|
|
and depth images. Each pixel the target covers is kept only if the target
|
|
is nearer than what is already there. That one rule handles a target in
|
|
front of Earth, a target behind Earth's limb, and parts of the target
|
|
hiding other parts.
|
|
|
|
Each point on the target is shaded like the bodies: albedo times cos(i)
|
|
times the sunlight that reaches it. A panel modeled with no thickness, such
|
|
as a solar array, has one surface but two sides. Whichever side faces the
|
|
camera is the side that is lit. A closed shape never shows its inside, so
|
|
this only matters for thin parts.
|
|
|
|
### Shadows on the target
|
|
|
|
Code: `shadow.rs`, `gpu/shadow.rs`, `shaders/mesh.wgsl`. From lighting L1.
|
|
|
|
A solar array shades the bus behind it, an antenna dish shades its feed,
|
|
and a second spacecraft close by can shade the first. Rasterization draws
|
|
each triangle on its own, so by itself it cannot know that something else
|
|
stands between a point and the Sun. The renderer finds out with a shadow
|
|
map.
|
|
|
|
- **Shadow map.** Before the camera's view is drawn, the target is drawn
|
|
once more as the Sun sees it, looking straight down the Sun's rays. That
|
|
drawing keeps only depth: for each texel (a pixel of the map), how far
|
|
along the Sun's direction the first surface is. The Sun is far enough
|
|
away that its rays are parallel across a spacecraft, so the view has no
|
|
perspective, and a map 4096 texels across covers the target's bounding
|
|
sphere. For Landsat 8, whose sphere is 16.6 m across, one texel is 4 mm.
|
|
- **Lookup.** When the camera's view shades a point, it finds that point
|
|
in the map. If the map holds a surface nearer the Sun than the point, the
|
|
point is in shadow.
|
|
|
|
Each target gets its own map. Another target is drawn into that map when it
|
|
is sunward of this one and close enough across the Sun's rays to shade it.
|
|
A target off to the side is left out, since it cannot cast a shadow here.
|
|
|
|
The Sun is a disk half a degree across, so a shadow edge is not sharp. A
|
|
point near the edge sees part of the disk past the blocker, the same
|
|
penumbra as in an eclipse. The width of the penumbra is the distance from
|
|
blocker to surface times the tangent of the Sun's angular diameter, 0.93 %
|
|
at 1 AU. A solar array 2 m above the bus casts an edge 19 mm wide. One 10 m
|
|
away casts an edge 9 cm wide. The renderer reproduces this with
|
|
percentage-closer soft shadows:
|
|
|
|
1. Search the map around the point for blockers, and average their depth.
|
|
That gives the blocker-to-surface distance, and from it the penumbra
|
|
width.
|
|
2. Compare the point's depth against a 6 x 6 grid of map texels spread over
|
|
that width, and take the fraction that are not blocked as the share of
|
|
the Sun the point sees.
|
|
|
|
Both steps use fixed grids rather than random samples, so the image has no
|
|
noise and the same scene always gives the same pixels.
|
|
|
|
A surface can wrongly shadow itself, because the map only stores its depth
|
|
once per texel. Where the surface is tilted to the Sun, depth changes
|
|
across a texel, and half of each texel looks like it is behind itself. The
|
|
result is a pattern of dark stripes called shadow acne. The renderer
|
|
prevents it three ways. The stored depth is pushed away from the Sun in
|
|
proportion to the surface's tilt. The lookup is made from a point 1.5
|
|
texels off the surface, toward the Sun. And a blocker has to be more than 2
|
|
texels nearer the Sun to count.
|
|
|
|
Tests (`tests/shadows.rs`) check, on the GPU and lavapipe:
|
|
|
|
- A plate under a 2 m square: the shadow reads zero direct light, and its
|
|
edge falls where the square's edge projects, to within one texel.
|
|
- The measured penumbra width is within 2.2 % of 0.93 % of the occluder's
|
|
height, at heights of 1, 2 and 3 m.
|
|
- No pixel darkens compared with L0 on a plate tilted 80 and 87 degrees
|
|
from the Sun, or on a faceted sphere.
|
|
- A target sunward of another shades it; one to the side does not.
|
|
|
|
On the RTX 3060, shadows cost 0.14 ms of GPU time per frame for Landsat 8
|
|
(85,000 triangles) and 0.34 ms for ICESat-2 (752,000), plus about 0.15 ms
|
|
in the camera view's lookups. Maps 2048 texels across save about 0.15 ms.
|
|
|
|
Limitations:
|
|
|
|
- **Penumbra shape.** The grid averages a square of the map, so brightness
|
|
rises in a straight line across the penumbra. A real disk-shaped Sun
|
|
rises along an S-shaped curve: the full width is the same, but the middle
|
|
80 % of the change is 0.69 of the width rather than 0.8.
|
|
- **Resolution.** A detail smaller than a texel casts no reliable shadow. A
|
|
gap narrower than about 2 texels between a blocker and the surface it
|
|
shades (a few mm on Landsat 8) lets light through, because of the acne
|
|
margins above.
|
|
- **One sample per pixel.** Shadows are sampled at the pixel center, like
|
|
everything else, until M3's pixel-area sampling.
|
|
- **Bodies.** Earth and the Moon do not use shadow maps. Their shadows on
|
|
the target are the eclipse factor below, applied to the whole target.
|
|
Light from Earth and the Moon (below) is not shadowed at all.
|
|
|
|
### Eclipses
|
|
|
|
Code: `eclipse.rs`.
|
|
|
|
The Sun is a disk, not a point. So as a spacecraft passes into Earth's
|
|
shadow, the light fades over a few seconds instead of switching off:
|
|
|
|
- **Umbra.** Earth covers the whole solar disk. No direct sunlight.
|
|
- **Penumbra.** Earth covers part of the disk. 7000 km from Earth's
|
|
center, on the night side, the penumbra is about 65 km wide; a satellite
|
|
crosses it in under 10 seconds.
|
|
- **Annular.** A body small enough to sit inside the solar disk blocks a
|
|
ring-shaped bite. From orbit this happens with the Moon.
|
|
|
|
The renderer finds the visible fraction of the solar disk from the overlap
|
|
of two circles on the sky. The circles are the Sun's disk and the
|
|
occulter's, each with its apparent angular radius. It multiplies the
|
|
target's sunlight by that fraction. The model is the standard one for
|
|
satellites (Montenbruck and Gill), the same as astrokit's. A test checks it
|
|
against tracing lines from the target to thousands of points spread over
|
|
the solar disk.
|
|
|
|
The renderer counts both Earth and the Moon as occulters. The fraction is
|
|
computed once per target, at its center. The whole target
|
|
dims together, which is accurate unless the target is a sizable fraction of
|
|
the penumbra's width. Light reaching the target from Earth itself is the
|
|
next section.
|
|
|
|
### Light from Earth and the Moon
|
|
|
|
Code: `gpu/earthshine.rs`, `shaders/earthshine.wgsl`, `shaders/mesh.wgsl`.
|
|
From lighting L1.
|
|
|
|
Earth reflects about a third of the sunlight that reaches it, and some of
|
|
that lands on the target: Earthshine. From low orbit it is large. At 500 km,
|
|
Earth fills a circle 68 degrees in radius around the point straight down,
|
|
3.9 of the sky's 12.6 steradians. A plate facing straight down over the
|
|
sunlit Earth receives up to about a quarter as much light from Earth as a
|
|
plate facing the Sun receives from the Sun. A white Earth would give 0.85
|
|
(the test below measures that case). At geostationary distance, Earth is 17
|
|
degrees across and a plate facing a full Earth gets about 0.5 % of sunlight.
|
|
The Moon does the same, Moonshine, much more weakly: a full Moon seen from
|
|
near Earth gives about a millionth of sunlight. From 100 km above the Moon,
|
|
though, a plate facing it gets about a tenth of sunlight.
|
|
|
|
This matters most where the Sun does not reach: the target's night side,
|
|
and parts in a shadow of its own. Without it those read as black, the bias in
|
|
DN; with it they show their structure, dimly.
|
|
|
|
The renderer works out, for each target and each frame, how much light
|
|
arrives from Earth and the Moon on a surface facing any direction:
|
|
|
|
1. **What the target sees of each body.** The circle on the sky that the body
|
|
fills is cut into 1024 cells of equal solid angle: 32 rings from the
|
|
center out to the edge, 32 cells around each. A line from the target
|
|
through each cell's center finds the point it meets on the body, and that
|
|
point is shaded exactly as the camera's view of the body is shaded:
|
|
reflectance from the Blue Marble image (or the stand-in spectrum), lit by
|
|
the cosine of the Sun's angle there, dark on the night side. Each texture
|
|
read averages the patch of ground the cell covers, about 1000 square
|
|
kilometers straight down from 500 km and more toward Earth's edge, so a
|
|
cell sees the mean color of that patch rather than one point. A line
|
|
that meets the other body first counts nothing, so the Moon can hide part
|
|
of Earth and Earth part of the Moon.
|
|
2. **Light for every direction a surface can face.** For a surface facing
|
|
direction n, the light it receives is the sum over all cells of the cell's
|
|
brightness, times its solid angle, times the cosine between n and the
|
|
cell's direction (zero for cells behind the surface). The renderer adds
|
|
this up for a grid of directions, a cube map 16 texels across on each of
|
|
its six faces.
|
|
|
|
Drawing the target, each point reads that cube at its surface normal and
|
|
adds albedo / pi times the result to the direct sunlight. The cube is computed
|
|
at the center of each target's bounding sphere and used for the whole target;
|
|
across a spacecraft the view of Earth barely changes.
|
|
|
|
Earthshine is not dimmed by the target's eclipse factor, because the two
|
|
are different views of Earth. From anywhere in Earth's umbra only the night
|
|
side is in view, so Earthshine is zero there without being told. In the
|
|
penumbra the Sun is partly hidden but a crescent of lit ground is still in
|
|
view, and that light still arrives.
|
|
|
|
Tests (`tests/earthshine.rs`) compare a plate's brightness on the GPU and
|
|
lavapipe against the same sum done in 64-bit floating point with 90 times as
|
|
many cells: over the subsolar point at 500 km facing down, sideways and up;
|
|
toward the terminator; in Earth's shadow; at GEO; 100 km above the Moon at
|
|
full and quarter phase; with each body hiding the other; with the plate seen
|
|
from several sides, so each cube face a camera-facing surface reads is checked. Every
|
|
case where the bodies send more than a thousandth of sunlight agrees to
|
|
within 0.5 %. The largest differences come from the cube: between texels
|
|
5.6 degrees apart its interpolation rounds off the peak of a surface facing
|
|
a full Earth or Moon, by 0.4 %. A cube 32 texels across cuts that to 0.13 %
|
|
but costs four times as much, and the cells' own error on a half-lit distant
|
|
Earth (0.4 %) stays. A thin crescent of lit ground at Earth's edge, seen from
|
|
low orbit near the terminator, is off by 4 % of a small amount, 7e-5 of
|
|
sunlight, about 0.1 DN on a gray surface in the example camera. Another test
|
|
checks the sum itself against the classical formula for a distant sphere,
|
|
(2/3) albedo (R/D)^2 Phi(alpha), to within 0.3 %.
|
|
|
|
On the RTX 3060 this costs 0.04 ms of GPU time per target (0.28 ms for
|
|
eight), and 1.4 ms per target on lavapipe.
|
|
|
|
Limitations:
|
|
|
|
- **No shadows.** The target's own parts do not block light from Earth or
|
|
the Moon. A panel facing Earth does not shade the bus behind it from
|
|
Earthshine, and the inside of a dish facing Earth is lit as if open to the
|
|
whole Earth. Earth fills too much of the sky for one shadow map to stand in
|
|
for it; the ray tracer (M5) handles this.
|
|
- **One spectrum per channel.** Light from Earth is bluer than sunlight, but
|
|
the target's albedo in each channel was worked out under sunlight. This is
|
|
the shortcut described in "Folding spectra into per-channel constants",
|
|
and its error is largest for strongly colored surfaces.
|
|
- **Lambertian bodies.** Earth and the Moon reflect as Lambertian surfaces
|
|
here, as they do in the camera's view. The Moon's real reflection is
|
|
brighter toward the Sun (M4), and Earth has no clouds or atmosphere yet
|
|
beyond the stand-in haze.
|
|
- **No eclipses on the bodies.** Earth's shadow on the Moon and the Moon's
|
|
shadow on Earth are not modeled yet (M2), so a Moon in eclipse still lights
|
|
the target.
|
|
|
|
## Where the Sun and Moon appear
|
|
|
|
Code: `ephemeris.rs` (feature `astrokit`).
|
|
|
|
With the `astrokit` feature, the Sun's and Moon's positions come from JPL's
|
|
DE440s ephemeris, and Earth's orientation from the ITRF93 kernel, which
|
|
includes the slow wobble of Earth's pole. These are geometric positions:
|
|
where each body is at the epoch.
|
|
|
|
A camera sees something slightly different, its apparent position:
|
|
|
|
- **Light time.** Light from the Moon takes 1.3 s to reach Earth, and from
|
|
the Sun 8.3 minutes. The camera sees where each body was when the light
|
|
left it. For the Moon that is about 1.3 km of motion, a few microradians.
|
|
- **Aberration.** A moving camera sees incoming light tilted toward its
|
|
direction of motion, as rain seems to slant toward a moving car. Earth's
|
|
orbital speed (30 km/s) tilts the Sun's apparent direction by about 20
|
|
arcseconds, 100 microradians. A spacecraft's own orbital velocity adds up
|
|
to about 5 arcseconds more.
|
|
|
|
100 microradians is about one pixel of the example camera, and 18 pixels of
|
|
the narrow 1 m focal length camera in the limb test. So the geometric
|
|
positions are close enough for M1, but not for precise navigation against
|
|
the Sun, Moon or stars. Rendering apparent positions needs the observer's
|
|
velocity as an input, which the scene does not carry yet.
|
|
|
|
## Not modeled yet
|
|
|
|
| Effect | Where |
|
|
|---|---|
|
|
| Eclipses seen on the bodies: Earth's shadow on the Moon, the Moon's on Earth | M2 |
|
|
| Earthshine on the Moon's night side | M4 |
|
|
| The Sun as a disk that is darker toward its edge (limb darkening) | M2 |
|
|
| Pixel-area sampling (anti-aliasing): graded edges, thin parts | M3 |
|
|
| Lens distortion | M3 |
|
|
| Blur from diffraction and lens imperfections (point spread function) | M3 |
|
|
| Noise: photon shot noise, read noise, dark current, pixel-to-pixel variation | M3 |
|
|
| Saturation spilling into neighbors (blooming) | M3 |
|
|
| Non-Lambertian materials (BRDFs), glints, lunar backscatter | M4 |
|
|
| Moon albedo map | M2 |
|
|
| Clouds on Earth | M4 |
|
|
| Earth's atmosphere: scattering, the bright limb | M4 |
|
|
| Motion blur, rolling shutter, stray light | M6 |
|
|
| Apparent positions: light time and aberration | M3 |
|