resolved/docs/optics.md

Optics and radiometry guide

This explains the physics the renderer models, for a reader who knows orbital mechanics but not optics. Each section says which code implements it. When a feature adds a new optical effect, it adds or updates a section here in the same change.

Numbers in the examples come from CameraConfig::example() unless stated otherwise: 1920 x 1080 pixels, 5.5 um pitch, 50 mm focal length, f/4, three color channels, 12-bit output.

The chain from Sun to number

Every pixel value comes from the same sequence:

1. The Sun emits power across a range of wavelengths.
2. Some of it lands on a surface (Earth, the Moon, the target), which reflects part of it in various directions.
3. The fraction reflected toward the camera enters the lens aperture. The optics focus it onto the detector, losing a little to absorption. Each point of the scene lands as a small spot rather than a point.
4. The pixel converts some of the arriving photons into electrons. The fraction depends on wavelength and on the color filter over that pixel.
5. The electrons pile up during the exposure, up to a limit.
6. The readout electronics turn the electron count into an integer, the digital number (DN).

The renderer follows this chain with physical units at every step. Image brightness therefore comes from the scene and the camera settings, not from a tuned scale factor. Change the exposure time, and the image changes exactly as a real camera's would.

Geometry: the pinhole camera

Code: camera.rs (Intrinsics), scene.rs.

A pinhole camera maps each direction in space to one point on the focal plane. Put the pinhole at the origin, looking along +Z. A point at camera coordinates (x, y, z) lands on the focal plane at (f x / z, f y / z), where f is the focal length. Dividing by the pixel pitch p converts focal-plane distance to pixels:

u = (f / p) * x / z + cx
v = (f / p) * y / z + cy

f / p is the focal length in pixels. (cx, cy) is the principal point, where the boresight hits the image, normally the image center. For the example camera, f / p = 50 mm / 5.5 um = 9091 pixels.

Some quantities that follow from that:

• IFOV, the instantaneous field of view. The angle one pixel covers: p / f = 110 microradians. At 10 m that is 1.1 mm per pixel; at 7000 km it is 770 m per pixel. This sets the smallest detail the camera can resolve.
• FOV, the field of view. The angle the whole image covers: 2 atan(width / 2 / (f / p)), about 12.1 x 6.8 degrees here. A longer focal length or smaller sensor gives a narrower field of view and finer IFOV.

Real lenses are not perfect pinholes. They bend straight lines slightly (distortion, next) and spread each point into a small blur (the point spread function, below). The pinhole model is the starting point those effects are added to.

Lens distortion

Code: camera.rs (BrownConrady, Intrinsics); gpu/warp.rs, gpu/blur.rs, shaders/blur.wgsl, shaders/distortion.wgsl.

A real lens moves each point a little away from where the pinhole puts it, by an amount that grows toward the edge of the image. Straight lines in the scene come out curved. When points move outward more the farther out they are, the corners stretch (pincushion distortion); when they move inward, the image bulges (barrel distortion). A lens element slightly off center adds a smaller sideways shift (tangential distortion).

The standard model, Brown-Conrady, works on the pinhole's normalized coordinates x = X / Z and y = Y / Z, with r^2 = x^2 + y^2:

x_d = x (1 + k1 r^2 + k2 r^4 + k3 r^6) + 2 p1 x y + p2 (r^2 + 2 x^2)
y_d = y (1 + k1 r^2 + k2 r^4 + k3 r^6) + p1 (r^2 + 2 y^2) + 2 p2 x y
u = (f / p) x_d + cx,  v = (f / p) y_d + cy

k1, k2 and k3 are the radial terms, positive for pincushion and negative for barrel; p1 and p2 are the tangential terms. This is OpenCV's convention, so the coefficients a camera calibration produces apply directly (OpenCV lists them in the order k1, k2, p1, p2, k3). For the example camera, k1 = 0.1 alone moves the corners 1.6 px outward.

That matters for navigation, not just for looks. A limb fit, a star identification or a target's centroid all assume where each pixel looks; an unmodeled 1.6 px is a 0.2 mrad pointing error at the corner, which at 36,000 km is 6 km. Intrinsics::project and Intrinsics::ray include the distortion (the inverse by Newton's method, to 1e-10 px), so navigation code and the renderer share one model; the projection matches OpenCV's to 1e-9 px.

Rendering it, from camera tier C1 (C0 is always a pinhole): the GPU draws the target's triangles with a linear projection, which keeps straight edges straight, and moving their corners alone would leave long edges straight when they should curve. So the renderer draws the whole frame without distortion, the bodies and targets sorted together as usual, on a slightly larger and offset area that covers everything the lens bends into view. Then it resamples it: for each pixel of the focal plane it finds the undistorted point the lens bends onto that pixel and reads the render there, interpolating between the four nearest render pixels. The blur's first pass (below) does this as it reads its input, so the distorted image is never written out. Radiance is the same at both ends of a ray, so moving it is all distortion does to an extended scene. The resampling works on the finer image C1 renders (below), before the blur, which is where the lens puts it in a real camera.

Two effects of distortion fall on the detector instead. The cos^4 falloff uses each pixel's real line of sight, the undistorted one. And where the lens stretches the image, the same light covers more pixels: each pixel's signal is divided by how much the distortion stretches area there (the determinant of its derivative), 0.6 % at the corners for k1 = 0.1.

Measured with coefficients of 0.3, -0.2, 0.1 (radial) and 0.001, -0.0008 (tangential), which move the corners about 4 px: a small Moon's light lands centered where Intrinsics::project puts it to within 0.004 px across the frame, and a long straight edge follows the predicted curve to within 0.07 px in every row and 0.02 px averaged over rows. The interpolation adds a little blur where it falls between render pixels, at most a variance of 1/16 px^2 at 2 x 2 rendering (computed), which takes sigma 0.5 px to 0.56 px at worst.

Distortion costs 0.25 to 0.32 ms per 1920 x 1080 frame on an RTX 3060 at 2 x 2 rendering, and 1.1 to 1.3 ms at 3840 x 2160, almost all of it the blur's first pass reading through it, and under 0.1 ms the larger undistorted render. A separate pass writing the distorted image out and the blur reading it back cost 0.7 to 0.8 ms and 2.8 to 3.3 ms: at those sizes both are limited by how fast the card moves the image through memory. On lavapipe distortion adds 10 to 11 ms to 1920 x 1080 frames.

Coordinate conventions

The camera frame has +Z along the boresight, +X toward increasing column u, and +Y toward increasing row v. This is the computer-vision convention: row 0 is the top of the image, so +Y points down in the picture. Pixel (i, j) covers u in [i, i+1) and v in [j, j+1), so its center is at (i + 0.5, j + 0.5).

Why the GPU sees camera-relative coordinates

GPUs compute in 32-bit floats, which carry about 7 significant digits. A spacecraft 7000 km from Earth's center, stored in Earth-centered f32 coordinates, can only be placed to the nearest 0.5 m. A target 10 m away would jitter by several pixels. So all positions stay in 64-bit floats on the CPU. They are made relative to the camera there, and converted to 32-bit only after that. A target 10 m from the camera is then placed to about a micrometer.

Depth

When two things overlap in the image, the nearer one must win. The GPU keeps a depth value per pixel for this. The renderer stores near / z, which is 1 at the near clip distance and falls toward 0 with distance. Combined with how floating-point numbers are spaced, this gives roughly the same relative precision at every distance. That matters in a scene that contains a 10 m target, Earth at thousands of kilometers, and the Sun at 150 million kilometers. The scheme is called reversed-Z with an infinite far plane.

Radiometric quantities

Code: spectrum.rs, radiometry.rs.

Light has a few distinct measures, and the camera measures a specific one.

• Irradiance, E, in W/m^2. Power arriving per unit area of a surface. Sunlight at 1 AU delivers about 1361 W/m^2 to a surface facing the Sun. It falls off with the square of the distance from the Sun. A surface tilted by angle theta from the Sun direction receives E cos(theta).
• Solid angle, in steradians (sr). The 2D version of an angle: the area a cone cuts out of a unit sphere. The full sphere is 4 pi sr. One pixel of the example camera covers (110 urad)^2, about 1.2e-8 sr.
• Radiance, L, in W/m^2/sr. Power per unit area per unit solid angle, traveling in a particular direction. It is what a camera pixel responds to: how bright a surface looks from a given direction.

Radiance has a property that explains a lot of what a camera sees: it does not change along a ray through empty space. Move a camera twice as far from a wall, and each pixel sees four times as much wall area. But each square meter now contributes a quarter of the power, so the pixel value is unchanged. An object that covers many pixels has the same surface brightness at any distance; it just covers fewer pixels farther away.

That stops being true once the object is smaller than a pixel. The pixel then also sees dark space around the object, so the signal falls as one over distance squared. That is the regime of a distant satellite seen as a point (space situational awareness), as opposed to the resolved imagery this renderer is mostly for.

Reflection: albedo and Lambertian surfaces

A Lambertian surface reflects light equally in every direction. Matte paint, the Moon (roughly) and cloud tops behave close to this. If such a surface receives irradiance E and reflects a fraction rho (its albedo), its radiance in every direction is:

L = rho * E / pi

The pi comes from integrating the reflected radiance over the hemisphere above the surface. A white Lambertian sheet facing the Sun at 1 AU, with rho = 1, has a radiance of about 1361 / pi = 433 W/m^2/sr, summed over all wavelengths. Earth's average albedo is about 0.3.

Real spacecraft surfaces are not Lambertian. Multilayer insulation (MLI) and solar cells are partly mirror-like and produce bright glints at particular angles. These are modeled by a BRDF, a function giving reflected radiance for every pair of incoming and outgoing directions. BRDFs are not modeled yet.

Wavelength: spectra, channels and photons

Code: spectrum.rs (Spectrum), camera.rs (Channel).

Everything above varies with wavelength. Sunlight peaks around 500 nm and extends from the ultraviolet into the infrared. Surfaces reflect different wavelengths differently; that is what color is. Optics transmit some wavelengths better than others. The sensor's sensitivity also depends on wavelength.

The renderer describes each of these as a Spectrum: a value as a function of wavelength, in meters.

• Quantum efficiency (QE). The fraction of arriving photons that become electrons in a pixel. A color camera puts red, green and blue filters over different pixels, so each channel has its own QE curve. The example camera uses idealized Gaussian curves centered at 620, 540 and 460 nm, 80 nm wide at half height, peaking at 60 %.
• Transmission (tau). The fraction of light the lenses pass, about 0.9 for the example camera.

Detectors count photons, not energy. A photon of wavelength lambda carries energy h c / lambda (h is Planck's constant, c the speed of light). So one watt of red light delivers more photons than one watt of blue. The renderer converts power to photons per wavelength before applying QE.

The solar spectrum

Code: radiometry.rs (solar_spectrum), generated into solar_data.rs by scripts/fetch-solar-spectrum.sh.

The renderer uses a measured solar spectrum, the TSIS-1 Hybrid Solar Reference Spectrum (Coddington et al. 2021). It is the current international reference, built mainly from the TSIS-1 instruments on the International Space Station and filled in with a high-resolution model of the solar absorption lines. It gives the irradiance at 1 AU, outside the atmosphere, every 1 nm from 202 to 2730 nm. Its uncertainty is 0.3 % from 460 to 2365 nm. Over that range it sums to 1325.8 W/m^2. The other 35 W/m^2 of the 1361 W/m^2 total is at wavelengths silicon detectors do not see.

The Sun is often approximated as a 5772 K blackbody: a smooth curve with the right total power. The real spectrum has thousands of absorption lines from the Sun's outer layers, and its shape differs from the smooth curve. For the example camera's channels, the blackbody gives 4.6 % (red), 7.5 % (green) and 9.7 % (blue) fewer electrons than the measured spectrum. That is far outside what absolute radiometry can tolerate.

Folding spectra into per-channel constants

Code: radiometry.rs (Radiometry).

Doing all of this wavelength by wavelength for every pixel would be slow. Most factors do not depend on the pixel: the solar spectrum, the optics, and the QE curves. The renderer multiplies and integrates those over wavelength once, when the camera is set up. That yields a few numbers per channel, and the GPU only multiplies by them.

The central number is the solar flux of each channel k:

S_k = integral of E_sun(lambda) * tau(lambda) * QE_k(lambda) * lambda / (h c) d lambda

It is the rate of electrons per square meter that sunlight at 1 AU would produce if it fell straight onto the detector through the optics. For the example camera, S_k is about 2.4e20 electrons per second per m^2 in red and green, and 2.0e20 in blue.

The renderer then works in channel radiance: scene radiance weighted the same way (transmission, QE, photons per joule), in electrons per second per m^2 per steradian. A Lambertian surface with albedo rho, lit at angle i from its normal, at distance d from the Sun in AU, has channel radiance:

L_k = rho * cos(i) * S_k / (pi * d^2)

The solar disk has channel radiance S_k / Omega_sun, where Omega_sun is the Sun's solid angle at 1 AU, 6.79e-5 sr (it is 0.53 degrees across). Seen from anywhere, the Sun's surface has that same radiance, about 2.0e7 W/m^2/sr in total. Looked at directly, it puts about 5e8 electrons into a pixel of the example camera's green channel in 0.1 ms, 26,000 times the full well.

A surface whose reflectance varies with wavelength is reduced to an effective albedo per channel. That is the albedo of a flat-spectrum surface that would give the same signal under sunlight (Radiometry::effective_albedo). This step is where the error of the shortcut comes in. Two spectra that both vary inside one channel's band interact in ways a single number per channel cannot capture. One example is a surface whose reflectance changes steeply across green, lit partly by sunlight and partly by bluish light scattered from Earth's atmosphere. Rendering more, narrower channels reduces that error, and the code keeps the channel count generic for that reason.

From radiance to electrons: the camera equation

Code: radiometry.rs (Radiometry::electrons); gpu/sensor.rs, shaders/sensor.wgsl.

A lens with focal length f and aperture diameter D has f-number N = f / D. The example camera is f/4: a 50 mm lens with a 12.5 mm aperture. For a scene radiance L, the irradiance on the focal plane at the image center is:

E_image = pi * L * tau / (4 N^2)

The aperture collects light over a solid angle proportional to (D / f)^2 = 1 / N^2. So opening the aperture one "stop", which divides N by the square root of 2, doubles the light. Image irradiance is independent of distance for the same reason radiance is.

Away from the center, image irradiance falls off as cos^4(theta), where theta is the angle from the boresight. Two factors come from the longer path to the off-axis pixel. One comes from the aperture appearing foreshortened. One comes from light reaching the pixel at a slant. At the corner of the example camera, theta is 6.9 degrees, and the falloff is 3 %. Real lenses add more falloff of their own, called vignetting.

The electrons a pixel collects in one channel are then:

electrons = E_image * (pixel area) * (exposure time) * (photons per joule) * QE

The last two factors are wavelength-dependent. In channel radiance they are already folded in, so the camera equation reduces to

electrons = L_k * pi * (pixel area) * (exposure time) / (4 N^2) * cos^4(theta)

For the example camera, pi * A * t / (4 N^2) is 1.48e-16 m^2 sr s.

From electrons to DN: the detector

Code: camera.rs (Detector::dn, Detector::variance); gpu/sensor.rs, shaders/sensor.wgsl.

• Full well. The maximum number of electrons a pixel holds. Beyond it the pixel saturates and reads its maximum value whatever the light. Brighter scenes do not look brighter. In real sensors, excess charge can also spill into neighboring pixels (blooming, below).
• Gain. Electrons per DN. With a gain of 5, every 5 electrons add 1 to the output.
• Bias. A fixed offset added to every pixel, so that noise around zero signal does not get clipped at 0. The example uses 64 DN.
• ADC bits. The output resolution. 12 bits gives values 0 to 4095.

So DN = bias + min(electrons, full well) / gain, rounded and clipped to the ADC range. The renderer does this on the GPU for every pixel and channel, as the last step of a frame, and reads back only the resulting integers. From camera tier C1, noise enters between the electrons and the gain (see Noise below).

Two consequences of these numbers show up in every image:

• Saturation is not always the ADC's maximum. In the example camera the full well fills first: 20,000 / 5 + 64 = 4064 DN, below the 12-bit maximum of 4095. A saturated pixel there reads 4064. Real cameras are designed either way, and an algorithm that detects saturation by looking for 4095 would miss it in this one.
• Black is not zero. A pixel that receives no light reads the bias, 64 DN, so empty space in a raw frame displays as dark gray.

Raw frames are also not white-balanced. The camera reports what each channel collected, and the channels collect different amounts from the same gray surface. In the example camera, sunlight on a gray surface gives about 15 % fewer electrons in blue than in red, so gray things look slightly yellow-green in a raw image. A white balance step, applied after capture, would scale the channels to make them equal. The renderer leaves it out, because it simulates what the sensor records.

Worked example

Take a white Lambertian surface facing the Sun at 1 AU, seen by the example camera's green channel. Integrating the solar spectrum through the green QE curve, f/4 optics, 5.5 um pixels and a 0.1 ms exposure gives 11,139 electrons. Red gets 11,408 and blue 9,626. (A test in radiometry.rs checks the green figure.)

• DN = 64 + 11,139 / 5, about 2292, a little over half the 12-bit range.
• The full well is 20,000 electrons, so this surface is at 56 % of saturation.
• Earth, with albedo near 0.3, reads about 64 + 3,342 / 5, about 732 DN.
• The same white surface at the image corner, 6.9 degrees off axis, gets cos^4 of that, 97 %: 10,819 electrons, 2228 DN.

The sensor tests check the 2292, 2228, 732, 4064 and 64 DN figures exactly, on the GPU and on lavapipe.

Picking the exposure time is the same trade a photographer makes. A longer exposure saturates the sunlit target. A shorter one leaves the side of the target lit only by Earthshine lost in noise.

Noise

Code: Detector::variance; sensor.wgsl (NOISE).

A real image is noisy, and the noise decides what can be detected at all. From camera tier C1 the renderer adds three kinds, following the EMVA 1288 standard's model of a camera:

• Shot noise. Light arrives as photons at random times, so a pixel expecting N electrons collects a Poisson-distributed count with mean N and standard deviation sqrt(N). It is a property of light, not of the camera: a perfect detector still has it. A pixel expecting 10,000 electrons varies by 100 (1 %); one expecting 100 varies by 10 (10 %).
• Dark current. The detector makes electrons of its own from heat, Detector::dark_current per second, which add to the light's and carry shot noise of their own. It grows with temperature, roughly doubling every 6 to 8 degrees C; the value given is for the detector's operating temperature.
• Read noise. Reading a pixel out adds a few electrons of random error, Detector::read_noise RMS, to every pixel in every frame, light or not.

In order: the light's electrons and the dark electrons together are drawn as one Poisson count (a sum of Poisson counts is itself one); the count is clipped to the full well; read noise is added; then the gain, bias and rounding to DN as before. Below a mean of 50 electrons the count is drawn exactly. Above it a Gaussian of the same mean and variance takes its place, which a Poisson count's shape is already close to and which costs the same at any level. With read noise the DN can fall below the bias, which is what the bias is for.

Each pixel's variance in electrons^2 is then

variance = (electrons + dark current x exposure) + read noise^2 + gain^2 / 12

where the last term is the rounding to whole DN. Divided by gain^2 it is in DN^2. Plotted against the mean signal in DN, the variance is a straight line with slope 1 / gain and intercept (read noise^2 + dark shot noise) / gain^2: the photon transfer curve, the standard way to measure a camera's gain and read noise from flat fields alone.

For the example camera (5 electrons read noise, 20 electrons per second dark current, gain 5) at 0.1 ms, the signal-to-noise ratio, signal over the square root of the variance, is:

What	Electrons	SNR
A white surface facing the Sun, green channel	11,139	105
Earth, albedo 0.3	3,342	58
A white surface lit by 1 % of sunlight, as Earthshine can	111	9.5
A V = 0 star, over the 4 pixels it covers	56	4.4
A V = 4 star, over 4 pixels	1.4	0.13

So at the exposures a sunlit target needs, the brightest stars are just detectable and the rest are lost in read noise. At 50 ms, a star tracker's exposure, a V = 6 star reaches an SNR of 7.5.

Dark current hardly matters at these exposures: 20 electrons per second is 0.002 electrons in 0.1 ms. It matters for long ones: 1 electron in 50 ms, 100 in 5 s, with 10 electrons of shot noise.

Reproducible randomness. The noise comes from a counter-based generator: a hash (Jarzynski and Olano's pcg4d) of the pixel and channel, the frame number and a 64-bit seed (RendererOptions::noise_seed), turned into uniform numbers, then into Gaussians (Box-Muller) or a Poisson count (by inverting its distribution). Nothing is carried from one pixel or frame to the next, so every value is computed independently on the GPU. The renderer numbers its frames from 0 (Frame::number), and Renderer::set_frame_number renders a frame again with exactly the same noise. Different frame numbers or seeds give independent noise. The hash is identical on every device, but the logarithms and cosines that shape it may differ in the last bit, so the GPU and lavapipe agree statistically rather than pixel for pixel.

Noise-free images. A detector with no read noise, no dark current, shot noise off and no fixed pattern (Detector::noiseless) renders C1 without noise, exactly as before noise existed, for ground truth and ML labels. The renderer then compiles the noise out entirely. At C0 there is never noise.

Measured, identically on the GPU and lavapipe: flat fields at six levels from 100 to 16,000 electrons, two frames each, give half the variance of their difference within 2 % of the formula (0.982 to 1.008 of it, against a sampling error of 0.6 %), and the photon transfer curve's slope is 1 / gain to 0.1 %. A dark frame of 100 electrons reads 20.005 DN above the bias, with its variance within 0.6 %. Counts below 50 match the Poisson probabilities count by count. Consecutive frames, different seeds and neighboring pixels are uncorrelated (|r| at most 0.005 over 196,608 values). 400 faint stars of 300 electrons each vary as predicted to 0.4 %, an SNR of 7.35. Noise costs nothing measurable on an RTX 3060 (0.476 ms for the sensor pass at 1920 x 1080 with or without it, 1.89 ms at 3840 x 2160); on lavapipe it adds 1.5 ms at 1920 x 1080 and 9.6 ms at 3840 x 2160.

Fixed pattern: PRNU and DSNU

Code: fixed_pattern.rs (FixedPattern); sensor.wgsl (PATTERN).

No two pixels of a real detector are quite alike, and the differences are the same in every frame. That sets them apart from the noise above, which is new each frame:

• PRNU (photo-response nonuniformity). Each pixel converts light to electrons a little more or less efficiently than average, typically by 0.5 to 1.5 % RMS. Under even light the image is mottled by that much.
• DSNU (dark signal nonuniformity). Each pixel's dark current differs from the average, by tens of percent. A few pixels, from defects in the silicon, make tens to hundreds of times the average: hot pixels.

From camera tier C1 each value (pixel and channel) has a response factor, which multiplies its light's electrons (stars' included), and a dark factor, which multiplies the dark current, before the shot noise and the full well. The example camera has 1 % PRNU, 10 % DSNU, and 1 value in 10,000 hot at 50 times the dark current (Detector::prnu, dsnu, hot_pixel_fraction, hot_pixel_multiple). Because the dark factor scales the dark current, the dark pattern grows with the exposure, as a real one does: at 0.1 ms the example camera's hot pixels make 0.1 electrons, invisible; over 5 s, 5,000.

The pattern comes from Detector::pattern_seed, apart from the noise's seed, so one camera keeps its pattern whatever the frame or noise: the same hash as the noise, of the seed and the value's index, gives a Gaussian response and dark factor and decides whether the value is hot. The shader computes it for each value as it goes, so it takes no memory and no setup. FixedPattern::generate computes the same pattern on the CPU, for tests or to correct images with. A pattern measured from a real camera can be used instead (FixedPattern::from_calibration, RendererOptions::fixed_pattern): a flat, a frame of even light, and a dark, a frame with none, each the average of many so their own noise is small, bias removed; each is divided by its mean.

This is why real images are flat-fielded and dark-subtracted before use: divided by a flat and with a dark frame taken off, which removes the pattern and leaves the noise. It matters for detection in particular. A hot pixel in a long exposure is a bright, sharp point that stays put from frame to frame, just like a faint star or a distant target; only its single-pixel sharpness (a real point source is spread by the PSF) or a dark frame gives it away.

Measured, on the GPU and lavapipe: with noise off, a flat field and dark frames at two exposures read the CPU's pattern to the rounding (within 0.505 DN of the unrounded value), the hot pixels are the same ones, and the spreads are the requested 1 % and 10 % to 2 %. With noise on, at 10,000 electrons the pattern and the noise are equal parts of the spread, and two frames or two noise seeds correlate by the predicted 0.50, while another pattern seed shares nothing. Calibration maps are applied to the rounding. The pattern costs nothing measurable on an RTX 3060; on lavapipe it adds about 2.3 ms at 1920 x 1080 and 6.3 ms at 3840 x 2160. A loaded pattern takes 8 bytes per value, 199 MB at 3840 x 2160 in RGB.

Blooming

Code: Blooming; sensor.wgsl (BLOOMING: collect_charge, bloom_columns, read_out).

A saturated pixel's extra charge does not vanish. Where it goes depends on the sensor, and matters whenever something in or near the frame is far past the full well: the Sun at the example camera's 0.1 ms is about 26,000 full wells at the center of its disk, and a sunlit specular part can be hundreds.

• CCDs move charge down columns to read it out, and the channels that hold it are only walled off from each other across the columns. Excess charge spills along the column, up and down, filling each pixel it reaches before passing on: the familiar vertical streak through the Sun or a glint, which can cross a whole frame and hide a target in it.
• CMOS sensors read each pixel on its own, and most have an anti-blooming drain that carries excess charge away. What escapes it spills into the pixels around.

From camera tier C1, Detector::blooming chooses: None (the default: the pixel clips and the excess is lost), Column { efficiency } or Neighbors { efficiency }, where the efficiency is the fraction of the excess that spreads and the rest drains. Along a column, half goes up and half down; each pixel it reaches takes what it has room for and passes the rest on, until it runs out or leaves the frame. Into the neighbors, each of the 8 takes an eighth, and what overfills them drains.

Blooming happens to the charge, so it comes after the shot noise (and the fixed pattern) and before the read noise and the ADC. With blooming on, the sensor pass works in three steps rather than one: it writes each value's charge to a buffer, spreads it, and reads it out. Along columns the spread is a walk up and down each column, one at a time; the first step marks the columns that hold any excess, and the walk skips the rest, which in most frames is all of them. The CPU reference in the tests does the same walk in f64.

Measured, on the GPU and lavapipe: a single overfull pixel on an even background fills the pixels it should along its column to the electron, charge that leaves the frame is lost, and nothing else changes; the neighbors share equally; below the full well, or at efficiency 0, blooming changes no value. The Sun at 2e-5 efficiency, in a 256 x 256 frame of the example optics, streaks 22 pixels each way from the middle of its disk, matching the CPU column by column to 0 DN. Column blooming costs 0.23 ms per 1920 x 1080 frame on an RTX 3060 with nothing past the full well and 0.75 ms with the Sun in the frame (1.0 and 2.2 ms at 3840 x 2160); into the neighbors, 0.24 ms (1.0 ms). On lavapipe, about 3 ms and 10 to 12 ms at 1920 x 1080.

How shapes become pixels

Two techniques turn 3D shapes into an image. The renderer uses one for the bodies and the other for the target.

• Mesh. A surface approximated by many small flat triangles that share corners (vertices). A cube needs 12 triangles. A curved surface needs many more, and it is still faceted up close. Each vertex stores a position and usually a normal, the direction the surface faces there. STL and glTF files are meshes, and so is the target spacecraft.
• Rasterization. The GPU's built-in way of drawing a mesh. Each triangle's corners are projected into the image with the pinhole equations above. The GPU then finds every pixel whose center falls inside the projected triangle, and runs a small program (a fragment shader) for each of those pixels to compute its value. It is fast because it only touches the pixels each triangle covers, and GPUs have dedicated hardware for it. It can only draw triangles.
• Ray casting. The reverse direction: for each pixel, follow the line of sight from the camera through the pixel center and solve for where it first meets a surface. For a sphere or an ellipsoid that is a quadratic equation with an exact answer, so there are no triangles and no facets. Every pixel does the work, whether or not it ends up seeing the body.

The bodies use ray casting, because their shapes have exact equations and their edges are what limb-based navigation measures. The target uses rasterization, because a spacecraft model has no equation, only its mesh. The meshes are drawn first, with their depth. The bodies pass then casts its rays and compares depths at every pixel, so the two sort against each other correctly: a target in front of Earth covers it, and a target behind the limb is hidden. Ray casting a mesh is also possible, by testing rays against its triangles. A ray tracer, planned, does that, because following rays gives exact shadows and light bouncing between surfaces. Rasterization can only approximate those, for example with the shadow maps lighting L1 uses (below).

Pixel area

At camera tier C0, both techniques decide each pixel's value from one point, the pixel's center. A real pixel collects light over its whole area, about 5.5 um square on the detector. Where a pixel straddles an edge, say half on a sunlit panel and half on dark space, the real pixel reads about half the panel's value. At C0 the renderer reads either the full value or none, depending on which side the center falls.

This shows in two ways. Edges are stair-stepped instead of graded. And anything thinner than a pixel, like the booms of the example satellite seen from 40 m, breaks into dashes, because only some pixel centers land on it. A regular pattern finer than the pixels, like the cells of a solar array, beats against the pixel grid and shows as curved bands (moiré). For an algorithm that finds edges or measures brightness at sub-pixel precision, the graded edge is the signal, so this matters for limb navigation and pose estimation.

Camera tier C1 samples each pixel at several points and averages them (supersampling), in a way that costs little where nothing is crossing the pixel:

• Meshes use the GPU's multisampling. The rasterizer tests each triangle against 4 points in every pixel, and keeps a depth for each, but runs the fragment shader once per pixel. A pixel half covered by a panel then gets the panel's value in 2 of its 4 samples, and reads half. The covered fraction comes in steps of a quarter, and the light is computed at one point inside the covered part (the centroid of the covered samples), so a pixel on an edge is still shaded as one surface.
• Bodies are ray cast, so the bodies pass first decides whether a body's edge could cross the pixel: whether the line of sight through its center passes within one pixel's angle of the edge of Earth, the Moon or the Sun. Most pixels do not, and cast one ray as at C0. A pixel on an edge casts a grid of 4 x 4 rays spread over its area and averages them. A body only a few pixels across needs a finer grid, because its brightness falls off toward its edge and a coarse grid misjudges that: a 4 x 4 grid gets a full Moon 0.45 pixels wide 45 % wrong in total light. So the grid has at least 8 rays across the body's radius, up to 64 x 64 for one smaller than a pixel, which keeps the total within 1 %. Only a handful of pixels pay for that.
• Both together. Where a target and a body share a pixel, each of the grid's rays takes the mesh sample nearest to it, and keeps the target if that sample is nearer than the body along the ray. So a target seen just past Earth's limb takes its share of the pixel, and Earth keeps the rest.

A grid of 4 x 4 is still a coarse estimate of an area. For an edge nearly aligned with the pixel grid, its 4 columns of rays resolve the covered fraction in quarters, so a pixel on the limb can be off by up to an eighth of the difference across the edge (measured 0.10 on Earth's limb from low orbit, against up to a half at C0). Patterns that stagger the rays fix aligned edges but do worse at other angles; the regular grid had the smallest error on average over all angles. At C1 the point spread function (below) then spreads every edge over a few pixels, and C1 also renders each pixel as 2 x 2 finer ones by default, each sampled this way, which divides these errors further.

At one rendered pixel per image pixel, C1 costs 0.4 to 0.5 ms more per 1920 x 1080 frame on an RTX 3060, and 1.2 to 1.4 ms at 3840 x 2160, measured on Landsat 8, ICESat-2 and the moonrise scene. On the CPU through lavapipe it costs much more, 2.6 to 8 times the mesh pass at C0 (ICESat-2's 752,000 triangles go from 74 to 580 ms per frame), because lavapipe appears to run the fragment shader for every sample rather than every pixel. The 4 samples of radiance and depth also take 80 bytes per pixel of GPU memory: 166 MB at 1920 x 1080 and 660 MB at 3840 x 2160.

Blur: the point spread function

Code: camera.rs (Psf); gpu/blur.rs, shaders/blur.wgsl.

No camera focuses a point to a point. Light from one point of the scene lands on the detector as a small spot, and the shape of that spot is the point spread function (PSF). Several things widen it:

• Diffraction. Light passing through a round aperture spreads into a pattern of rings (the Airy pattern) whose size depends only on the wavelength and the f-number. At f/4 and 550 nm the first dark ring is 2.7 um from the center, half a pixel of the example camera.
• Aberrations and defocus. Real lenses bring different parts of the aperture, different wavelengths and different parts of the field to slightly different points.
• The detector. Electrons freed deep in a pixel can drift into its neighbors before they are collected (charge diffusion).
• Jitter. The camera shakes a little during the exposure.

Each of these blurs the image of the one before it, and many such blurs together come out close to a Gaussian, a bell curve in 2D. So a common model of a camera's PSF is a Gaussian with one width, its standard deviation sigma, measured in pixels. A spot with sigma 0.5 px is 1.2 px wide at half its peak, and a point centered on a pixel puts about 47 % of its light in that pixel and the rest in its neighbors; at 0.7 px, 28 %. Longer wavelengths diffract more, so sigma is set per channel and usually grows toward the red.

How wide the spot should be depends on the camera. Published measurements, converted to a Gaussian's sigma with the pixel's own area taken out:

• Earth-observation imagers are built sharp: about 0.2 px for Landsat 8's OLI, the sharpest, and 0.35 to 0.55 px for WorldView-3, Pléiades Neo and Sentinel-2's specification. The Lunar Reconnaissance Orbiter's cameras are specified in the same range.
• Navigation and small-satellite cameras are softer: about 0.6 to 0.9 px for DART's DRACO camera in flight, and 0.55 to 0.7 px for Satellogic's imagers.
• Star trackers are defocused so each star covers several pixels. The one measured in flight here (on Lunar Flashlight) has a sharp core and wide wings that no single Gaussian matches: its spot is as wide at half its peak as a 0.4 px Gaussian, and spreads its light as widely as a 1.25 px one.

The imagers' numbers are for the whole system, including the atmosphere and image processing, so the optics alone are somewhat sharper. The example camera uses 0.5 px in every channel, a sharp camera, not derived from its optics; diffraction alone at f/4 would give about 0.17 px.

Without blur, a star or a distant spacecraft lands in one pixel or in none, and every edge is as sharp as the pixel grid allows. Algorithms that find the center of a star or a target to a fraction of a pixel (centroiding) need the spot spread over more than one pixel, so they cannot be tested on an unblurred image.

The blur is applied to radiance on the focal plane, before light becomes electrons. That order matters for anything bright enough to saturate. The Sun's image is tens of thousands of times brighter than a full well. Blurred first, its spot spreads that light into the neighboring pixels, which also fill, so the saturated region is wider than the Sun's disk, as in a real camera. Clipped first and then blurred, the saturated region would shrink back inside the disk and its edge would fade over a pixel or two, as if the Sun were barely brighter than the full well. The blur's weights sum to 1, so it moves light without adding or removing any.

Blurring before the pixels average

The blur has to act on the light where it lands, before each pixel averages it. Blurring an image of pixel averages afterward is not the same once the spot is about as small as a pixel: a point of light that fell anywhere in a pixel has become that pixel's value, and where in the pixel it fell is gone. Its blurred spot should lean toward wherever it was, and no blur of the averages can tell which way. Measured with sigma 0.4 px, points of light blurred after averaging put up to 51 % of their light in the wrong pixel, and edges were off by up to 12 % of their step.

So from camera tier C1 the renderer draws the scene finer than the pixels, n x n rendered pixels to each image pixel, blurs that by a Gaussian PSF set in CameraConfig::psf, and only then averages each n x n block into one pixel. Each rendered pixel still collects light over its own area the way the previous section describes, so the finer image is itself made of small-pixel averages, and the same error remains at a smaller scale:

Rendered pixels per pixel	Points, worst pixel	Edges, worst pixel
1 x 1	51 % of the light	12 % of the step
2 x 2	19 %	2.2 %
4 x 4	4.5 %	0.5 %

(sigma 0.4 px, points at 16 and edges at 129 positions within a pixel; the worst edges lie on a pixel boundary.) C1 renders 2 x 2 by default; RendererOptions::supersampling sets 1 to 4. Point sources, stars in particular, need better than that, and will be drawn with the blurred spot computed exactly over each pixel instead.

A 2D Gaussian is the product of a 1D Gaussian across and one down, and averaging an n x n block is likewise an average across and one down. So the blur runs as two passes, one along rows and one along columns, each taking a weighted sum of the rendered pixels out to 4 sigma on either side (beyond which a Gaussian holds less than 1e-4 of its light) and averaging n of them into one pixel at once. Light from just outside the frame also blurs into it, so the renderer draws a margin of that width around the image, and the blur reads it and drops it. The weights are the Gaussian at each rendered pixel's offset, which keeps a spot's width (its variance) within 1 % of sigma squared once sigma is 0.65 rendered pixels or more: 0.33 px in image pixels at 2 x 2.

Rendering finer costs about n^2 times the drawing, and the blur grows with it. To keep that down, the meshes are covered at 2 points per rendered pixel rather than 4 when rendering finer: at 2 x 2 that still puts samples at 4 distinct offsets across and down each image pixel, as 4 samples did at 1 x 1, so mesh edges still grade in quarters or better. (A device that cannot render 2 samples, such as lavapipe, uses 4.) GPU time per frame at C1, from the first pass to the last, on an RTX 3060 (Landsat 8, ICESat-2 and the moonrise scene, at lighting L1, sigma 0.5 px):

Rendered pixels per pixel	1920 x 1080	3840 x 2160
1 x 1	1.9 to 2.5 ms	6.1 to 6.9 ms
2 x 2 (default)	3.7 to 4.4 ms	12.6 to 14.6 ms
3 x 3	6.7 to 7.9 ms	34 to 38 ms
4 x 4	10.9 to 13.0 ms	59 to 66 ms

Of the 3.7 to 4.4 ms at 2 x 2, the bodies take 1.3 to 1.4 ms, the meshes 0.7 to 1.1 ms and the blur 1.0 ms. With 2 samples instead of 4 the meshes and bodies took 0.3 to 0.6 ms less in all; with 1 sample, 0.7 to 1.0 ms less, but a mesh edge then grades only in halves at 2 x 2, coarser than at 1 x 1. The rendered images take 60 bytes per rendered pixel (100 at 1 x 1, with 4 samples), and the blur 16 bytes per image pixel per rendered row: about 0.27, 0.60, 1.25 and 2.15 GB at 1920 x 1080 for 1 x 1 to 4 x 4, and four times that at 3840 x 2160. The renderer reports running out of memory as an error. On the CPU through lavapipe, 2 x 2 adds 9 to 13 ms to 1920 x 1080 frames of 116 to 680 ms, and 3 x 3 at 3840 x 2160 runs out of memory.

Earth, Moon and Sun

Code: bodies.rs, gpu/bodies.rs, shaders/bodies.wgsl.

The three bodies are not drawn as triangle meshes; they are ray cast. For every pixel, the shader sends a ray from the camera through the pixel's center and solves for where it meets each body's exact shape. The nearest hit wins. This gives an exact edge at any zoom, at the cost of a few arithmetic operations per pixel. A mesh would show its facets along the limb, the edge of the disk, which is the feature horizon-based navigation measures.

• Earth is the WGS84 ellipsoid: 6378.137 km equatorial radius, 6356.752 km polar. The 21 km difference matters at fine pixel scales. A camera with 5.5 urad pixels at 200,000 km sees the north limb 19 pixels away from where a spherical Earth would put it. A test checks the limb against the exact tangent to the ellipsoid to within a pixel.
• The Moon is a sphere of 1737.4 km radius. From Earth's distance it is about 0.52 degrees across, 82 pixels in the example camera.
• The Sun is a sphere of 695,700 km radius: 0.53 degrees across, about 85 pixels in the example camera. Averaged over its disk it has the solar radiance from the folding section above; from lighting L1 its center is brighter than its edge (see "The Sun's disk" below). It is drawn only where no nearer body covers it, so Earth eclipses it naturally.

Shading

Earth and the Moon are Lambertian, at lighting L0 each with one reflectance spectrum over the whole body, set by Appearance. The Moon's is a flat 0.12 (from L1 it comes from a map; see "The Moon's surface"). Earth's default is a stand-in shaped like cloud-free ocean seen from space: about 0.3 in the blue falling to 0.07 in the red, then back up in the near infrared. Most of that blue is sunlight scattered by the atmosphere above the ocean, the same scattering that makes the sky blue; the ocean itself is dark. In the example camera it gives channel albedos of 0.09, 0.15 and 0.26 (red, green, blue), so Earth renders blue. Its overall brightness is below the real Earth's average of about 0.3, which includes bright clouds.

Each point's radiance is rho cos(i) S_k / (pi d^2), with i the angle between the surface normal and the Sun. The night side has i past 90 degrees and is black. The Sun's distance d is taken from each body's center, since the difference across a body is a few parts in 10^5.

The total light a sphere sends a far-away camera has a closed form. A Lambertian sphere of albedo rho and radius R, at distance D, delivers an irradiance of (2/3) rho E (R/D)^2 Phi(alpha), where alpha is the phase angle (Sun-sphere-camera) and Phi(alpha) = (sin alpha + (pi - alpha) cos alpha) / pi. It falls from 1 at full phase to 0.61 at 60 degrees and 0.11 at 120 degrees. Up close it is not exact: the near side of the sphere is nearer the camera than its center, so it counts for more. At 40,000 km from the Moon (R/D = 0.043), the real total is 3 % above the formula. The renderer's image agrees with the exact surface integral to 0.03 % or better (tests/gpu/analytic.rs).

This gets the overall brightness and the phase (how much of the disk is lit) right. It misses some things that are visible in real images:

• Earth's oceans, land and clouds differ in brightness by a factor of several. From lighting L1, Earth's surface comes from an image (next section); clouds are not modeled yet.
• The full Moon looks almost uniformly bright out to its edge. A Lambertian sphere instead darkens toward the edge, because the light there arrives and leaves at grazing angles. Lunar soil scatters light back toward its source much more strongly than Lambert predicts, which is also why it brightens so steeply toward full Moon. At lighting L1 the Moon's total brightness is scaled to its measured phase curve ("The Moon's surface"), but its edge still darkens; a dedicated lunar reflectance model would fix that. The Moon's maria (the dark plains of old basalt lava, the "seas" seen with the naked eye) and its bright highlands also differ in albedo by about a factor of two, which from lighting L1 the Moon's map shows.
• Earth's atmosphere softens the limb and adds a thin bright band around it (not modeled yet). Here Earth's edge is a sharp line.

Earth's surface

Code: earth.rs, shaders/bodies.wgsl; data from scripts/fetch-earth.sh.

At lighting tier L1 (RendererOptions::fidelity, with the image in RendererOptions::earth_texture; --lighting L1 on the command line), Earth's reflectance comes from an image of the whole surface instead of one spectrum. The image is NASA's Blue Marble Next Generation: a cloud-free mosaic of the 2004 MODIS satellite record, one image per month, so snow cover and vegetation follow the season. The CLI picks the month from the scene's epoch.

The image is equirectangular: each column is a longitude and each row a latitude. For every pixel that sees Earth, the shader finds the surface point's geodetic latitude and longitude from the ellipsoid's normal there, in Earth-fixed axes, and reads the image at that place. Earth's orientation from the scene turns the continents with the epoch.

The full image is 500 m per pixel at the equator, resized to 660 m to fit the largest texture lavapipe allows (16384 x 8192). Whether that is fine or coarse depends on the view:

• From low orbit looking straight down, a camera pixel of the example camera covers 77 m of ground, so each texture pixel spans about 8 image pixels and looks soft.
• From geostationary orbit, a camera pixel covers about 4 km, so each image pixel averages about 35 texture pixels (6 x 6).

For the second case the texture is stored with mipmaps: copies at half, a quarter, an eighth of the size, and so on. The GPU reads from the copy whose pixels best match the image pixel, so a far view shows the true average color of a region rather than a shimmering sample of individual texture pixels.

Two details matter for brightness:

• Averaging has to happen on reflectance, not on the encoded values image files store (sRGB, which compresses the darks). Averaging encoded black and white gives a reflectance of 21 %, not 50 %. So the texture is decoded to linear reflectance once at load and kept in 16-bit floating point, and both the mipmaps and the GPU's blending between neighboring texture pixels work on those values. A coastline seen up close blends correctly from sea to land.
• Blue Marble is surface reflectance with no atmosphere, and open ocean is dark, a few percent. From orbit the ocean looks bright blue because the air above it scatters sunlight toward the camera: Rayleigh scattering, which grows as 1 / wavelength^4, which is also why the sky is blue. Until the atmosphere is modeled, a stand-in adds a reflectance of that shape, 0.07 at 450 nm falling to 0.017 at 650 nm (Appearance::earth_haze). It brightens and blues the whole lit disk evenly, where the real effect grows toward the limb, where the line of sight crosses more air.

The texture's red, green and blue are reflectances in the same three bands a glTF base color uses (see the target model section), folded into the camera's channels once at setup as a small matrix, so any channel layout works.

Blue Marble is a visual product built from calibrated MODIS reflectances, not a calibrated reflectance product itself, so its values are close but not exact reflectances. A calibrated alternative (MODIS's albedo product) is a possible later refinement.

The Moon's surface

Code: moon.rs, shaders/bodies.wgsl; data from scripts/fetch-moon.py. From lighting L1, with the map in RendererOptions::moon_texture.

At lighting L1 the Moon's reflectance comes from a map instead of one spectrum. The map is the LRO Camera's Wide Angle Camera mosaic at 643 nm (red), built from more than 100,000 images taken under every kind of lighting. The camera team used a model of how each spot on the Moon reflects light at different angles (Hapke's, fitted to the images; Sato et al. 2014) to work out what every spot would look like under one standard geometry: the Sun 60 degrees from overhead, seen straight down. The map's values are that reflectance, I/F, the ratio of the surface's brightness to that of a perfect white surface facing the Sun. Over 98 % of the Moon they lie between 0.019 in dark maria and 0.067 on the brightest highlands and crater rays, and they average 0.042.

The map is 400 m per pixel and covers 70 degrees south to 70 degrees north; the polar caps beyond are filled with the average, so they render as smooth gray (6 % of the surface, seen only near the limb from Earth). It is resampled to 16384 x 8192, 666 m per pixel, the largest texture lavapipe takes, and read exactly as Earth's image is: by latitude and longitude in the Moon's own axes, with mipmaps, and without a seam at 180 degrees.

The Moon's axes come from Scene::moon_attitude. The Moon keeps one face toward Earth, but rocks back and forth by up to 8 degrees (libration), so which features are near the limb changes from month to month. With an epoch, the renderer uses the IAU's model of the Moon's rotation; it puts the point facing Earth within 0.008 degrees (230 m) of JPL Horizons' high-precision frame, under a third of a map pixel. Without one, Scene::moon_facing_earth turns the Moon's prime meridian toward Earth and is off by the libration, up to about 10 degrees.

Two choices turn the map into something the renderer can shade:

• Brightness. The renderer shades the Moon as Lambertian, starting from the albedo that gives the map's reflectance at the map's geometry: I/F / cos(60 degrees), on average 0.084. On its own that makes the Moon far too dark at full: the real Moon is not Lambertian, and it brightens steeply toward full Moon, as rough soil hides its own shadows when lit from behind the viewer. A Lambert sphere with these albedos is about a third as bright as the real full Moon, and slightly too bright past quarter. So each point's albedo is multiplied by a phase factor, from the phase angle between the Sun and the viewer seen from that point. The factor is the whole Moon's measured brightness at that phase (Allen's lunar phase curve, 0.026 g + 4e-9 g^4 magnitudes below full, g in degrees) over a Lambert sphere's, scaled so that the map's full Moon has the measured geometric albedo (the full Moon's brightness against a white Lambertian disk of the same size), 0.121 in the V band. It is 2.8 at 4 degrees, 1.16 at 60, 0.89 at 90 and 0.75 at 120, and holds its 150 degree value beyond. At 60 degrees it would be 1 if the map and the measured curve agreed exactly; the 16 % between them is the Lambert shape's error at that phase together with the two calibrations'.

  Rendered whole in the example camera's green channel, the Moon is then within 5 % of the measured curve from 2 to 30 degrees of phase, and within 11 % out to 120 degrees. At the same phase, the side of the Moon holding Oceanus Procellarum, the largest of the maria, is darker than the other: 0.89 of the curve at 90 degrees against 1.06. The real Moon differs between waxing and waning in the same way, which the curve averages out. At 150 degrees it is 9-15 % bright.

  Across the disk the light still falls off as Lambert's does, darkening toward the edge, while the real full Moon looks nearly flat out to its limb. A lunar reflectance model would fix that.

• Color. The map is one band. Its color comes from the Moon's average spectrum across the WAC's seven bands, 321 to 689 nm, from the same mosaics averaged over the whole Moon by area: relative to 643 nm, 0.42 at 321 nm, 0.61 at 415 nm, 0.87 at 566 nm and 1.08 at 689 nm. The Moon is distinctly red, which is why it looks yellowish gray to the eye. Each pixel's 643 nm value scales that spectrum, which is folded into the camera's channels once at setup: in the example camera the Moon's albedo in red, green and blue is 0.96, 0.83 and 0.69 times its 643 nm value over cos 60. Beyond 689 nm the spectrum continues the slope from 643 to 689 nm to 900 nm (1.45) and holds there, roughly following real lunar soils, which flatten into an absorption by iron-bearing minerals near 1000 nm; below 321 nm it holds the 321 nm value. Real maria rich in titanium are up to about 10 % bluer than this, and highlands slightly redder; the seven-band mosaic could give each pixel its own color later. The real Moon also reddens slightly with phase, which the phase factor leaves out.

Light the Moon sheds on targets (Moonshine) reads the same map, with the phase factor for the angle between the Sun and the target.

Tests (tests/gpu/moon_texture.rs), with small synthetic maps, on the GPU and lavapipe: the pixel under the camera reads the texel at its latitude and longitude, and moves when the Moon's attitude does; the color is the spectrum's; a uniform map draws exactly the Moon a flat albedo draws times the phase factor, at every pixel; there is no seam at 180 degrees; a far view averages a fine checker; Moonshine comes from the part of the map that faces the target.

Cost: the map adds about 0.04 ms to the bodies pass on the RTX 3060, for working out where to read it at every pixel (the read itself happens only where the Moon is). On lavapipe the pass got faster, from 2.7 to 2.0 ms with the Moon filling the view, because Earth's image is now also read only where Earth is. Loading the map takes 0.5 s and uploading it 0.17 s; it occupies 360 MB on the GPU.

The Sun's disk

Code: sun.rs, shaders/common.wgsl, shaders/bodies.wgsl. From lighting L1.

The Sun's disk is brighter at its center than at its edge: limb darkening. Light leaving the Sun comes from a layer a few hundred kilometers deep in which the gas gets hotter with depth. Looking at the center of the disk, the line of sight goes straight down into hot gas; near the edge it runs through the layer at a slant and sees higher, cooler gas, which shines less.

Where the line of sight meets the Sun's surface, call mu the cosine of the angle between it and the surface's normal: 1 at the center of the disk, 0 at the edge, 0.5 at 87 % of the way out. Neckel and Labs (1994) measured the brightness across the disk at 30 wavelengths from 303 to 1099 nm, and Hestroffer and Magnan (1998) showed that at each wavelength it is close to mu^alpha times the center's, to about 1 %. Alpha is 0.94 at 303 nm, 0.48 at 580 nm and 0.26 at 1099 nm: blue light darkens much more toward the edge than red. The renderer uses their values of alpha, interpolated between those wavelengths and held constant outside them.

The total light from the disk does not change; that is the measured solar spectrum. Only its spread does. A disk of brightness mu^alpha averages 2 / (alpha + 2) of its center, so the center is (alpha + 2) / 2 times the mean. Folded through the example camera's channels, the center is 1.22, 1.26 and 1.31 times the mean in red, green and blue, and 87 % of the way to the edge (mu = 0.5) the disk is 0.70 of its center's brightness in green. The folding is exact: for each channel the renderer weights every wavelength's profile by that channel's solar flux and tabulates the result, so any channel layout works.

It matters in two places:

• The Sun in an image. A camera that looks at the Sun almost always saturates, but not with a dark filter, a Sun sensor, or once the lens's blur, once modeled, spreads the Sun's light into the pixels around it.
• Eclipses. When Earth or the Moon covers part of the Sun, what counts is how much light it hides, not how much of the disk's area. Near the umbra only a sliver of the dim edge is left, so less light gets through than the area says: 7000 km behind Earth, 25 km inside the penumbra's dark side, 4.5 % in green against 6.1 %. Near the outside of the penumbra the covered part is dim edge, so more gets through: 88.2 % against 86.0 %. In an annular solar eclipse the Moon covers the bright center and leaves the dim ring: 1 % of the light reaches the ground, not the 2.4 % of the disk's area.

The renderer computes the light hidden by an occulter in two parts. Circles around the Sun's center that the occulter covers completely count through a table of the light inside each radius. Circles it covers in part count by adding up, over eight radii, each circle's brightness times the length of its arc inside the occulter, with the radii placed by a Gauss-Legendre rule; eight radii are within 2.4e-5 of what 32 give. The same calculation runs on the CPU for each target's eclipse and on the GPU for every pixel of Earth and the Moon in an eclipse, per channel.

Tests: every pixel of the Sun seen through a 250 mm lens (420 pixels across) reads its mean radiance times the profile at its mu, to 6e-5, and the disk's light summed over pixels is the channel's solar flux to 7e-5. The light hidden by an occulter matches a fine sum over the disk to within 1e-4, and a uniform disk gives the area overlap. In eclipses the tests of the sections below use the limb-darkened light.

Cost: nothing measurable on the RTX 3060 unless an eclipse is in view. With Earth filling the image and most of it in the Moon's penumbra, the bodies pass takes 0.43 ms instead of 0.21 ms; lavapipe 9.5 ms instead of 5.0 ms.

Limitations:

• The very edge. Hestroffer and Magnan fitted the profiles over the inner 90 % of the radius, where measurements are reliable. Right at the edge mu^alpha falls to zero, faster than the measured Sun does, over the outermost fraction of a percent of the radius.
• Wavelengths outside 303 to 1099 nm use the end values. The example camera's channels are inside that range.
• A quiet Sun. Sunspots and bright faculae change the Sun's output by about 0.1 % over the solar cycle and are not modeled.
• Shadows on the target (above) still treat the Sun as a square of uniform brightness in their penumbra.

Depth from the bodies

The pass also writes each hit's depth, near / z, as described in the geometry section. The target spacecraft, drawn after the bodies, uses the same depth buffer. So a target in front of Earth covers it, and a target behind Earth's limb is hidden.

The target model and its materials

Code: model.rs.

The target spacecraft is a triangle mesh loaded from STL or glTF. Each piece (an STL file, or a glTF primitive) is a part with one material, and for now a material is a Lambertian albedo spectrum, as for Earth and the Moon.

Surface normals decide how bright each triangle is, through the cos(i) factor. STL files store a normal per triangle, but exporters often write zeros or wrong values, so the loader ignores them. It recomputes each normal from the order of the triangle's corners: counterclockwise as seen from outside. Every triangle is then lit as a flat facet, which matches the flat panels and boxes most spacecraft are built from. glTF files that store normals keep them.

glTF describes a material's color as red, green and blue reflectances (its "base color"). Three numbers do not define a spectrum: very different spectra can look the same color to the eye. The loader turns them into a stepped spectrum, blue below 490 nm, green from 500 to 580 nm, and red above 590 nm. That is a reasonable guess for painted surfaces seen by an RGB camera. It is a poor one for a camera whose channels differ from the eye's, or for anything in the near infrared, where the red value simply continues. For absolute radiometry of a real spacecraft, give the important materials measured reflectance spectra (not supported yet).

Texture images are not used yet; each material renders with its base color value alone. The loader removes textures before reading a glTF file, so a model still loads when its textures are in a format or file the loader cannot read. A model that relies on textures for its colors (one material with a painted image) renders in that material's flat base color, often gray.

STL has no units. CAD tools usually export millimeters, so loading a CAD model usually needs scale: 0.001. glTF is meters by its specification, but models in the wild do not always follow it: of the NASA example models, Landsat 8 and ICESat-2 are in meters, while the SSL-1300 needs a scale of 0.1 and ACE about 0.005. Checking a model's size against the real spacecraft's is worth the minute it takes.

Drawing the target

Code: gpu/mesh.rs, shaders/mesh.wgsl.

The target's mesh is rasterized after the bodies, into the same radiance and depth images. Each pixel the target covers is kept only if the target is nearer than what is already there. That one rule handles a target in front of Earth, a target behind Earth's limb, and parts of the target hiding other parts.

Each point on the target is shaded like the bodies: albedo times cos(i) times the sunlight that reaches it. A panel modeled with no thickness, such as a solar array, has one surface but two sides. Whichever side faces the camera is the side that is lit. A closed shape never shows its inside, so this only matters for thin parts.

Shadows on the target

Code: shadow.rs, gpu/shadow.rs, shaders/mesh.wgsl. From lighting L1.

A solar array shades the bus behind it, an antenna dish shades its feed, and a second spacecraft close by can shade the first. Rasterization draws each triangle on its own, so by itself it cannot know that something else stands between a point and the Sun. The renderer finds out with a shadow map.

• Shadow map. Before the camera's view is drawn, the target is drawn once more as the Sun sees it, looking straight down the Sun's rays. That drawing keeps only depth: for each texel (a pixel of the map), how far along the Sun's direction the first surface is. The Sun is far enough away that its rays are parallel across a spacecraft, so the view has no perspective, and a map 4096 texels across covers the target's bounding sphere. For Landsat 8, whose sphere is 16.6 m across, one texel is 4 mm.
• Lookup. When the camera's view shades a point, it finds that point in the map. If the map holds a surface nearer the Sun than the point, the point is in shadow.

Each target gets its own map. Another target is drawn into that map when it is sunward of this one and close enough across the Sun's rays to shade it. A target off to the side is left out, since it cannot cast a shadow here.

The Sun is a disk half a degree across, so a shadow edge is not sharp. A point near the edge sees part of the disk past the blocker, the same penumbra as in an eclipse. The width of the penumbra is the distance from blocker to surface times the tangent of the Sun's angular diameter, 0.93 % at 1 AU. A solar array 2 m above the bus casts an edge 19 mm wide. One 10 m away casts an edge 9 cm wide. The renderer reproduces this with percentage-closer soft shadows:

1. Search the map around the point for blockers, and average their depth. That gives the blocker-to-surface distance, and from it the penumbra width.
2. Compare the point's depth against a 6 x 6 grid of map texels spread over that width, and take the fraction that are not blocked as the share of the Sun the point sees.

Both steps use fixed grids rather than random samples, so the image has no noise and the same scene always gives the same pixels.

A surface can wrongly shadow itself, because the map only stores its depth once per texel. Where the surface is tilted to the Sun, depth changes across a texel, and half of each texel looks like it is behind itself. The result is a pattern of dark stripes called shadow acne. The renderer prevents it three ways. The stored depth is pushed away from the Sun in proportion to the surface's tilt. The lookup is made from a point 1.5 texels off the surface, toward the Sun. And a blocker has to be more than 2 texels nearer the Sun to count.

Tests (tests/gpu/shadows.rs) check, on the GPU and lavapipe:

• A plate under a 2 m square: the shadow reads zero direct light, and its edge falls where the square's edge projects, to within one texel.
• The measured penumbra width is within 2.2 % of 0.93 % of the occluder's height, at heights of 1, 2 and 3 m.
• No pixel darkens compared with L0 on a plate tilted 80 and 87 degrees from the Sun, or on a faceted sphere.
• A target sunward of another shades it; one to the side does not.

On the RTX 3060, shadows cost 0.14 ms of GPU time per frame for Landsat 8 (85,000 triangles) and 0.34 ms for ICESat-2 (752,000), plus about 0.15 ms in the camera view's lookups. Maps 2048 texels across save about 0.15 ms.

Limitations:

• Penumbra shape. The grid averages a square of the map, so brightness rises in a straight line across the penumbra. A real disk-shaped Sun rises along an S-shaped curve: the full width is the same, but the middle 80 % of the change is 0.69 of the width rather than 0.8. Limb darkening (from L1) would steepen it a little more, and is not modeled here either.
• Resolution. A detail smaller than a texel casts no reliable shadow. A gap narrower than about 2 texels between a blocker and the surface it shades (a few mm on Landsat 8) lets light through, because of the acne margins above.
• One shadow lookup per pixel. Shadows are looked up once per pixel, even at camera C1, so the edge of a shadow inside a pixel is not graded by its 4 samples; the penumbra's own width usually grades it.
• Bodies. Earth and the Moon do not use shadow maps. Their shadows on the target are the eclipse factor below, applied to the whole target. Light from Earth and the Moon (below) is not shadowed at all.

Eclipses

Code: eclipse.rs.

The Sun is a disk, not a point. So as a spacecraft passes into Earth's shadow, the light fades over a few seconds instead of switching off:

• Umbra. Earth covers the whole solar disk. No direct sunlight.
• Penumbra. Earth covers part of the disk. 7000 km from Earth's center, on the night side, the penumbra is about 65 km wide; a satellite crosses it in under 10 seconds.
• Annular. A body small enough to sit inside the solar disk blocks a ring-shaped bite. From orbit this happens with the Moon.

The renderer finds the visible fraction of the solar disk from the overlap of two circles on the sky. The circles are the Sun's disk and the occulter's, each with its apparent angular radius. It multiplies the target's sunlight by that fraction. The model is the standard one for satellites (Montenbruck and Gill), the same as astrokit's. A test checks it against tracing lines from the target to thousands of points spread over the solar disk.

From lighting L1 the fraction is of the Sun's light rather than its area, per channel, because the disk is brighter at its center (see "The Sun's disk"); that makes a penumbra darker toward the umbra and brighter toward the outside than the area alone says.

The renderer counts both Earth and the Moon as occulters. The fraction is computed once per target, at its center. The whole target dims together, which is accurate unless the target is a sizable fraction of the penumbra's width. Light reaching the target from Earth itself is the next section but one.

Eclipses on Earth and the Moon

Code: shaders/common.wgsl (sun_visible), shaders/bodies.wgsl. From lighting L1.

Earth and the Moon also shadow each other. When the Moon passes through Earth's shadow it is a lunar eclipse; when the Moon's shadow falls on Earth, a solar eclipse. For every pixel that sees Earth or the Moon, the shader works out the fraction of the Sun's light that reaches that point on the surface past the other body, with the same overlap of circles and the same limb-darkened disk as for the target, and multiplies the point's sunlight by it, per channel.

The sizes involved:

• At the Moon's distance, Earth's umbra reaches 4600 km from the shadow's axis and its penumbra 8200 km. The Moon, 3474 km across, fits inside the umbra with room to spare, so a total lunar eclipse lasts over an hour.
• The Moon's umbra only just reaches Earth. When the Moon is near its closest, the umbra touches down as a dark spot up to about 270 km across: a total solar eclipse. When it is farther, the umbra ends in space, and beyond its tip the Moon covers only the middle of the Sun's disk, leaving a ring: an annular eclipse, where about 1 % of the Sun's light still arrives, the dim edge of the disk. The penumbra around either is about 7000 km across.

The overlap is computed in 32-bit floats, which is enough: a lunar surface point is placed to about 20 m out of 384,000 km, 6e-8 radians against a solar radius of 4.6e-3 radians seen from there. One formula needed care. The area where the two circles overlap involves angles whose cosines are close to 1 when a sliver of the Sun is covered, and the inverse cosine of a number near 1 loses most of its precision in 32 bits. The shader finds those angles from their sine and cosine together instead. With the inverse cosine, pixels at the shadow's edges came out wrong: a point the 64-bit model puts fully in the umbra read slightly lit, and one just outside the shadow slightly dimmed.

Tests (tests/gpu/eclipses.rs), on the GPU and lavapipe, compare each pixel of Earth and the Moon against the 64-bit model at that pixel's surface point: the Moon entirely in Earth's umbra, the Moon across the umbra's edge, a total solar eclipse and an annular one. Pixels the model puts fully in shadow read zero, those it puts fully outside are unchanged (both to within 1e-6, the 32-bit rounding at the shadow's edges), and the rest agree to within 1e-4 of the Sun's light (1.2e-6 at most on the GPU, 5e-5 on lavapipe). The command-line tests render two real eclipses from their epochs: the total lunar eclipse of 2025 March 14 at mid-totality, where the whole Moon reads the bias, and the total solar eclipse of 2026 August 12, where the point of greatest eclipse in NASA's bulletin (65.2 N 25.2 W) is in the umbra (examples/scenes/lunar-eclipse.toml, solar-eclipse.toml). In the second, the umbra is 220 by 410 km in the image, stretched along the direction of the Sun, which is 26 degrees high there; NASA gives the path as 294 km wide.

The light Earth and the Moon reflect onto targets (next section) includes these shadows, so an eclipsed Moon sheds less light.

On the RTX 3060 this adds nothing measurable to the bodies pass unless an eclipse is in view; with Earth filling the image and most of it in the Moon's penumbra, the pass takes 0.43 ms instead of 0.20 ms, most of it the limb-darkened sum (see "The Sun's disk"). On lavapipe, 9.5 ms instead of 3.9 ms.

Limitations:

• Earth's atmosphere. It bends sunlight into the umbra, mostly red, so a real eclipsed Moon glows a dim copper color instead of going black, and it makes the umbra about 2 % larger. Neither is modeled yet.
• Smooth shapes. Earth's shadow is cast by a sphere of its equatorial radius, as for the target's eclipses; its flattening would move the shadow's edge on the Moon by about 20 km. The Moon's mountains and valleys, which shape the edge of a solar eclipse's path, are not modeled.

Light from Earth and the Moon

Code: gpu/earthshine.rs, shaders/earthshine.wgsl, shaders/mesh.wgsl. From lighting L1.

Earth reflects about a third of the sunlight that reaches it, and some of that lands on the target: Earthshine. From low orbit it is large. At 500 km, Earth fills a circle 68 degrees in radius around the point straight down, 3.9 of the sky's 12.6 steradians. A plate facing straight down over the sunlit Earth receives up to about a quarter as much light from Earth as a plate facing the Sun receives from the Sun. A white Earth would give 0.85 (the test below measures that case). At geostationary distance, Earth is 17 degrees across and a plate facing a full Earth gets about 0.5 % of sunlight. The Moon does the same, Moonshine, much more weakly: a full Moon seen from near Earth gives about a millionth of sunlight. From 100 km above the Moon, though, a plate facing it gets about a tenth of sunlight.

This matters most where the Sun does not reach: the target's night side, and parts in a shadow of its own. Without it those read as black, the bias in DN; with it they show their structure, dimly.

The renderer works out, for each target and each frame, how much light arrives from Earth and the Moon on a surface facing any direction:

1. What the target sees of each body. The circle on the sky that the body fills is cut into 1024 cells of equal solid angle: 32 rings from the center out to the edge, 32 cells around each. A line from the target through each cell's center finds the point it meets on the body, and that point is shaded exactly as the camera's view of the body is shaded: reflectance from the Blue Marble image (or the stand-in spectrum), lit by the cosine of the Sun's angle there, dark on the night side. Each texture read averages the patch of ground the cell covers, about 1000 square kilometers straight down from 500 km and more toward Earth's edge, so a cell sees the mean color of that patch rather than one point. A line that meets the other body first counts nothing, so the Moon can hide part of Earth and Earth part of the Moon.
2. Light for every direction a surface can face. For a surface facing direction n, the light it receives is the sum over all cells of the cell's brightness, times its solid angle, times the cosine between n and the cell's direction (zero for cells behind the surface). The renderer adds this up for a grid of directions, a cube map 16 texels across on each of its six faces.

Drawing the target, each point reads that cube at its surface normal and adds albedo / pi times the result to the direct sunlight. The cube is computed at the center of each target's bounding sphere and used for the whole target; across a spacecraft the view of Earth barely changes.

Earthshine is not dimmed by the target's eclipse factor, because the two are different views of Earth. From anywhere in Earth's umbra only the night side is in view, so Earthshine is zero there without being told. In the penumbra the Sun is partly hidden but a crescent of lit ground is still in view, and that light still arrives.

Tests (tests/gpu/earthshine.rs) compare a plate's brightness on the GPU and lavapipe against the same sum done in 64-bit floating point with 90 times as many cells: over the subsolar point at 500 km facing down, sideways and up; toward the terminator; in Earth's shadow; at GEO; 100 km above the Moon at full and quarter phase; with each body hiding the other; with the plate seen from several sides, so each cube face a camera-facing surface reads is checked. Every case where the bodies send more than a thousandth of sunlight agrees to within 0.5 %. The largest differences come from the cube: between texels 5.6 degrees apart its interpolation rounds off the peak of a surface facing a full Earth or Moon, by 0.4 %. A cube 32 texels across cuts that to 0.13 % but costs four times as much, and the cells' own error on a half-lit distant Earth (0.4 %) stays. A thin crescent of lit ground at Earth's edge, seen from low orbit near the terminator, is off by 4 % of a small amount, 7e-5 of sunlight, about 0.1 DN on a gray surface in the example camera. Another test checks the sum itself against the classical formula for a distant sphere, (2/3) albedo (R/D)^2 Phi(alpha), to within 0.3 %.

On the RTX 3060 this costs 0.04 ms of GPU time per target (0.28 ms for eight), and 1.4 ms per target on lavapipe.

Limitations:

• No shadows. The target's own parts do not block light from Earth or the Moon. A panel facing Earth does not shade the bus behind it from Earthshine, and the inside of a dish facing Earth is lit as if open to the whole Earth. Earth fills too much of the sky for one shadow map to stand in for it; a ray tracer would handle this.
• One spectrum per channel. Light from Earth is bluer than sunlight, but the target's albedo in each channel was worked out under sunlight. This is the shortcut described in "Folding spectra into per-channel constants", and its error is largest for strongly colored surfaces.
• Lambertian bodies. Earth and the Moon reflect as Lambertian surfaces here, as they do in the camera's view, with the Moon's map scaled by its phase curve as seen from the target. Earth has no clouds or atmosphere yet beyond the stand-in haze.

Where the Sun and Moon appear

Code: scene.rs (Velocities, aberrate), ephemeris.rs (feature astrokit), shaders/bodies.wgsl (seen).

With the astrokit feature, the Sun's and Moon's positions come from JPL's DE440s ephemeris, and Earth's orientation from the ITRF93 kernel, which includes the slow wobble of Earth's pole. These are geometric positions: where each body is at the epoch.

A camera sees something slightly different, its apparent position:

• Light time. Light from the Moon takes 1.3 s to reach Earth, and from the Sun 8.3 minutes. The camera sees where each body was when the light left it. For the Moon that is about 1.3 km of motion, a few microradians.
• Aberration. A moving camera sees incoming light tilted toward its direction of motion, as rain seems to slant toward a moving car. Earth's orbital speed (30 km/s) tilts the Sun's apparent direction by about 20 arcseconds, 100 microradians. A spacecraft's own orbital velocity adds up to about 5 arcseconds more.

100 microradians is about one pixel of the example camera, and 18 pixels of a narrow camera with a 1 m focal length.

The two effects combine simply. Light time moves a body back along its own velocity by (distance / c) times that velocity; aberration moves it along the observer's velocity by the same factor. To first order, then, a body appears moved in direction by the observer's velocity relative to the body, divided by c, the part of it across the line of sight. Only relative velocities matter. For the Sun that is Earth's velocity around it plus the spacecraft's: the 20 arcseconds. For the Moon, Earth's 30 km/s cancels, since the Moon shares it, and what is left is the Moon's own 1 km/s around Earth (0.7 arcseconds from Earth's center) plus the spacecraft's velocity. For Earth itself it is the spacecraft's velocity, up to 5 arcseconds in low orbit, about 0.2 px of the example camera. The terms left out are of the order of the velocity squared over c squared, 1e-8 rad.

From camera tier C1 the renderer draws Earth, the Moon and the Sun at their apparent positions. The scene carries the velocities (Scene::velocities: the observer's relative to Earth, Earth's relative to the Sun and the Moon's relative to Earth), and Ephemeris::scene fills in Earth's and the Moon's; the observer's is the caller's. All zero, as in C0, the images are geometric. Each pixel's ray is turned back, per body, by that body's correction before it is tested against the body, so the correction follows the ray's direction across a wide view of Earth, and the lighting, shadows and eclipses, which happen where the bodies really are, stay geometric. Scene::apparent_sun and Scene::apparent_moon give the same directions on the CPU. From Earth's center they match JPL Horizons' apparent positions (light time and stellar aberration) to under 0.002 arcseconds, against 20 arcseconds for the Sun's geometric position.

The camera's attitude is taken as the observer would measure it: a star tracker sees the stars aberrated by the same motion, so the attitude it reports already includes it. A simulation that passes a true, geometric attitude gets images shifted by the aberration, as a real camera's would be against such an attitude.

Still geometric: the targets, whose light time and aberration come from their velocity relative to the observer, meters per second in rendezvous, under 0.01 microradians; the rotation and lighting of a body during its light time (the Moon turns 3 microradians in 1.3 s); and the light-time delay between the Sun lighting a body and the camera seeing it.

Stars

Code: stars.rs (StarCatalog, StarPhotometry); gpu/stars.rs, shaders/stars.wgsl; scripts/fetch-stars.py.

Below lighting tier L1 the sky is black. A real camera shows stars wherever its exposure is long enough, and they matter: star trackers find their attitude from them, a camera measuring the Moon or a target measures it against them, and to an algorithm looking for a target a star is a false one.

Stars are part of the scene, so they come with lighting tier L1, but only a camera at tier C1 or above shows them. A C0 pixel takes its value from the one point at its center, and a star is a point that almost never lies exactly there; from C1 each pixel collects the light over its whole area, which is what records a star.

The catalogs. scripts/fetch-stars.py merges three of ESA's catalogs into data/stars/stars.bin: Hipparcos, 118,000 stars complete to about magnitude 7.3; Tycho-2, 2.5 million stars to about 11.5 (99 % complete to 11.0); and Gaia DR3, from which it takes every star brighter than G 15 by default, 37 million. Each star comes from the first catalog that has it, so the brightest, which saturate Gaia's detectors, keep their Hipparcos or Tycho measurements. Gaia measures brightness in its own bands, G and the color BP - RP: V comes from them by Gaia's published relation (to 0.03 mag), and B - V by a fit to the Hipparcos stars Gaia matched. Each star keeps its position on the sky (ICRS, at epoch J2000), its proper motion and its brightness and color. The renderer moves each star by its proper motion to the scene's epoch when the catalog is loaded. Barnard's star, the fastest, moves 10 arcseconds a year; most move under 0.1. Parallax, under 1 arcsecond for every star, is left out.

Magnitudes. A star's brightness is given as a magnitude: each step of 5 is a factor of 100, and larger is fainter. Vega is about 0, the faintest stars the eye sees about 6, Tycho-2's limit about 11.5, 25,000 times fainter than 0. The magnitude in the Johnson V band, a filter close to the eye's green-yellow, fixes how much light the star sends through that band: V = 0 is 3.631e-11 W/m^2/nm averaged over it (Bessell, Castelli and Plez 1998). The color index B - V, the difference between the star's magnitudes in blue and in V, fixes its temperature: hot stars are blue, B - V below 0, and cool ones red, up to about 2. The renderer takes the temperature from B - V (Ballesteros 2012: the Sun's 0.65 gives 5,780 K), takes a blackbody spectrum at that temperature, scales it so its light through the V band matches the star's magnitude, and folds it through the camera's optics and channels, as for sunlight. A star's spectrum is not a blackbody's: absorption lines and, in cool stars, molecular bands move a few percent of the light between channels. For the example camera, a Sun-colored star of V = 0 on the boresight gives 56 electrons in green in the 0.1 ms exposure, against an independent estimate of 56.4.

Exposure. That 56 electrons is why the example scenes show no stars: an exposure set for a sunlit spacecraft or Earth, a fraction of a millisecond, leaves even the brightest stars at a few hundred electrons and the rest below one. Star trackers expose for 10 to 200 ms, and a telescope's larger aperture collects more: a 1 m f/8 telescope, with a 12.5 cm aperture, gives 100 times what the example camera does.

Drawing. Each frame the renderer finds the stars in the field on the CPU, from bands of declination sorted by right ascension, so it looks only at the stars near the field. It places each at its apparent position, the annual aberration of the observer's motion (about 20 arcseconds, see the previous section) through the lens's distortion. It works out its electrons in each channel: a point source loses light off axis only to the aperture's foreshortening, cos(theta), not the cos^4(theta) of an extended surface, since all of a star's light that enters lands in its spot. A star is not rendered and then blurred: a point of light rendered into a pixel loses where in the pixel it fell (see the blur section). Instead its electrons are spread over the pixels around it as the Gaussian PSF integrated over each pixel's area, which is exact for any position within a pixel. The detector adds them to the light from everything else before the full well, so a bright star saturates like any other light. A star is hidden wherever anything nearer is drawn, Earth, the Moon or a target: stars are at infinity, so a star shows only where the depth image is empty. Stars giving fewer than 0.1 electrons in every channel, far below any detector's noise, are skipped, and the catalog is loaded only to the magnitude that can reach that.

Measured: a star moved across a pixel in tenths, behind a lens with distortion and with Earth's 30 km/s of aberration, lands where the projection puts its apparent direction to 0.01 px, with its electrons matching the photometry to 0.2 %. A star behind Earth's night side or behind a target leaves the bias alone.

Costs. A catalog is loaded only as deep as the camera's exposure can show: for the example camera at 0.1 ms that is V 7.9, 42,000 stars, in 6 ms and 7 MB; at 50 ms, V 14.7, 21 million stars, in 2.8 s and 670 MB; for a 1 m telescope at 0.2 s, the whole catalog, 37 million stars, in 5 s and 1.2 GB. Each frame, finding and placing the stars takes 23 microseconds for the telescope's 0.6 degree field (490 stars toward the Milky Way in Monoceros) and 4.8 ms for the example camera's 12 degree field at 50 ms (52,000 stars). Drawing them takes 0.24 ms for 40,000 stars on an RTX 3060, and 5.6 ms on the CPU through lavapipe.

A catalog's depth matters most for a telescope's narrow field. Tycho-2 alone averages 60 stars per square degree, so a 1 m telescope's 0.2 square degrees would hold about a dozen; with no sky background in space such a telescope records stars to magnitude 14 or 15 in half a second, and Gaia to G 15 averages about 900 per square degree. Deeper still (G 16, 78 million stars) is a run of the script with --gaia 16, at twice the size.

Not modeled yet

• Earthshine on the Moon's night side
• The diffraction pattern's rings, and a lens's measured point spread function
• Non-Lambertian materials (BRDFs), glints, and the full Moon's edge as bright as its center
• Clouds on Earth
• Earth's atmosphere: scattering, the bright limb
• Motion blur, rolling shutter, stray light