Lens distortion

docs/optics.md

@@ -61,9 +61,82 @@ Some quantities that follow from that:
   IFOV.
 
 Real lenses are not perfect pinholes. They bend straight lines slightly
-(distortion), not modeled yet, and spread each point into a small blur (the
-point spread function, below). The pinhole model is the starting point
-those effects are added to.
+(distortion, next) and spread each point into a small blur (the point
+spread function, below). The pinhole model is the starting point those
+effects are added to.
+
+### Lens distortion
+
+Code: `camera.rs` (`BrownConrady`, `Intrinsics`); `gpu/warp.rs`,
+`gpu/blur.rs`, `shaders/blur.wgsl`, `shaders/distortion.wgsl`.
+
+A real lens moves each point a little away from where the pinhole puts it,
+by an amount that grows toward the edge of the image. Straight lines in the
+scene come out curved. When points move outward more the farther out they
+are, the corners stretch (pincushion distortion); when they move inward,
+the image bulges (barrel distortion). A lens element slightly off center
+adds a smaller sideways shift (tangential distortion).
+
+The standard model, Brown-Conrady, works on the pinhole's normalized
+coordinates x = X / Z and y = Y / Z, with r^2 = x^2 + y^2:
+
+```text
+x_d = x (1 + k1 r^2 + k2 r^4 + k3 r^6) + 2 p1 x y + p2 (r^2 + 2 x^2)
+y_d = y (1 + k1 r^2 + k2 r^4 + k3 r^6) + p1 (r^2 + 2 y^2) + 2 p2 x y
+u = (f / p) x_d + cx,  v = (f / p) y_d + cy
+```
+
+k1, k2 and k3 are the radial terms, positive for pincushion and negative
+for barrel; p1 and p2 are the tangential terms. This is OpenCV's convention,
+so the coefficients a camera calibration produces apply directly (OpenCV
+lists them in the order k1, k2, p1, p2, k3). For the example camera, k1 =
+0.1 alone moves the corners 1.6 px outward.
+
+That matters for navigation, not just for looks. A limb fit, a star
+identification or a target's centroid all assume where each pixel looks;
+an unmodeled 1.6 px is a 0.2 mrad pointing error at the corner, which at
+36,000 km is 6 km. `Intrinsics::project` and `Intrinsics::ray` include the
+distortion (the inverse by Newton's method, to 1e-10 px), so navigation code
+and the renderer share one model; the projection matches OpenCV's to 1e-9
+px.
+
+Rendering it, from camera tier C1 (C0 is always a pinhole): the GPU draws
+the target's triangles with a linear projection, which keeps straight edges
+straight, and moving their corners alone would leave long edges straight
+when they should curve. So the renderer draws the whole frame without
+distortion, the bodies and targets sorted together as usual, on a slightly
+larger and offset area that covers everything the lens bends into view.
+Then it resamples it: for each pixel of the focal plane it finds the
+undistorted point the lens bends onto that pixel and reads the render
+there, interpolating between the four nearest render pixels. The blur's
+first pass (below) does this as it reads its input, so the distorted image
+is never written out. Radiance is
+the same at both ends of a ray, so moving it is all distortion does to an
+extended scene. The resampling works on the finer image C1 renders (below),
+before the blur, which is where the lens puts it in a real camera.
+
+Two effects of distortion fall on the detector instead. The cos^4 falloff
+uses each pixel's real line of sight, the undistorted one. And where the
+lens stretches the image, the same light covers more pixels: each pixel's
+signal is divided by how much the distortion stretches area there (the
+determinant of its derivative), 0.6 % at the corners for k1 = 0.1.
+
+Measured with coefficients of 0.3, -0.2, 0.1 (radial) and 0.001, -0.0008
+(tangential), which move the corners about 4 px: a small Moon's light lands
+centered where `Intrinsics::project` puts it to within 0.004 px across the
+frame, and a long straight edge follows the predicted curve to within 0.07
+px in every row and 0.02 px averaged over rows. The interpolation adds a
+little blur where it falls between render pixels, at most a variance of
+1/16 px^2 at 2 x 2 rendering (computed), which takes sigma 0.5 px to 0.56 px
+at worst.
+
+Distortion costs 0.25 to 0.32 ms per 1920 x 1080 frame on an RTX 3060 at
+2 x 2 rendering, and 1.1 to 1.3 ms at 3840 x 2160, almost all of it the
+blur's first pass reading through it, and under 0.1 ms the larger
+undistorted render. A separate pass writing the distorted image out and the
+blur reading it back cost 0.7 to 0.8 ms and 2.8 to 3.3 ms: at those sizes
+both are limited by how fast the card moves the image through memory. On
+lavapipe distortion adds 10 to 11 ms to 1920 x 1080 frames.
 
 ### Coordinate conventions
 
@@ -1252,7 +1325,6 @@ velocity as an input, which the scene does not carry yet.
 ## Not modeled yet
 
 - Earthshine on the Moon's night side
-- Lens distortion
 - The diffraction pattern's rings, and a lens's measured point spread function
 - Noise: photon shot noise, read noise, dark current, pixel-to-pixel variation
 - Saturation spilling into neighbors (blooming)