17-EFT.WP.Methods.Imaging v1.0 | Chapter 9 Geometric Calibration, Distortion Correction, and Registration

Chapter 9 Geometric Calibration, Distortion Correction, and Registration

One-Sentence Goal
Establish a reversible geometric mapping from physical space to pixel space; output auditable K, d, {R_i,t_i}, LUT_undistort, and cross-view/cross-modality registration transforms; and ensure reprojection and epipolar errors meet contractual limits while remaining stable in streaming scenarios.

I. Scope & Targets

1. Inputs
   • Calibration images and metadata: { I_i }, pattern ∈ { chessboard, charuco, aruco, dot-grid }, f_mode, focus/zoom, T_cam, ts | tau_mono.
   • Target geometry: calibration board pitch d_board, board-plane normal, and planarity tolerance.
   • Priors / extrinsics: multi-camera assembly initial guesses or online synchronized observations (Chapters 5 and 11).
2. Outputs
   • Intrinsics and distortion: K, d = { k1,k2,k3,p1,p2,... } or fisheye { k1,k2,k3,k4 }.
   • Extrinsic set: { R_i,t_i } (camera frame C relative to world frame W), plus per-frame availability tags.
   • Registration/alignment: planar homography H, epipolar geometry E,F, rigid transform T_12 ∈ SE(3), rectify maps, and warp map.
   • Artifacts & manifest: LUT_undistort, LUT_rectify, manifest.imaging.geom, hash_sha256(blob), signature.
3. Applicability & boundaries
   • Default camera model: pinhole + Brown–Conrady distortion; fisheye uses equidistant or equisolid-angle models.
   • Plane-board assumption for initialization; final parameters from bundle adjustment that globally minimizes reprojection error.
   • Rolling shutter: optional linear motion compensation; strongly dynamic scenes require coupling with time-sync cleansing (see Methods.Cleaning v1.0, Chapter 5).

II. Terms & Variables

1. Camera & coordinates
   • K ∈ R^{3×3}: intrinsics (fx, fy, cx, cy, s).
   • R ∈ SO(3), t ∈ R^3: extrinsics, X_C = R * X_W + t.
   • Normalized & pixel coordinates: x_n = ( x_c / z_c , y_c / z_c ), x_p = (u,v,1)^T ~ K * ( x_d, y_d, 1 )^T.
2. Distortion
   • Brown–Conrady: d = { k1,k2,k3,p1,p2 }, with r^2 = x_n^2 + y_n^2.
   • Fisheye (equidistant): r_d = f * theta * ( 1 + k1 theta^2 + k2 theta^4 + ... ), theta = atan2( sqrt( x_n^2 + y_n^2 ), 1 ).
3. Registration
   • Homography: H ∈ PGL(3), s * x_2 = H * x_1 (planar/far scene).
   • Epipolar: x_2^T * F * x_1 = 0, E = K_2^T * F * K_1, E = [t]_x * R.
   • Error metrics: eps_reproj (px), eps_epi (px), coverage_r (radial coverage).
4. Units & dimensions
   unit(u,v)="px", unit(X_W)="m", unit(theta)="rad"; check_dim must pass.

III. Axioms P209-*

• P209-1 (Observability): The calibration set must cover r ∈ [0, r_max] with sufficient angle/distance variation to avoid degeneracy.
• P209-2 (Consistent model choice): For a given lens configuration, the intrinsics/distortion model is fixed across acquisition and publication, with controlled model_id.
• P209-3 (Canonical pixel frame): Pixel coordinates follow a right-handed system; u rightward, v downward; skew s defaults to 0 and must be published if nonzero.
• P209-4 (Planarity & scale): Board planarity must meet tolerance; the nominal pitch d_board is the single scale anchor.
• P209-5 (Robust minimization): Final parameters arise from nonlinear least squares with robust loss rho(•); outlier features are down-weighted or removed.
• P209-6 (Temporal consistency): Frames used for registration/extrinsics obey time-sync contracts; if needed, use tau_mono with interpolation (see Methods.Cleaning v1.0, Chapter 5).
• P209-7 (Invertibility & bounds): Published LUT_undistort is monotone and invertible on Ω_valid; edge extrapolation remains bounded.

IV. Minimal Equations S209-*

• S209-1 (Imaging projection)
• X_C = R * X_W + t；
• x_n = ( X_C.x / X_C.z , X_C.y / X_C.z )；
• x_r = ( 1 + k1*r^2 + k2*r^4 + k3*r^6 ) * x_n；
• x_t = ( 2*p1*x_n*y_n + p2*( r^2 + 2*x_n^2 ), p1*( r^2 + 2*y_n^2 ) + 2*p2*x_n*y_n )；
• x_d = x_r.x + x_t.x，y_d = x_r.y + x_t.y；
• [u, v, 1]^T ~ K * [ x_d, y_d, 1 ]^T。
• S209-2 (Planar-board homography)
  If X_W = (X,Y,0,1)^T lies on the board plane,
  s * x = K * [ r1 r2 t ] * [ X, Y, 1 ]^T for intrinsics and per-frame extrinsic initialization (Zhang).
• S209-3 (Reprojection error)
  eps_reproj = ( 1 / N ) * ( ∑ || x_obs - x_hat ||_2 ), with rmse, p95, p99 tracked.
• S209-4 (Epipolar error & essential matrix)
  E = [t]_x * R, F = K_2^{-T} * E * K_1^{-1};
  eps_epi = ( 1 / N ) * ( ∑ | x_2^T * F * x_1 | / || F * x_1 ||_2 ).
• S209-5 (Fisheye equidistant model)
• theta = atan2( sqrt( x_n^2 + y_n^2 ), 1 )；
• theta_d = theta * ( 1 + k1*theta^2 + k2*theta^4 + k3*theta^6 + k4*theta^8 )；
• r_d = f * theta_d，x_d = ( r_d / max( eps, sqrt( x_n^2 + y_n^2 ) ) ) * x_n。
• S209-6 (Rolling-shutter approximation)
  tau(v) = tau_0 + v * dt_row (row time);
  R(v) ≈ Exp( hat(omega) * tau(v) ), t(v) ≈ t_0 + v * v_lin * dt_row for linear compensation.
• S209-7 (Bundle adjustment)
  argmin_{K,d,{R_i,t_i}} ∑_i ∑_j rho( || x_{ij} - Pi( K,d,R_i,t_i; X_j ) ||_2 ).

V. Calibration & Registration Workflow M90-*

• M90-1 Preparation & readiness: Verify units/dimensions; load d_board and board planarity tolerance; set model type and initialization strategy.
• M90-2 Acquisition strategy: Cover near/far, center/edge, rotation/tilt; for multi-camera, co-observe the same board and record tau_mono.
• M90-3 Feature detection: Detect corners/markers per frame; perform subpixel refinement within Ω_valid; discard motion-blurred or overexposed frames.
• M90-4 Initialization: From S209-2, compute per-frame homographies and derive K0 and per-frame R0,t0; for fisheye, use a wide-angle initializer.
• M90-5 Nonlinear refinement: Minimize S209-7 with Huber/Cauchy loss and gradient-based stopping; solve configurations in parallel (focus/zoom variants).
• M90-6 Model selection: Compare AIC/BIC and cross-validated errors to choose Brown–Conrady vs. fisheye polynomial order.
• M90-7 Artifact generation: Build LUT_undistort, rectify maps; lock Ω_valid and edge extrapolation policy.
• M90-8 Multi-view geometry: Compute E,F and T_12; apply epipolar rectification and stereo rectification; run global BA for multi-camera arrays if needed.
• M90-9 Quality & contracts: Evaluate eps_reproj, eps_epi, coverage_r, cond(K); enforce assertions and rollback policies.
• M90-10 Persistence & signing: Output { K, d, {R_i,t_i}, LUT_undistort, rectify maps }; write manifest.imaging.geom and sign.

VI. Contracts & Assertions

• assert reproj_rmse: rmse(eps_reproj) ≤ tol_reproj_rmse and p99(eps_reproj) ≤ tol_reproj_p99.
• assert epi_error (multi-camera): p95(eps_epi) ≤ tol_epi; no systematic bias in the epipolar-direction residuals.
• assert radial_coverage: coverage_r ≥ r_min_cover; edge regions contribute at least q_edge of corners.
• assert condK: 1 ≤ cond(K) ≤ tol_condK; focal is reasonable and fx/fy within tol_aspect.
• assert dist_monotone: dr_d/dr ≥ 0 for r ∈ [0, r_max]; if violated, lower polynomial order and re-estimate.
• assert cheirality: all inliers satisfy z_c > 0; negative-depth fraction ≤ tol_neg_depth.
• assert rs_model (optional): fit quality for rolling-shutter parameters R^2 ≥ tol_rs_r2.
• assert manifest_bind: hash_sha256( LUT_undistort ) matches the manifest; camera pose, time, and temperature are traceable.

VII. Implementation Bindings I90-*

• I90-1 detect_calib_points(I, pattern_cfg) -> { points }, quality_tags
• I90-2 init_homographies({ points }, d_board) -> { H_i }
• I90-3 solve_intrinsics({ H_i }) -> K0, pose0
• I90-4 refine_with_bundle({ points }, K0, pose0, model_cfg) -> K, d, { R_i,t_i }, report
• I90-5 build_undistort_LUT(K, d, Ω_valid) -> LUT_undistort
• I90-6 undistort_apply(I, LUT_undistort) -> I_ud
• I90-7 estimate_EF_T({ matches }, K1, K2) -> E, F, T_12, inliers
• I90-8 stereo_rectify(K1,d1,K2,d2,T_12) -> rectify_maps, Q
• I90-9 global_bundle_multi({ tracks }, { K_m, d_m }) -> { R_i^m, t_i^m }, report_global
• I90-10 validate_geom_contracts(metrics, contracts) -> assert_report.geom
• I90-11 emit_manifest_geom(params, artifacts, hashes) -> manifest.imaging.geom

VIII. Cross-References

• Radiometric metrology & linear domain: this volume’s Chapter 4—decouples geometry and radiometry while keeping units consistent.
• Optical imaging & PSF: this volume’s Chapter 5—note potential coupling between edge MTF and geometric distortion.
• Sampling & interpolation: this volume’s Chapter 6—resampling kernels and spectral-preservation strategies for undistort/rectify.
• Noise & robust estimation: this volume’s Chapter 7—confidence and down-weighting for corner detection and matching.
• Flat/dark calibration: this volume’s Chapter 8—apply LUT_undistort preferably in the RAW domain after FPN correction.
• Time sync & streaming: Methods.Cleaning v1.0, Chapters 5 and 11—registration nodes participate in the backpressure-controlled execution graph.
• Data specification & manifests: EFT.WP.Core.DataSpec v1.0 and the companion EFT white papers for interface fields and artifact signatures.

IX. Quality Metrics & Risk Control

1. Core metrics
   • rmse/p95/p99(eps_reproj), p95(eps_epi), coverage_r, inlier_rate, cond(K).
   • Runtime drift: drift_fx, drift_cx, drift_k1 evaluated over windows Delta_t; alert thresholds and rollback strategies.
   • Resource metrics (streaming): latency_undistort, cpu/mem, drop_rate.
2. Key risks & playbooks
   • Board planarity or scale error: enable scale cross-checks or dual-board mutual verification.
   • Motion/rolling-shutter mismatch: enable S209-6 model or drop high-speed frames; couple with time-sync cleansing.
   • Model overfit: if dist_monotone fails, reduce polynomial order or switch to piecewise LUT.
   • Focal/thermal drift: map { focus, zoom, T_cam } -> { K,d } and switch configurations at runtime.
   • Cross-modality registration failure: switch to mutual information or edge-structure measures; when needed, use semi-automatic anchors.

Summary
This chapter provides a unified workflow for geometry and registration—from planar calibration and epipolar geometry to global BA—producing K, d, {R_i,t_i}, LUT_undistort/rectify, and quality reports. Manifests and signatures guarantee traceability and rollback, while contracts and metrics sustain stable operation in multi-camera / multi-modality systems and streaming deployments.