17-EFT.WP.Methods.Imaging v1.0 | Chapter 8 Flat/Dark Calibration and Fixed-Pattern Noise (PRNU/DSNU)

Chapter 8 Flat/Dark Calibration and Fixed-Pattern Noise (PRNU/DSNU)

One-Sentence Goal
Using a reproducible flat-field / dark-field calibration workflow, construct PRNU k(x,y) and DSNU b(x,y), form an in-operation non-uniformity correction (NUC), and ensure fixed-pattern noise is reduced below contractual thresholds in the radiometric linear domain—with full auditability and replay.

I. Scope & Targets

1. Inputs
   • Linear-domain raw frame sets: flat sequence { I_flat^{(i)} }, dark sequence { I_dark^{(j)} }, and metadata { gain g, black B, exposure E, T_cam, ISO, CFA(pattern) }.
   • Illumination-field uniformity or optical shuttering info: LSF(x,y) (from Chapter 5), integrating-sphere or uniform-panel calibration conditions.
   • Linearization and unit mapping (Chapter 4): unit(I_lin), dim(I_lin), check_dim(expr).
2. Outputs
   • Fixed-pattern parameters and correction artifacts: b(x,y), k(x,y), defect_mask, NUC = { B_map, G_map }, where B_map = b(x,y) and G_map = ( 1 + k(x,y) )^{-1}.
   • Temperature / exposure dependencies: d_rate(T), eta_T, plus stability and residual metrics RNU_pre, RNU_post.
   • Release artifacts & traceability: manifest.imaging.fpn, hash_sha256(blob), signature.
3. Boundaries & Applicability
   • Default execution and application of NUC are in the RAW linear domain (Bayer or monochrome); RGB-domain use is for acceptance checks only.
   • Multi-gain / multi-conversion-gain (dual/quad CG) modes must be calibrated per mode and switched at runtime per operating condition.
   • Saturated, overflow, or clipped pixels are excluded from fitting (define Ω_valid).

II. Terms & Variables

1. Fixed-pattern components
   • DSNU b(x,y): additive non-uniformity (dark-signal non-uniformity), with the same unit as I_lin.
   • PRNU k(x,y): multiplicative non-uniformity (pixel response non-uniformity), dimensionless with E[k]=0.
2. Observation model & correction
   • Observation: I_obs = ( 1 + k(x,y) ) * I_true + b(x,y) + n_add.
   • Dark current: I_dark(E,T) = B + d_rate(T) * E + n_dark.
   • Runtime correction (NUC): I_corr = ( I_obs - b_T ) / ( 1 + k ), where b_T = b_0 + eta_T * ( T_cam - T_ref ) + d_rate(T) * E.
3. Other
   • Optical/illumination field S(x,y) (includes vignetting / non-uniform illumination), approximated by low-pass: S(x,y) = LP( I_flat_corr ).
   • Residual non-uniformity: RNU = std( I / mean(I) ), evaluated over Ω_valid.

III. Axioms P208- (FPN Baseline)*

• P208-1 (Linear domain, no clipping): All fitting is on I_lin; pixels used in fitting satisfy not_saturated(I).
• P208-2 (Additive–multiplicative decomposition): Fixed pattern decomposes into additive b(x,y) and multiplicative k(x,y), combined linearly in the correction equation.
• P208-3 (Separable illumination field): A flat frame is modeled as I_flat ≈ ( 1 + k ) * S + b + n, where S(x,y) is dominated by spatial low frequencies.
• P208-4 (Robust statistics): Estimation of b,k employs robust statistics (median, Huber/biweight) against outliers and flicker noise.
• P208-5 (Explicit temperature/exposure dependence): Publish estimates or bounds for d_rate(T) and eta_T.
• P208-6 (Pixel-level contracts): Pixels exceeding limits are flagged into defect_mask; runtime uses interpolation or neighborhood fill.
• P208-7 (Reproducibility): All intermediate results, windows, and thresholds are recorded in manifest.imaging.fpn and verifiable via hash_sha256.

IV. Minimal Equations S208-*

• S208-1 (Dark-field fit: bias & current)
  I_dark^{(j)}(x,y) = B + d_rate(T_j) * E_j + n_dark.
  Per-pixel linear regression yields B_hat(x,y) and d_rate_hat(x,y); define
  b_0(x,y) = B_hat(x,y) - mean( B_hat ), b_T(x,y) = b_0 + ( d_rate_hat * E_ref ) + eta_T * ( T_cam - T_ref ).
• S208-2 (Flat-field normalization & illumination separation)
  Dark correction: F_corr = I_flat - ( B_hat + d_rate_hat * E_flat ).
  Low-pass illumination estimate: S = LP( F_corr ; r ) or directly use LSF(x,y); multiplicative residual:
  k(x,y) = F_corr / ( S * mean( F_corr / S ) ) - 1.
• S208-3 (NUC application)
  I_corr = ( I_obs - b_T ) / ( 1 + k ), with clipping to dynamic range.
• S208-4 (Stability & residuals)
  RNU_pre = std( I_obs / mean(I_obs) ), RNU_post = std( I_corr / mean(I_corr) );
  require RNU_post ≤ rho_rnu * RNU_pre and RNU_post ≤ tol_rnu_abs.
• S208-5 (Defect detection)
  z(x,y) = ( k / sigma_k ) or z(x,y) = ( b / sigma_b ); label defects when |z| ≥ z_thresh.
  Stripe/column noise may be detected via NPS(f) peak criteria along row/column directions.

V. Calibration Workflow M80-*

• M80-1 Linearization & readiness: Obtain I_lin per Chapter 4; verify unit/dim; mask saturation and initial bad pixels.
• M80-2 Acquisition plan: Collect { I_dark }, { I_flat } across several T_cam setpoints and E levels; record illum_ref and uniform-lighting apparatus details.
• M80-3 Dark modeling: For each pixel, regress I_dark = B + d_rate * E to obtain B_hat, d_rate_hat and fit quality r2_dark.
• M80-4 Flat preprocessing: Dark-correct flats with B_hat, d_rate_hat; discard anomalous frames (illumination jitter, transient states).
• M80-5 Illumination separation: Estimate S = LP( F_corr ) or import LSF; for color sensors, compute per CFA channel.
• M80-6 PRNU estimation: Compute k(x,y), perform robust centering, and evaluate sigma_k.
• M80-7 DSNU normalization: Compute b_0(x,y) and eta_T; if needed, group by temperature / gain and store per group.
• M80-8 Defect masking: Using z_thresh, hot/cold rules, and NPS stripe tests, generate defect_mask and specify neighbor repair.
• M80-9 Contract checks: Evaluate RNU_pre/post, r2_dark, and eps_uniform = max_dev( S / mean(S) ); execute assertions.
• M80-10 Persistence & signing: Produce NUC = { B_map, G_map }, defect_mask; write manifest.imaging.fpn and sign.

VI. Contracts & Assertions

• assert linear_domain: check_dim( I_lin ) passes; saturation_rate ≤ tol_sat.
• assert dark_fit: r2_dark ≥ tol_r2_dark, d_rate_hat ≥ 0.
• assert flat_uniformity: eps_uniform ≤ tol_uniform.
• assert prnu_range: std(k) ≤ tol_prnu, |mean(k)| ≤ tol_bias_k.
• assert dsnu_range: std(b_0) ≤ tol_dsnu, |mean(b_0)| ≤ tol_bias_b.
• assert rnu_reduction: RNU_post ≤ rho_rnu * RNU_pre and RNU_post ≤ tol_rnu_abs.
• assert channel_consistency (color): per-CFA-channel std(k_c) does not exceed tol_cross_prnu across channels.
• assert reproducible: hash_sha256( NUC_blob ) matches the manifest.

VII. Implementation Bindings I80-*

• I80-1 build_master_dark(darks, meta) -> B_hat, d_rate_hat, r2_dark
• I80-2 build_master_flat(flats, meta, B_hat, d_rate_hat) -> F_corr_set
• I80-3 estimate_shading(F_corr_set, method, r) -> S
• I80-4 estimate_prnu(F_corr_set, S) -> k_map, sigma_k
• I80-5 normalize_dsnu(B_hat, d_rate_hat, T_ref) -> b0_map, eta_T
• I80-6 detect_defects(k_map, b0_map, nps_cfg) -> defect_mask, tags
• I80-7 apply_nuc(I_raw, B_map, G_map, E, T_cam) -> I_corr
• I80-8 validate_fpn_contracts(metrics, contracts) -> assert_report.fpn
• I80-9 emit_manifest_fpn(params, metrics, hashes) -> manifest.imaging.fpn

VIII. Cross-References

• Linearization & unit consistency: this volume’s Chapter 4; check_dim per Methods.Cleaning v1.0, Chapter 4.
• Optical illumination/vignetting fields: this volume’s Chapter 5—used to model S(x,y) or to import LSF.
• Sampling & channel processing: this volume’s Chapter 6 (independent/joint CFA-channel handling and interpolation impacts).
• Noise modeling & denoising: this volume’s Chapter 7, where PRNU/DSNU serve as fixed-pattern components in the decomposition.
• Missingness, masks, and imputation: Methods.Cleaning v1.0, Chapter 7; defect_mask publication and fill strategies must align.
• Streaming operations & rollback: Methods.Cleaning v1.0, Chapter 11—apply_nuc can run online within the backpressure loop.

IX. Quality Metrics & Risk Control

1. Metrics
   • RNU_pre, RNU_post, std(k), std(b_0), eps_uniform, r2_dark.
   • Frequency domain: directional peaks and band integrals of NPS(f) for stripe/column noise auditing.
   • Stability: cross-temperature/exposure drift_k, drift_b, and u(metric).
2. Risk playbooks
   • Illumination non-uniformity too large: upgrade the uniform-light source or substitute LSF for LP; reacquire.
   • Poor fit: when r2_dark is low, switch to grouped fitting or exclude unstable pixels.
   • Over-correction / texture residues: if RNU_post does not decrease and NPS shows side lobes, reduce filter radius or switch to frequency-domain separation.
   • Temperature drift: if eta_T exceeds threshold, raise thermal-control alerts or enable temperature-lookup for b_T.
   • Defect proliferation: when defect_mask grows, downgrade q_score and trigger sensor health audits.

Summary
This chapter presents unified modeling, calibration, and runtime NUC for PRNU/DSNU—from dark regression and flat-field illumination separation to contract checks and manifest publication—ensuring fixed-pattern noise is controlled, traceable, and rollback-ready under both metrological and visual criteria.