17-EFT.WP.Methods.Imaging v1.0 | Chapter 12 Computational Imaging (Deconvolution / Super-Resolution / Compressive Sensing)

Chapter 12 Computational Imaging (Deconvolution / Super-Resolution / Compressive Sensing)

One-Sentence Goal
In a calibrated, linear radiometric domain, solve deconvolution, super-resolution, and compressive-sensing reconstructions from explicit forward models plus explicit/implicit priors—ensuring photometric consistency, frequency-domain interpretability, and auditable publication.

I. Scope & Targets

1. Inputs
   • Observations & metadata: y (or multi-frame { y_k }), ts | tau_mono, meta (t, ISO, G, ND, etc.).
   • Optics & sampling: PSF/OTF h(x,y) or H, sampling/downsampling operator S or D, distortion model and registration W_k.
   • Radiometric calibration: camera response f, black level D, flat-field and PRNU/DSNU (Chapters 4 and 8).
   • Noise parameters: sigma_r, sigma_s or noise spectrum S_n (Chapter 7).
   • Compressive sensing: sensing matrix Phi, sparse basis/dictionary Psi, measurement mask mask.
2. Outputs
   • Reconstruction: x_hat (linear, scene-referred); optional high-resolution x_hat^HR.
   • Quality & spectra: MTF_out, PSNR, SSIM, LPIPS (optional), ringing/aliasing indicators.
   • Artifacts & manifest: ci_profile.v1 (algorithm/params/priors/stopping), manifest.imaging.ci, hash_sha256(profile), signature.
3. Applicability
   • Deconvolution: spatially invariant or block-wise variant PSF; strongly space-variant PSF via tiling or coordinate convolution.
   • Super-resolution: single-frame and multi-frame (y_k = D W_k H x + n_k); cross-modal SR requires prior color & geometry binding (Chapters 10, 9).
   • Compressive sensing: adequate m/n, mutual incoherence between Phi and Psi; RIP verification as an optional audit.

II. Terms & Variables

1. Variables & operators
   • x: ideal high-quality image; y: observation; n: noise; h/H: PSF/convolution; W_k: registration/warp; D: downsample; S: sample; Phi: measurement; Psi: sparsifying basis; S_x, S_n: signal/noise spectra.
   • R(x): prior regularizer (L2, TV, L1 in transform, deep prior); prox_R: proximal operator.
   • lambda, rho, mu, alpha: weights and hyperparameters.
2. Spectra & quality
   • MTF_in / MTF_out, MTF_gain(f) = MTF_out(f) / MTF_in(f); ringing_rate, zippering_rate.
   • Data-consistency residual: res = || A x_hat - y ||_2 / || y ||_2, where A is the forward operator.

III. Axioms P212- (CI Baseline)*

• P212-1 (Linear-domain solving): All reconstructions operate in the linear radiometric domain: lin = f^{-1}(raw) - D, with flat-field and PRNU/DSNU already corrected.
• P212-2 (Physical consistency): Deconvolution uses energy-normalized PSF, sum(h) = 1 ± tol_psf_norm; SR forward D must respect Nyquist semantics; Phi must be consistent with exposure/geometry metadata.
• P212-3 (Noise awareness): Cost functions and weights explicitly depend on the noise model or S_n to avoid high-frequency noise blow-up.
• P212-4 (Geometry first): Multi-frame / multiview reconstructions require Chapter 9 registration W_k and alignment on tau_mono (Methods.Cleaning, Ch. 5).
• P212-5 (Interpretable priors): Regularizers or deep priors must specify form, strength, and their effects on energy/chrominance/edges.
• P212-6 (Chrominance conservation): Perform color reconstructions in the scene-referred color domain per Chapter 10; prevent cross-channel sharpening inconsistencies.
• P212-7 (Auditable & revertible): Log algorithm ID, random seed, hyperparameters, and stopping rules; if contracts fail, fall back to a conservative filter.
• P212-8 (Publication separation): Rendering (tone/OETF) must not feed back into the reconstruction solution.

IV. Minimal Equations S212-*

• S212-1 (Deconvolution forward model)
  y = H x + n, or in frequency Y = OTF * X + N.
• S212-2 (Wiener deconvolution)
  X_hat = ( conj(OTF) / ( |OTF|^2 + S_n / S_x ) ) * Y, x_hat = F^{-1}( X_hat ).
• S212-3 (Tikhonov/MAP)
  x_hat = argmin_x ( (1/(2 sigma^2)) * || Hx - y ||_2^2 + lambda * || L x ||_2^2 ),
  normal equations: ( H^T H + lambda * L^T L ) x = H^T y.
• S212-4 (TV deconvolution, ADMM)
  x_hat = argmin_x ( (1/(2 sigma^2)) * || Hx - y ||_2^2 + lambda * || ∇x ||_1 ).
  ADMM iterations:
• v^{t+1} = prox_{(lambda/rho)||•||_1}( ∇x^t + u^t )；
• x^{t+1} = argmin_x ( (1/(2 sigma^2)) * || Hx - y ||_2^2 + (rho/2) * || ∇x - v^{t+1} + u^t ||_2^2 )；
• u^{t+1} = u^t + ( ∇x^{t+1} - v^{t+1} )。
• S212-5 (Multi-frame SR, MAP)
  y_k = D W_k H x + n_k;
  x_hat = argmin_x ( ∑_k (1/(2 sigma_k^2)) * || D W_k H x - y_k ||_2^2 + lambda * R(x) ).
• S212-6 (Plug-and-Play / RED sketch)
  Alternate data-consistency and prior:
• z^{t+1} = argmin_z ( (1/(2 sigma^2)) * || A z - y ||_2^2 + (rho/2) * || z - x^t ||_2^2 )；
• x^{t+1} = Denoiser_sigma( z^{t+1} )。
• S212-7 (Compressive sensing, L1)
  y = Phi x + n, x = Psi alpha;
  alpha_hat = argmin_alpha ( 0.5 * || Phi Psi alpha - y ||_2^2 + lambda * || alpha ||_1 );
  x_hat = Psi alpha_hat.
• S212-8 (CS-ADMM / FISTA step)
  alpha^{t+1} = soft( alpha^t - tau * (Psi^T Phi^T (Phi Psi alpha^t - y)), tau*lambda ).
• S212-9 (Data-consistency metric)
  res = || A x_hat - y ||_2 / || y ||_2, with contract res ≤ tol_res.
• S212-10 (MTF gain)
  MTF_gain(f) = MTF_out(f) / MTF_in(f), with contract median_{f∈B}(MTF_gain(f)) ≥ tol_mtf_gain, where band B is set by system NA and sampling limits.

V. Pipeline & Operational Flow M120-*

• M120-1 Readiness: Load raw/y, meta, PSF/OTF, S/D, Phi/Psi; run check_dim and unit checks.
• M120-2 Linearization & calibration: lin = f^{-1}(raw) - D, then flat-field and PRNU/DSNU corrections (Chapters 4, 8).
• M120-3 Noise estimation: Estimate sigma_r, sigma_s or S_n (Chapter 7) and build noise weights.
• M120-4 PSF/OTF binding: Import h/OTF from Chapter 5; for space-variant PSF, use blocks or coordinate convolution.
• M120-5 Sampling & geometry: Bind D/S and registration W_k (Chapters 6, 9); align multi-frame data on tau_mono.
• M120-6 Model selection: Choose deconvolution / multi-frame SR / CS, select R(x) and solver (Wiener/ADMM/FISTA/Plug-and-Play).
• M120-7 Initialization: Frequency-domain inverse, bicubic upsampling, or zero-filled sparse coefficients, as appropriate.
• M120-8 Iterative reconstruction: Alternate data-consistency and prior steps; monitor res, objective descent, and early-stopping criteria.
• M120-9 Color path: Regularize channel consistency in the scene-referred domain; keep rendering separate (Chapter 10).
• M120-10 Artifact suppression: Detect ringing; protect edges; reduce lambda or apply spectral windows if needed.
• M120-11 Quality assessment: Compute PSNR/SSIM/LPIPS, MTF_gain, ringing_rate/zippering_rate.
• M120-12 Contracts & fallback: If res or artifacts exceed thresholds, fall back to conservative solutions (Wiener/non-blind sharpening/single-frame SR).
• M120-13 Persistence & signature: Output x_hat, spectra, and metrics; generate ci_profile.v1 and manifest.imaging.ci and sign.

VI. Contracts & Assertions

• assert psf_norm: | sum(h) - 1 | ≤ tol_psf_norm.
• assert data_consistency: res ≤ tol_res, monotonically decreasing to stopping.
• assert mtf_gain: median_{f∈B}(MTF_gain(f)) ≥ tol_mtf_gain.
• assert photometric: | mean(x_hat) - mean_ref | ≤ tol_dc (with mean_ref from energy conservation or reference patches).
• assert ringing: ringing_rate ≤ tol_ringing; zippering_rate ≤ tol_zipper.
• assert geometry_bind: for multi-frame, EPE ≤ tol_align_epe, inlier_ratio ≥ tol_inlier.
• assert cs_feasibility: in CS, m/n ≥ tol_sampling and || Phi x_hat - y ||_2 / || y ||_2 ≤ tol_cs_res.
• assert reproducibility: hash_sha256(ci_profile.v1) matches the runtime log.

VII. Implementation Bindings I120-*

• I120-1 build_forward_model(meta, H, D|S, Phi, Psi) -> A
• I120-2 estimate_psf(frames, charts) -> h|OTF
• I120-3 wiener_deconv(y, OTF, Sx, Sn) -> x_hat
• I120-4 tv_admm_deconv(y, H, lambda, rho) -> x_hat
• I120-5 multiframe_sr_map({ y_k }, { W_k }, H, D, lambda, R) -> x_hat^HR
• I120-6 sr_pnp(y, A, denoiser, sigma, iters) -> x_hat
• I120-7 cs_reconstruct_fista(y, Phi, Psi, lambda, iters) -> x_hat
• I120-8 data_consistency_step(y, A, x, weight) -> z
• I120-9 eval_mtf_psnr_ssim(x_in, x_out) -> { MTF_gain, PSNR, SSIM }
• I120-10 detect_artifacts(x) -> { ringing_rate, zippering_rate }
• I120-11 emit_ci_profile(params, hashes) -> ci_profile.v1
• I120-12 bind_manifest_ci(profile, metrics) -> manifest.imaging.ci

VIII. Cross-References

• Optics & imaging kernels: Chapter 5 PSF/OTF/MTF—ensure H matches system frequency response.
• Sampling & interpolation: Chapter 6—declare D/S and their relations to Bayer/multi-resolution layouts.
• Noise modeling: Chapter 7—drive weights and regularization strength.
• Flat/dark & pattern noise: Chapter 8—avoid PRNU/DSNU amplification post-deconvolution.
• Geometry & registration: Chapter 9—W_k and alignment metrics for multi-frame SR.
• Color management: Chapter 10—reconstruct in scene-referred space; keep rendering decoupled.
• Time & synchronization: Methods.Cleaning v1.0, Chapter 5—ensure multi-frame consistency on tau_mono.
• Data spec & signing: EFT.WP.Core.DataSpec v1.0—fields & process for ci_profile.v1 and manifest.imaging.ci.

IX. Quality Metrics & Risk Control

1. Metrics
   • Absolute/relative: PSNR, SSIM, LPIPS (optional), MTF_gain, res, ringing_rate, zippering_rate.
   • Runtime: convergence rate, per-frame latency, throughput and memory ceilings (Threads SLOs).
2. Risk playbooks
   • H mismatch / non-stationary PSF: use blocks or adaptive kernel estimation; fall back to conservative filtering.
   • Over-sharpening: lower lambda or employ edge-preserving priors; cap MTF_gain.
   • Noise amplification: noise-aware weights, spectral windows, or post denoising (Chapter 7).
   • Registration errors: strengthen robust W_k estimation and occlusion masks; locally revert to single-frame.
   • CS under-sampling / model drift: increase measurements, switch to robust priors, or relax residual thresholds and mark as degraded.

Summary
This chapter unifies the forward models and solvers—deconvolution y = Hx + n, multi-frame SR y_k = D W_k H x + n_k, and CS y = Phi x + n—and solves them with noise-aware, interpretable priors using Wiener/Tikhonov/TV/ADMM/FISTA/Plug-and-Play families. Contracts on data consistency, MTF_gain, photometric fidelity, and artifact suppression enforce safe publication; failures trigger rollback. With ci_profile.v1 and manifest.imaging.ci, results remain reproducible and auditable across devices and scenes.