Appendix C Data/Model/Pipeline Cards (OT)
I. Abstract & Scope
This appendix defines unified schemas for four cards used in OT (Oriented Tension) workflows—DatasetCard, ModelCard, MetrologyCard, and PipelineCard: required fields, Unit/Dim carriage rules, see: anchors, hashing & versioning, validation, and release flows. If ToA fields are present, both forms MUST be recorded in parallel with explicit {gamma(ell), d ell} and delta_form.
II. Dependencies & References
- Layout & numbering: EFT Technical Whitepaper & Memo Template – Comprehensive Checklist v0.1.
- In-volume anchors: Ch.2 (P80-1/4/5/6), Ch.3 (S80-1/2), Ch.4 (S80-3/4), Ch.5 (M80-1…4), Ch.6 (S80-5/6), Ch.7 (S80-7/8), Ch.10 (S80-15…18), Ch.12 (I80-*).
- Data & pipeline system: I75-*/M75-* (Chapter 14 conventions, reusing this volume’s symbols & units).
III. Normative Anchors (added here, I75-/M75-)
- I75-OT-0 (Card Schemas & Registry): JSON Schemas for the four cards with unified required fields, Unit/Dim, see: anchors, {code_hash,data_hash,rng_state}, and semantic versioning.
- I75-OT-1 (ToA Field Requirement): when ToA is present, cards MUST list T_arr^A/T_arr^B/delta_form and specify integrals:{path:"gamma(ell)", measure:"d ell"}.
- M75-OT-1 (Validation & Release Flow): Schema check → Unit/Dim audit → metric-gate validation → registry publishing; failures block release.
IV. Body Structure
I. DatasetCard Schema & Example
- Minimum fields (★ required)
| Section | Field | Description | Example/Unit |
|---|---|---|---|
| ★meta | dataset_id, version, instrument, band, time_span | basics | "OT_LC_V1","1.0.0","POLCAM","400–800 nm","2023-2025" |
| ★spec | columns[{name, unit, dim, description, see}] | columns with Unit/Dim | Q11: 1, 1; Phi_E: W·m^-2 |
| | sampling, calibration | sampling & calibration | "scan=HEALPix; calib=v2" |
| ★quality | systematics, covariance, masks | systematics, covariance, masks | masks: eta_dom |
| integrals | path, measure | ToA path & measure (if applicable) | "gamma(ell)","d ell" |
| ToA | T_arr^A, T_arr^B, delta_form | dual-form ToA (if applicable) | s, s, "A" |
| ★hash | data_hash, card_hash | integrity | sha256:* |
| ★see | anchors | physics/method anchors | S80-1/4/7 | - JSON (excerpt)
{
"meta": {"dataset_id":"OT_LC_V1","version":"1.0.0","instrument":"POLCAM","band":"400–800 nm","time_span":"2023-2025"},
"spec": {
"columns": [
{"name":"E","unit":"eV","dim":"E","description":"photon energy","see":"Ch.6"},
{"name":"Q11","unit":"1","dim":"1","description":"order tensor component","see":"S80-2"},
{"name":"Phi_E","unit":"W·m^-2","dim":"M T^-3","description":"energy flux","see":"S80-7"}
],
"sampling":"healpix-nside=64",
"calibration":"pol_v2_gainmap"
},
"quality":{"systematics":"3% abs","covariance":"provided","masks":["eta_dom"]},
"hash":{"data_hash":"sha256:...","card_hash":"sha256:..."},
"see":["S80-1","S80-7"]
}
II. ModelCard Schema & Example
- Minimum fields (★ required)
| Section | Field | Description | Example |
|---|---|---|---|
| ★meta | model_id, version, family | model family & version | "OT_S80_v1","1.1.0","S80" |
| ★params | {name, transform, prior, bounds, unit, dim, see} | see Appendix B | A,K,tau_relax,D_Q,Λ_{ijkl} |
| channels | {couplings} | {χ_E,χ_B,χ_u,D1,α_E,α_B,κ_s} | {...} |
| constraints | symmetry, SPD, traceless | physical/mathematical constraints | "Q:traceless, D_eff:SPD" |
| diagnostics | evidence, IC | (optional) evidence & info criteria | logZ, WAIC |
| ★hash | code_hash, card_hash | provenance | sha256:* |
| ★see | anchors | physics anchors | S80-3/4/5/6 | - JSON (excerpt)
{
"meta":{"model_id":"OT_S80_v1","version":"1.1.0","family":"S80"},
"params":[
{"name":"A","transform":"log","prior":"LogU[1e0,1e6]","bounds":[1e-2,1e9],"unit":"J·m^-3","dim":"M L^-1 T^-2","see":"S80-4"},
{"name":"tau_relax","transform":"log","prior":"LN(-2,1)","bounds":[1e-6,1e4],"unit":"s","dim":"T","see":"S80-4"},
{"name":"D_Q","transform":"log","prior":"LogU[1e-14,1e-8]","bounds":[1e-16,1e-6],"unit":"m^2·s^-1","dim":"L^2 T^-1","see":"S80-4"}
],
"channels":{"chi_E":{"prior":"N(0,σ)"}, "alpha_E":{"prior":"U[-0.2,0.2]"}},
"constraints":{"symmetry":"material-group","SPD":["D_eff"],"traceless":["Q"]},
"hash":{"code_hash":"sha256:...","card_hash":"sha256:..."},
"see":["S80-3","S80-4","S80-5","S80-6"]
}
III. MetrologyCard Schema & Example
- Minimum fields (★ required)
| Section | Field | Description | Example |
|---|---|---|---|
| ★meta | metrology_id, version, instrument | identifier | "OT_METRICS_V1","1.0.0","POL+MECH" |
| ★obs | R_inst, noise, covariance | instrument kernel & noise | R_inst: psf, MTF; S_n: PSD |
| ★products | observables | list & units | S_pol, strain, Phi_E |
| audit | UnitsAudit.log | unit audit ref | path/to/log |
| ★see | anchors | M80-1…4 | ["M80-1","M80-2"] | - JSON (excerpt)
{
"meta":{"metrology_id":"OT_METRICS_V1","version":"1.0.0","instrument":"POL+MECH"},
"obs":{"R_inst":{"type":"psf+mtf","version":"2.1"},"noise":{"PSD":"provided"},"covariance":"provided"},
"products":[{"name":"S_pol","unit":"1","dim":"1","see":"M80-1"},{"name":"strain","unit":"1","dim":"1","see":"M80-2"}],
"see":["M80-1","M80-2","M80-4"]
}
IV. PipelineCard Schema & Example
- Minimum fields (★ required)
| Section | Field | Description | Example |
|---|---|---|---|
| ★meta | pipeline_id, version | identifier | "OT_PIPE_V2","2.0.0" |
| ★graph | nodes[], edges[] | DAG node types ∈ {ingest, calibrate, simulate, fit, validate, export} | see below |
| node[i] | {type, inputs, outputs, image/env, seed, resources} | node params | docker://ot:1.3 |
| acceptance | thresholds | metric gates | SpecMAE, EDX-closure |
| exports | {products/, metrics.json, masks/, repro/} | export layout | fixed dirs |
| audit | {code_hash, data_hash, rng_state} | audit fields | — |
| ★see | anchors | related anchors | M80-31…38, M80-41 | - JSON (excerpt)
{
"meta":{"pipeline_id":"OT_PIPE_V2","version":"2.0.0"},
"graph":{
"nodes":[
{"id":"n1","type":"ingest","inputs":["cards/dataset_ot.json"],"outputs":["staged/"],"env":"docker://ot:1.3","seed":1729},
{"id":"n2","type":"fit","inputs":["staged/","cards/model_ot.json"],"outputs":["products/posterior.zarr"],"env":"docker://ot:1.3"},
{"id":"n3","type":"validate","inputs":["products/posterior.zarr"],"outputs":["metrics.json"],"env":"docker://ot:1.3"}
],
"edges":[["n1","n2"],["n2","n3"]]
},
"acceptance":{"SpecMAE":"<=0.03","EDX-closure":"<=1σ"},
"exports":{"products/":true,"metrics.json":true,"masks/":true,"repro/":true},
"audit":{"code_hash":"sha256:...","data_hash":"sha256:...","rng_state":"..."},
"see":["M80-31","M80-35","M80-41"]
}
V. Validation & Consistency (M75-OT-1)
- Schema validation: all four cards MUST pass JSON Schema and required-field checks.
- Unit/Dim audit: every numeric column MUST carry unit/dim; Q_ij is dimensionless with Tr(Q)=0; if spectral flux Phi(E) appears, it MUST be m^-2·s^-1·sr^-1·eV^-1.
- ToA compliance (if applicable): dual-form T_arr^A/T_arr^B/delta_form; integrals with {gamma(ell), d ell}.
- Hash consistency: data_hash matches payload; code_hash matches runtime environment; failures block release.
VI. Release & Audit (I75-OT-0 / M75-OT-1)
- Environment lock: export repro/ (container image, dependency versions, RNG seeds).
- Audit manifest: aggregate {code_hash, data_hash, rng_state, cards, masks, metrics} into the registry.
- Versioning & compatibility: MAJOR changes require migration scripts; MINOR/PATCH keep field backward-compatibility.
VII. Metrics & Gates (aligned with Ch.10/11)
- Metric set: SpecMAE, EDX-closure, AnisoMap-PSNR, logZ, ΔlogZ.
- Pass criteria: meet gate values → pass; otherwise fail with diff report and DiffReport.md.
VIII. Template Lists
- DatasetCard column template (excerpt)
| name | unit | dim | description | see |
|---|---|---|---|---|
| E | eV | E | photon energy | Ch.6 |
| Q11,Q22,Q33,... | 1 | 1 | order-tensor components | S80-2 |
| Phi_E | W·m^-2 | M T^-3 | energy flux | S80-7 |
| T_arr^A/B* | s | T | ToA dual-form | Ch.12 |
* if ToA is used.
- ModelCard parameter template (excerpt)
| name | transform | prior | bounds | unit | dim | see |
|---|---|---|---|---|---|---|
| A | log | LogU[1e0,1e6] | [1e-2,1e9] | J·m^-3 | M L^-1 T^-2 | S80-4 |
| tau_relax | log | LN(-2,1) | [1e-6,1e4] | s | T | S80-4 |
| D_Q | log | LogU[1e-14,1e-8] | [1e-16,1e-6] | m^2·s^-1 | L^2 T^-1 | S80-4 | - Pipeline node template (excerpt)
| type | inputs | outputs | Required | Notes |
|---|---|---|---|---|
| ingest | DatasetCard | staged/ | ★ | validation & standardization |
| fit | staged/, ModelCard | posterior, evidence | ★ | inversion & evidence |
| validate | products/ | metrics.json | ★ | metrics & gates |
| export | targets | bundle | ★ | registry & release |
— This appendix forms the baseline for cardization and engineering release in OT, ensuring verifiable, reproducible, and auditable data–model–pipeline flow end-to-end.