Skip to content

12. Conformance and Test Mapping

12.1 Conformance Claim

A baseline conformance claim requires:

  1. Normative behavior alignment with this requirement set, especially Chapter 4 and Chapters 5-9
  2. Passing mapped conformance tests for the baseline
  3. Versioned evidence artifacts for regression comparability

12.2 Coverage Domains

Conformance covers:

  1. Grammar/parser behavior
  2. AST to IR compilation and span guarantees
  3. Semantic validation contracts
  4. Type and unit checking
  5. Plan lowering behavior
  6. Runtime state transitions and error policy
  7. Material conservation and binding behavior
  8. Surface sugar to domain built-in lowering contracts
  9. Content classification whitelist and diagnostic-stage mapping
  10. Runtime material-accounting and report-projection behavior

Scope:

  1. Conformance scope includes single-source parsing, definition dependency loading, and CLI batch fan-out behavior.
  2. LOAD_* diagnostics and entry-source/dependency-loading hard-error behavior require dedicated conformance rows because source loading is part of the public execution contract.
  3. Protocol parameterization (protocol params + named call args + PLAN_CALL_*) is part of scope.
  4. Repeat-binding scope coverage MUST accept name reuse across sibling repeat statements whose scopes do not overlap and reject shadowing of parameters, local let names, or active enclosing repeat bindings.
  5. Entry-source script execution and omission of dependency-file top-level statements from execution are part of scope.
  6. CLI single-run and batch execution surfaces are part of scope.
  7. Direct execution of an entry source without an executable entry fails, while the same source remains valid as a definition dependency.

12.3 Requirement-to-Test Matrix Source

Requirement-to-test mapping is maintained across:

  1. culsma-reference/04_surface_grammar_and_statement_forms.md
  2. culsma-reference/05_operation_vocabulary_and_call_contracts.md
  3. culsma-reference/06_validation_and_diagnostics.md
  4. culsma-reference/07_execution_model.md
  5. the project conformance test suite that checks those contracts

The combined reference-plus-tests surface is the conformance registry for the baseline.

12.4 Minimal CI Gate

Minimum suite for a baseline conformance claim:

  1. parser/source-surface tests
  2. frontend resolution tests
  3. IR compilation tests
  4. semantic validation tests
  5. type/unit checking tests
  6. plan-lowering tests
  7. runtime execution tests
  8. material-state compute tests
  9. Appendix A fixture coverage
  10. container/content constructor and material-state tests

12.5 Evidence Artifacts

Recommended evidence bundle includes:

  1. parser / IR / plan assertions captured by unit tests
  2. runtime events, step_status, and protocol_outputs assertions from kernel runtime tests
  3. CLI artifact bundles (summary.json, ast.json, ir.json, validate.json, typecheck.json, plan.json, run.json, output.json, result.json) when generated for regression evidence
  4. Machine-readable single-run CLI output that keeps protocol returns separate from the derived run report
  5. Machine-readable batch CLI output where each item preserves an independent return value, report, diagnostics, and runtime artifact set
  6. Appendix A fixture evidence
  7. Diagnostic summaries and failing-case traces

12.6 Change Management Rule

Spec-changing updates must be synchronized across:

  1. Requirement text
  2. Requirement-to-test mapping rows
  3. Test implementation and snapshots

A change is not complete until all three layers are updated together.

12.7 Non-Conformance Cases

Typical non-conformance examples:

  1. Behavior changes without updating requirement mappings
  2. New diagnostics without stable code contracts
  3. Passing implementation tests that no longer map to declared requirements
  4. Sugar-lowering behavior changes without updating constructor contract tests
  5. Whitelist policy changes without updating semantic/type/runtime diagnostic mapping

12.8 Material Content Requirement Mapping

The following referenced requirements must have test rows before claiming baseline conformance:

  1. Chapter 6 content_kind whitelist enforcement under strict mode
  2. Chapter 6 closed canonical content_type table and other_* fallback behavior
  3. Chapter 4/6 constructor lowering equivalence (blood(...) vs canonical DefineContent(...))
  4. Chapter 8 earliest-decidable diagnostics for content contract failures
  5. Chapter 7/8 runtime material diagnostics for missing/invalid content identities
  6. Chapter 8 content diagnostic mapping table alignment (8.2.2)
  7. Chapter 6 attrs.role treatment without a top-level role API
  8. Chapter 7 and the Scientific Model Extension Contract: authored-decision precedence, explicit capability binding, built-in default provider, provenance, proposal validation, and unresolved results without mutation.
  9. Built-in Material Rulebook provider behavior, including canonical classification, exact ideal routing, relationship transitions, and explicit UNRESOLVED instead of conservative equal-split guesses.
  10. Author-supplied sep.transitions behavior, including ordered live-material index selectors, exact container-local entry-ID selectors, static selector aliases, omission of zero-quantity entries, exact program-owned output-enum matching, every pair in the closed author-settable MaterialRelation domain, rejection of strings, bare identifiers, cross-program output members, unknown members, and UNRESOLVED, typed component-entry association targets, positive-output checks, atomic candidate rejection, and the bead_bound -> free elution regression.
  11. Runtime component-entry collision allocation, including unsuffixed IDs, _1 / _2 ordinal suffixes, and preservation of pre-existing opaque IDs.

12.9 Conformance Evidence Categories

  1. culsma-reference/04_surface_grammar_and_statement_forms.md
  2. culsma-reference/05_operation_vocabulary_and_call_contracts.md
  3. culsma-reference/06_validation_and_diagnostics.md
  4. culsma-reference/07_execution_model.md
  5. conformance-linked project tests

12.10 Statement-First Atomic Contract Coverage

Conformance should include explicit rows for Chapter 6 atomic contracts:

  1. << parsing + style constraints (full-transfer vs quantified, program source shape).
  2. source.partition(program)[i] mutation source parsing, accepted binary partition program family, invalid program rejection, static binary index validation, and mutation-source-only context rejection.
  3. container.contents[i] mutation source parsing, static index validation, mutation-source-only context rejection, missing/stale/preservation-context contents-state runtime diagnostics, selected-part transfer, and same-container disturbance behavior.
  4. with env shape and required target binding.
  5. sep core/program split binding, concrete *_program(...) family checks, program-specific material component partition coverage, let-bound group return compatibility, and standalone contents-state behavior.
  6. frac ordered return, stable index usage, let-bound group return compatibility, and standalone contents-state behavior.
  7. img/ecp/phy readout call contracts and return type checks.
  8. agit over a single container, an explicit container group, and a let-bound container group, including independent per-member effects, preservation of material quantities, invalidation of prior indexed-contents organization, and failure before group effects when any member cannot be resolved.
  9. plate(...) selector expansion, including inheritance of an explicit per-well capacity, static indexing of the resulting ordered well group, original-well identity preservation, and out-of-range index rejection.
  10. Chapter 7 logical container identity distinctions across definition scopes, repeated or nested invocations, and allocating iterations, including their independence from labels, external identifiers, and physical-carrier metadata. Coverage checks observable identity behavior and does not prescribe an identifier representation or encoding.

12.10.1 Electrical Control and Disruption Method Mapping

The following hooks cover the accepted Chapter 6 program contracts and the Chapter 8/11 diagnostic and unit boundaries. Test identifiers refer to the companion Culsma implementation; they are evidence, not additional semantics.

RequirementConformance hookImplementation evidence
FIELD-CONTROL-01Accept exactly one of voltage, current, or the compatibility field alias; reject missing and conflicting controls.tests/test_kernel_validate.py: test_validate_field_program_accepts_one_electrical_control_mode, test_validate_field_program_requires_an_electrical_control_mode, test_validate_field_program_rejects_conflicting_control_modes
FIELD-CONTROL-02Emit SEM_FIELD_PROGRAM_FIELD_ALIAS for field; canonicalize inline and let-bound descriptors to voltage.tests/test_kernel_validate.py: test_validate_field_program_field_alias_warns; tests/test_kernel_plan.py: test_plan_canonicalizes_field_program_control_args, test_plan_canonicalizes_let_bound_field_program_alias
FIELD-CONTROL-03Accept electric-potential units for voltage/field and electric-current units for current; reject cross-dimension controls.tests/test_kernel_typecheck.py: test_typecheck_field_program_accepts_typed_electrical_control_modes, test_typecheck_field_program_rejects_wrong_control_dimensions
FIELD-CONTROL-04The bundled Electrophoresis portal lowers its voltage parameter through field_program(voltage=...).tests/test_kernel_plan.py: test_plan_let_bound_electrophoresis_stdlib_has_no_legacy_electrophoresis_step
DISRUPT-METHOD-01Accept omitted method and all five controlled method values; reject unknown values.tests/test_kernel_validate.py: test_validate_disrupt_program_accepts_current_optional_args, test_validate_disrupt_program_accepts_device_independent_methods, test_validate_disrupt_program_rejects_unknown_method
DISRUPT-METHOD-02Keep equipment binding outside disrupt_program; reject a device argument.tests/test_kernel_validate.py: test_validate_disrupt_program_rejects_device_binding

Additional coverage for environment targets and container target views:

Coverage areaRequired coverage
Env target declaration shapecanonical positional hold(target) form, scope legality, direct-child declaration collection, legacy named-argument compatibility, and preservation that hold(...) declarations do not narrow the lexical env context for executable body statements
Container target-view accessorsaccepted contents and structure.top/bottom/sidewall forms, plus rejection of incomplete, invalid, or deeper structure paths
Target-view typingtarget views accepted only in HoldTarget / ContainerTargetView positions and rejected as ordinary material containers
Grouped target viewsexplicit grouping of target-view expressions and rejection of unsupported broadcast-like member access
Diagnostic and error mappinghold scope/shape failures covered as source/compile hard errors; invalid target-view paths covered by SEM_CONTAINER_TARGET_VIEW_INVALID; target-view use in material-container positions covered by TYPE_CONTAINER_TARGET_VIEW_POSITION_INVALID

12.11 Observation Non-Mutation Coverage

Conformance should include checks that:

  1. img/ecp/phy do not mutate material-state quantities.
  2. Observation outputs are persisted as readout/result references.
  3. Material conservation assertions are not incorrectly applied to pure readout steps.

12.12 Protocol Return Projection Coverage

Conformance should include checks that:

  1. Returning a single concrete container projects container_ref.
  2. Returning multiple named concrete containers preserves named bindings and projects each binding as container_ref.
  3. Returning direct group([c1, c2]) projects container_group_ref with ordered members and member_count.
  4. Returning a let-bound group([c1, c2]) projects container_group_ref with ordered members and member_count.
  5. Returning a sep group projects both ordered material slots as container_group_ref.
  6. Returning a frac group projects all ordered material slots as container_group_ref.
  7. Returning a plate selector-derived well group projects all ordered selected wells as container_group_ref.
  8. Returning data_ref or data_group_ref remains a data/readout projection and is not coerced into material projection.
  9. CLI output renders container_group_ref clearly while preserving existing single-container display behavior.

12.13 Core Scope Boundary

Core conformance mapping stops at the atomic statement/operator layer.

Higher-level workflow libraries are outside the scope of this chapter.

12.14 Material-Accounting Report Coverage

Conformance tests for the Chapter 9 derived report contract MUST cover:

  1. registration of externally supplied initial inventory as input lots;
  2. registration of stocks introduced by LoadContent as input lots;
  3. source-to-destination accounting for multi-source and multi-target material movements;
  4. Chapter 9 runtime rejection with MAT_MOVEMENT_CONTRACT_MISSING when material quantities change without an auditable movement contract;
  5. exclusion of generated intermediate material from reagent consumption unless explicitly registered as inputs;
  6. complete, untruncated reagent-consumption output;
  7. Chapter 9 independent volume and mass accounting, including null for an unavailable quantity axis; and
  8. report assertions covering every semantic content area required by Chapter 9, independently of implementation-specific envelope or schema-title names.
  9. independent cell-count load, movement, accounting, and public-result projection without container-volume inflation; and
  10. preservation of count through sep and frac, including a suspension whose cellular count and carrier volume follow different partition fates.
  11. constructor finalization for explicit carrier, count-only inferred carrier, adherent count without carrier, invalid policy, and capacity overflow;
  12. count-to-volume resolution shared by ordinary, source-local partition, and indexed-contents transfers, including non-homogeneous rejection and quantity-axis conflicts; and
  13. separation fate precedence for authored rules and selected-provider decisions, including invalid rules, missing source content, invalid provider proposals, and no mutation on unresolved results.
  14. separation aggregate projection from routed component quantities for volume-only, mass-only, count-plus-carrier, and mixed-axis material; aggregate fields must equal the normalized detail projection; compatibility-only aggregate input must become movable, provenance-bearing component detail before separation, and stale aggregate cache values must not affect routing.
  15. exact built-in-provider routing of sedimentable cellular content under centrifuge_program: zero cellular count in supernatant, complete cellular count in pellet, conservation across both slots, independently routed carrier volume, and exclusion of zero-quantity cells from primary-component reporting.
  16. invariant material execution with and without external inventory input: undeclared sources and insufficient runtime material fail identically, conservation remains enforced, and no source is provisioned or topped up.
  17. optional post-runtime inventory reconciliation: checked/sufficient status, required/available/shortage/remaining quantities on independent axes, external shortage without runtime failure, and no mutation of runtime or supplied inventory state.
  18. cross-container movement of counted cells with container-surface association: the moved fraction becomes a suspension, the residual source remains adherent, canonical attrs.state and the compatibility attrs.culture_state spelling behave consistently, stale separation-output partition classes are replaced, and unrelated non-homogeneous states are not released.
  19. explicit third-party provider replacement and explicit composite-provider fallback, including recorded provider identity/version/configuration and proof that runtime never appends an implicit built-in fallback.

Public language reference for the current Culsma surface.