Completes the Wave M B3.5 consolidation slice (b–e), built on #728.
B3.5-b — calibration as a first-class evidence subject (`calibration_class`,
address `calibration:<class_name>`): RightInspector projection + Evidence
Chain Rail semantics (serving-discipline evidence, not runtime truth).
B3.5-c / B4a — nullable `LeewayEvidence` read model threaded through turn,
replay, cognition-proposal, and math-proposal surfaces, with a shared
absence-honest card. B4 is gated correctly: the tuple exists in typed data but
no producer populates it, so the card renders absence (verified: no non-null
producer in workbench/core/chat).
B3.5-d/e — UI-UX-GUIDE.md, b4-leeway-feasibility-gate.md, phase-a-residue-ledger.md.
Practice artifact — earn-it-for-real (runner-reproducible). The committed
`report.json` (additive earns PROPOSE @0.861, 95/5/50) is now emitted by a
deterministic runner rather than copied from the queue. `propose_runner`
gains `regenerate_practice_artifacts()`, which runs ONE sealed `resolve_pooled`
practice pass and writes BOTH report.json (the per-class ledger the calibration
reader consumes) and ratification_queue.json — two projections of one ledger,
coherent by construction and byte-reproducible. `runner.main()` delegates to
it (lazy import, no cycle), so both entry points produce the identical pair.
This closes the gap where a hand-copied report.json agreed with the queue but
no runner produced it. `resolve_pooled` is the aggressive sealed PROPOSE-regime
scorer (proposal-only/HITL, unsafe for serving, legitimate for
attempt-and-eliminate); wrong=5 is the sealed-practice learning signal, NOT the
serving wrong=0. No serving/derivation/reliability_gate source touched; the
practice lane is not in the serving-frozen SHA gate.
Validated:
- python -m pytest tests/test_workbench_{calibration,journal,replay,schemas}.py -> 31 passed
- python -m pytest tests/ -k "workbench or propose or learning_arena or practice"
-> 190 passed (3 failing tests in test_adr_0175_phase2_practice_lane.py are
PRE-EXISTING reds on clean origin/main: stale 4/0/46 assertions on build_report,
which this change does not touch)
- report.json + ratification_queue.json: deterministic (run1==run2) and
reproduced byte-identically by both `python -m ...runner` and `...propose_runner`
- pnpm build green; 144 UI tests across calibration/leeway/evidence/replay/
doctrine-gates/routes-docs-drift all pass
First Wave M / Phase B piece (GATING): read-only backend that makes the
calibrated-learning / serving-discipline loop inspectable — 'the engine
earns the right to guess', ADR-0175.
The workbench computes NONE of these numbers:
- GET /calibration/classes — per-class gold-tether view from the persisted
practice arena ledger (evals/gsm8k_math/practice/v1/report.json per_class).
Each class's reliability_floor is the engine's own one-sided Wilson
conservative_floor (via ClassTally.reliability); PROPOSE (θ=0.85) / SERVE
(θ=0.99) license verdicts come from core.reliability_gate.license_for.
Failures-first ordering. A test proves the reader's floor equals a direct
conservative_floor() call — no re-implementation.
- GET /serving/metrics — the live correct/refused/wrong counts read unchanged
from the committed train_sample + holdout_dev report.json (currently
4/46/0 and 5/495/0 — wrong=0). Never re-runs a lane.
Honest current state: the committed practice ledger's three classes
(additive/divisive/multiplicative) are all below N_MIN=10, so none has
earned a license yet — the reader shows exactly that, no fake green light.
- workbench/calibration.py: pure readers; imports core.reliability_gate;
EvidenceUnavailableError -> 501 (fail-closed) when the artifact is absent.
- schemas + TS mirrors (CalibrationClass, ServingMetrics); both snapshots
regenerated (deterministic); both drift gates pass.
- trust boundary: read-only over committed artifacts + engine-owned
derivation; no execution, no mutation, no license ever changed.
Verified: 30 Python tests (incl. the no-reimplementation proof + fail-closed),
390 vitest, both schema drift gates, snapshots deterministic.
First R3 (Theater) piece. Reworks the stale artifact-keyed ReplayRoute into
the turn-keyed hero over the #716 sealed-replay backend, and fully retires
the dead W-026 artifact-keyed machinery on both sides (the NOT_YET_MIRRORED
comment anticipated this).
The hero makes determinism *felt*: pick a journaled turn, CORE re-executes
its prompt in a sealed fresh runtime, and the result renders as a hash
verdict (≡ equivalent / ≠ diverged) + the original/replay DigestBadges + a
leaf diff (critical weighted above informational, each with a ≠ glyph). The
'What this proves' card surfaces comparison_basis + origin_state and states
the honest limit: a divergence means nondeterminism OR origin-state
influence, never rendered on its own as a determinism failure.
Retirement (verified zero serving uses):
- Python schemas.py: removed ReplayComparison/ReplayDivergence/ReplayStatus/
ReplayDivergenceSeverity; scripts/dump-enums.py drops the two replay enums;
both snapshots regenerated (deterministic: dump == committed)
- TS: removed ReplayComparison/ReplayDivergence/ReplayDivergenceSeverity;
added TurnReplayComparison/TurnReplayDivergence (+ basis/origin/severity
unions); NOT_YET_MIRRORED now empty (every engine schema mirrored)
- badges: removed ReplayStatusBadge + ReplayDivergenceSeverityBadge +
meta/enums + their enumCoverage cases (hero renders severity inline)
- api: fetchReplayComparison/useReplayComparison -> fetchTurnReplay/
useTurnReplay (/replay/<turnId>)
- deleted ArtifactList, ReplayComparisonPanel, ReplayDiffViewer,
ReplayMetadataTable, old replay.test.tsx
- App route /replay/:artifactId? -> /replay/:turnId?; conformance row
turn-keyed (loading 'Loading turns...', empty 'No turns recorded yet.')
Tests: ReplayRoute.test.tsx (equivalence hero + honesty card, diverged
critical leaf, informational-only label, replace-mode select, j/k spine).
Full vitest 358 green; workbench Python 34 green; both snapshots deterministic.
Follow-up flagged (not in this PR to keep it focused): the artifact query
hooks (useArtifacts/useArtifactDetail/fetchArtifacts*) are now orphaned but
entangled with the still-live artifact EvidenceSubject kind — a separate
cleanup once an Artifacts route or that subject's fate is decided.
Brief: docs/handoff/wave-R3-briefs-2026-06-13.md (all four R3 pieces).
Replaces the W-026 501 stub. Re-executes a journaled prompt in a sealed
fresh runtime (ChatRuntime(no_load_state=True): no checkpoint load, no
checkpoint write, no proposal lineage) and compares the envelope
leaf-by-leaf against the recorded TurnJournalEntry.
Design correction vs the scoping doc (amended in-doc): journaled turns
each ran in a fresh ChatRuntime(), never one continuous session, so
genesis-PREFIX replay would manufacture spurious divergence; shipped
basis is sealed_fresh_runtime_single_turn (O(1)). origin_state:
"unrecorded" — the journal does not record whether the original turn's
runtime loaded a checkpoint, so divergence is reported as nondeterminism
OR origin-state influence, never disambiguated.
- workbench/replay.py: pure comparison + injected executor; every
TurnJournalEntry field classified critical/informational exactly once
(exhaustiveness enforced by test)
- api.py: route wiring under _CHAT_TURN_LOCK; runtime failure -> 500
runtime_unavailable, no comparison may be fabricated
- schemas: additive TurnReplayComparison/TurnReplayDivergence (W-026
artifact-keyed pair retires with the frontend Replay Moment PR)
- tests: 10 obligations incl. tamper-prompt, tamper-leaf precision,
no-execution-no-comparison, no-trace (journal + engine_state bytes),
wall-clock tolerance, sealed-construction proof
- snapshot regenerated; NOT_YET_MIRRORED debt entries for the two new
classes (mirrors land with the frontend PR)
- api-contract-v1.md § Replay rewritten for the turn-keyed shape
Append-only JSONL journal records the exact ChatTurnResult envelope returned by /chat/turn with stable turn_id, trace_hash, all three surfaces, verdicts, and deterministic journal_digest.
GET /trace/turns and GET /trace/{turn_id} serve journal evidence for the Trace route frontend. Read model only; no teaching, pack, or journal-owned engine_state mutation.
Bundles three post-Tier-1 follow-ups into one PR (no scope change, no
new ADR — implementation tightening on the already-shipped corridor).
(1) Standalone JSONL self-containment
teaching/math_contemplation_proposal.py
+ to_jsonl_record() — emits proposal_id + full evidence_pointers
(nested dicts including audit_row) + full reasoning_trace.steps
+ from_jsonl_record() — inverse; goes through build_proposal()
so all invariants are re-validated; raises on proposal_id mismatch
canonical_bytes() UNCHANGED (still the content-hash function;
trace_id/proposal_id stability preserved)
core/cli.py W3 lane now writes to_jsonl_record() output instead of
canonical_bytes() — same compact-JSON encoding (sort_keys=True,
ensure_ascii=False, separators=(",", ":"))
workbench/readers.py loads via self-contained record fields directly;
decompose_audit() re-run removed. read_math_proposal() now reads
reasoning_trace.steps and evidence_pointers from the JSONL record.
(2) Widened change_kind heuristic dispatch
teaching/math_contemplation.py
+ _CHANGE_KIND_BY_PAIR table on (refusal_reason, missing_operator):
(unexpected_category, pre_frame_filler_sentence) → matcher_extension
(unexpected_category, multi_subject_sentence) → frame_reclassification
(unexpected_category, fraction_percentage_literal) → matcher_extension
(unexpected_category, descriptive_frame_question) → frame_reclassification
(unresolved_pronoun, pronoun_resolution) → matcher_extension
Single-key fallback (lexicon_entry/narrowness_violation/
frame_unrecognized) retained for completeness.
hypothesis-step justification text updated to reflect new table.
Result on audit_brief_11.json:
3 matcher_extension (was 0)
2 frame_reclassification (was 0)
3 injector_sub_shape (was 8)
0 vocabulary_addition (no unknown_word group ≥2 in train sample)
(3) shape_category structural gap
MathReaderRefusalEvidence does not carry shape_category, so the
proposal cannot derive it. All proposals continue to emit
ShapeCategory.UNCATEGORIZED with a structural-gap comment. No
invented values — handler dispatch decision (per ADR-0167-FOLLOWUPS
§1) drives ratification routing today, not shape_category.
Tests
+ W1: 5 new tests (to_jsonl_record self-containment, round-trip,
byte stability, proposal_id mismatch rejection, canonical_bytes
unchanged invariant)
+ W2: 3 new pair-dispatch tests + real-audit change_kind distribution
test + shape_category-uncategorized test
+ W3: 2 new tests (records are self-contained, round-trip via
from_jsonl_record); existing byte-comparison test updated to use
proposal_id ordering instead of canonical_bytes
+ W4: existing 6 tests updated to build JSONL via to_jsonl_record;
+ 1 new decoupling test that drops teaching.math_contemplation from
sys.modules and verifies the workbench still loads + serves detail
Verification
- core eval math-contemplation produces the expected 3/2/3 distribution
- core test --suite teaching -q → 33 passed
- core test --suite runtime -q → 20 passed
- All 57 ADR-0172 W1-W4 tests pass (49 existing + 8 new)
Determinism / invariants preserved
- canonical_bytes() byte-stable (test pins this)
- to_jsonl_record() byte-stable via sort_keys=True + no floats
- wrong=0 invariant: proposals stay evidence-only; no auto-apply
- ChangeKind Literal unchanged (4 values; no new ones invented)
Wires teaching/math_proposals/proposals.jsonl into the CORE Workbench
API (ADR-0160) alongside the existing cognition proposal queue:
workbench/schemas.py
- MathReasoningStep, MathProposalSummary, MathProposalDetail,
MathRatifyResult schemas
workbench/readers.py
- MATH_PROPOSALS_JSONL + _DEFAULT_MATH_AUDIT_PATH constants
- teaching/math_proposals added to ALLOWED_ARTIFACT_ROOTS
- _HANDLER_DISPATCH table (vocabulary_addition→LexicalClaim; all
others not yet implemented)
- list_math_proposals(), read_math_proposal(), ratify_math_proposal()
- read_math_proposal() re-runs decompose_audit() to recover full
4-step reasoning trace (canonical_bytes only carries trace_id)
- ratify_math_proposal() raises NotImplementedError with clear
"handler not yet implemented: {change_kind}" for unhandled kinds
workbench/api.py
- GET /math-proposals, GET /math-proposals/{id}
- POST /math-proposals/{id}/ratify → _math_ratify()
(vocabulary_addition→200/routed; unhandled→501 with loud message)
tests/test_adr_0172_w4_workbench_e2e.py — 6 tests:
1. loads from JSONL
2. renders domain:math badge (distinct from cognition /proposals)
3. ratify-vocabulary_addition routes to LexicalClaim (200)
4. ratify-matcher_extension fails loudly (501 "handler not yet
implemented")
5. all 4 trace steps visible in detail response
6. no cross-contamination between math and cognition queues
teaching + runtime suites green (28 + 20 passed).
Brief-gap note: canonical_bytes() excludes proposal_id and serialises
evidence pointers as hashes only. D1 loader derives proposal_id via
sha256(line_bytes) and re-runs decompose_audit() to recover full trace
for read_math_proposal(). This works but means the JSONL cannot be
loaded without the original audit file. If a future wave needs
standalone JSONL loading, C1 should emit a richer format.
