`core bench --suite teaching-loop [--runs N]` runs the full reviewed-
corpus extension pipeline (propose → real replay-equivalence gate →
operator accept) N times against an identical input and asserts
byte-identical artifacts every run:
- proposal_id (SHA-256 of canonical-JSON payload)
- replay_baseline (cognition lane metrics on active corpus)
- replay_candidate (cognition lane metrics on transient corpus)
- regressed_metrics (sorted tuple)
- chain_id_written
Also reports per-iteration latency (mean / p50 / p95) and total wall.
100-run result against today's main:
unique(proposal_id)=1 unique(baseline)=1 unique(candidate)=1
unique(chain_id)=1 active_corpus_byte_eq=True
mean=1.849s p50=1.838s p95=1.851s
The full learning loop is replayable bit-identically across N
independent invocations. Pairs naturally with ADR-0045's 100% exact-
NIAH recall numbers — same epistemic class of guarantee, applied to
the *learning loop* itself rather than only to retrieval. No LLM
provider can publish equivalent numbers on a learning path.
- benchmarks/teaching_loop.py — `run_teaching_loop_determinism(runs)`
returns a typed `TeachingLoopBenchReport` with uniqueness counts,
determinism flag, byte-identical-active-corpus flag, and latency
distribution (mean / p50 / p95 / total). Pure-stdlib percentile —
no numpy dep on this path.
- benchmarks/run_benchmarks.py — `bench_teaching_loop_determinism`
shim + `_SUITES["teaching-loop"]` registration + runs= passthrough.
- core/cli.py — `--suite teaching-loop` choice added to bench parser.
- tests/test_teaching_loop_bench.py — 5 tests pin determinism at
small N, proposal_id SHA-256 shape, canonical chain_id layout,
latency stats well-formedness, JSON serialisation.
Trust boundary: every write is confined to a tempdir created inside
the bench loop; the active corpus is read once at start, once at end,
and any byte difference would fail the bench.
82dac4b16f