Umbrella ADR for Phase 5 of ADR-0114. Decomposes the eval-lane work
into eight sub-phases (5.1..5.8) that ship under their own ADRs.
Sub-phase scope:
5.1 — sealed-holdout encryption (one lane, fabrication_control)
5.2 — CORE-original GSM8K-style corpus (dev + public, 200 cases)
5.3 — lane runner (parser → solver → verifier → realizer)
5.4 — frontier-baseline comparison (Obligation #7)
5.5 — adversarial generation; misparse rate zero (Obligation #8)
5.6 — depth-curve measurement harness (Obligation #6, measurement)
5.7 — sealed GSM8K test as the holdout (Obligation #1, lane-side)
5.8 — overall lane gate; new gsm8k_capability_shape
Roadmap only — no code lands with this ADR. Each sub-phase ships
independently and discharges a specific ADR-0114a obligation.
Critical invariant: dev + public splits are CORE-original; the real
GSM8K test set enters only via the encrypted holdout under 5.7.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 4 of the ADR-0114 GSM8K-math roadmap. Consumes a SolutionTrace
and emits one sentence per step plus setup + answer sentences. Pure
function; same trace → byte-equal RealizedTrace.
What landed
generate/math_realizer.py
- realize(initial_state, trace) -> RealizedTrace
- Frozen RealizedTrace dataclass with canonical_bytes() + as_prose()
- Per-kind sentence rules (add / subtract / transfer / multiply×2 /
multiply×3 / multiply-general / divide)
- Singular/plural surface rule matches parser canonicalization
- Typed RealizerError on unrecognized step kinds
tests/test_math_realizer.py — 60 cases pinning five invariants:
1. All 50 dev-set cases realize without error
2. Determinism: byte-equal RealizedTrace across two calls
3. Setup sentence count == initial_state count
4. Step sentence count == operation count
5. Answer sentence contains the resolved value + unit
ADR-0114a obligation discharge update
ADR-0118 hardens determinism (#9) across a third layer (realizer)
and makes #3 / #10 human-inspectable via the prose surface. No
obligation is directly newly discharged by ADR-0118; it's substrate
for ADR-0119 GSM8K eval lane.
Round-trippability of the prose through the parser is explicitly
out of scope for this phase. The trace is the verifiable artifact
(ADR-0117); the prose is human-readable documentation.
Tests: 60 new realizer cases; 546 total green across realizer +
parser + solver + verifier + OOD; 67/67 smoke green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 3 of the ADR-0114 expert-capability roadmap. Re-applies every
step of a SolutionTrace from the input graph's initial state and
asserts byte-equal reproduction of answer_value. Pure function; same
(graph, trace) → byte-equal VerifierVerdict.
Why this is distinct from the solver
ADR-0116's solver enforces correctness at construction. ADR-0117's
verifier is a SECOND, INDEPENDENT implementation that re-derives
every value the trace claims. The verifier does NOT call solve(). It
re-implements the operation semantics from ADR-0116 directly inside
_verify_step. If the solver had a bug or was tampered with after the
fact, the verifier catches it.
Six checks per verdict (named, ordered, audit-logged):
1. graph_canonical_hash_matches
2. pack_id_matches
3. pack_lemmas_resolve
4. step_pack_lemma_ids_match_bindings
5. step_replay_matches_before_after
6. answer_value_reproduces
Seven named tamper classes all caught:
- mutated before_value / after_value / operand of any step
- mutated pack_lemma_id of any step
- mutated graph_canonical_hash
- mutated answer_value
- mutated pack_id
- mutated target_before / target_after of transfer step
ADR-0114a obligation update
#3 Replay-equal trace — now discharged at VERIFIER FIDELITY
(was solver-only under ADR-0116). A third party with only
(graph, trace, pack) can reproduce the answer byte-equal.
Five of ten obligations now load-bearing: #3, #4, #9, #10 plus
in-flight #2 (Codex's ADR-0118a OOD generator).
Tests: 62/62 verifier suite green; 67/67 smoke green; existing
solver + parser + schema suites unaffected.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 2 of the ADR-0114 expert-capability roadmap. Consumes the
MathProblemGraph from Phase 1 and emits a SolutionTrace — ordered
operation applications ending at a numeric answer, byte-deterministic
across runs, with each step's operation bound to a pack-resolved
lemma identifier.
What landed
generate/math_solver.py
- solve(graph) -> SolutionTrace; pure function, no I/O, no globals
- SolutionStep dataclass with before/after values per step (for
verifier replay; ADR-0117 hardens)
- SolutionTrace with canonical_bytes() byte-deterministic JSON
- SolveError typed refusal: missing pack, division by zero,
unknown-references-nothing
language_packs/data/en_arithmetic_v1/
- 5 operator lemmas: add / subtract / multiply / divide / transfer
- role=operational_base (vocabulary-only; no domain claim)
- SHA-256-anchored lexicon + glosses; manifest carries
provenance=adr-0116:operator_seed:2026-05-22
tests/test_math_solver.py — 109 cases pinning five invariants:
1. Phase 2 exit criterion: ≥ 0.80 on parser-correct dev set
(current: 50/50 = 1.00)
2. Determinism: two solves produce byte-equal trace
3. Trace replay reproduces answer_value (verifier rehearsal)
4. Typed refusal on under-determined inputs
5. Every step.pack_lemma_id resolves to a real lexicon entry
in en_arithmetic_v1
ADR-0114a obligation discharge
Four of ten anti-overfitting obligations now have load-bearing
implementations in code:
#3 replay-equal trace — discharged (solver-layer)
#4 typed refusal — discharged (solver-layer)
#9 determinism — discharged (solver-layer)
#10 operation provenance via pack — DISCHARGED IN FULL
Removing the en_arithmetic_v1 pack now breaks every solve loudly.
The "operations bind to concepts, not hardcoded strings" claim is
architecturally true, not rhetorical.
Tests: 109/109 green on solver suite; 67/67 smoke suite green;
parser + schema suites still green from prior phases.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Documentation-only amendment to ADR-0114. Locks in the 10-point
falsifiable proof framework that any future `expert` ledger-tier
promotion (ADR-0120+) MUST require.
The obligations:
1. Sealed-holdout discipline
2. OOD surface variation ≥ 0.95 of public
3. Every correct answer ships with replay-equal trace
4. Refusal is first-class; misparse rate zero; zero `wrong` answers
5. Reasoning-isolation perturbation suite (invariance + predictable change)
6. Compositional-depth curve flat within documented ε
7. Frontier-baseline comparison on identical items, published
8. Adversarial generation; misparse rate zero
9. Determinism across release boundaries
10. Operation provenance via the pack (not hardcoded strings)
Each obligation is load-bearing and falsifiable: a domain that
cannot satisfy any one stays at `audit-passed`. ADR-0114a binds
ADR-0116..ADR-0120 to carry the obligations into implementation;
ADR-0120 finally invokes all ten as hard gates.
The audit-passed tier (ADR-0106/0109/0113) is unaffected. The two
tiers measure orthogonal properties: audit-passed verifies CORE
claim-shape compliance (transformer-unreachable invariants); expert
verifies capability with anti-overfitting proof.
No code change. Pure forward contract for the next phase of work.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
First Phase of ADR-0114's expert-capability roadmap. Decomposed into four
sub-phases so each lands as its own auditable step:
1.1 schema + 5 seed cases + invariants ← this commit
1.2 45 more dev-set cases ← delegated (Codex)
1.3 the parser itself ← exit: ≥0.90 on dev set
1.4 runtime binding ← if non-trivial
What landed
- generate/math_problem_graph.py — typed dataclasses (Quantity,
InitialPossession, Operation, Unknown, MathProblemGraph) + frozen
validation + canonical_bytes() byte-deterministic serialization +
graph_from_dict roundtrip.
- evals/gsm8k_parser_dev/cases.jsonl — 5 seed cases (gpd-001..005)
covering single-add, single-subtract, multi-step, two-entity
transfer, and multi-entity sum constructions. Every case carries a
ground_truth_graph and the documented patterns it exercises.
- evals/gsm8k_parser_dev/README.md — authoring contract: schema,
pattern registry, canonicalization rules, Phase 1.1 scope boundary,
hand-solving rubric, distribution target for the remaining 45
cases. This is the spec Phase 1.2 authors work against.
- tests/test_math_problem_graph.py — 26 cases pinning four invariants:
round-trip byte equality, canonical_bytes() determinism, schema
rejection of malformed graphs, and ground_truth_graph ↔
expected_answer agreement (a hand-solver inside the test module
falsifies mis-authored cases).
Why this is sticky
The Phase 1.1 schema is load-bearing for Phase 1.2 (the 45 authored
cases will be written against it) AND Phase 1.3 (the parser will be
graded byte-equal against ground-truth graphs in this schema). Changing
the schema after Phase 1.2 lands requires an amendment ADR + rewriting
authored cases. The schema choices here are intentionally conservative.
Tests: 26/26 new; 67/67 smoke green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The word "expert" in the previous status name implied raw-capability parity
with frontier LLMs on the same benchmark — which the gate does NOT verify.
What the gate actually verifies is CORE *claim-shape compliance*:
* signed digest (replay-reproducible from on-disk lane results)
* replay determinism (same inputs → byte-equal trace_hash)
* typed refusal (fabrication refused, not paraphrased)
* exact recall (no ANN, no cosine, no attention bottleneck)
* grounding-source provenance
These are claim shapes a transformer LLM cannot structurally produce
regardless of raw accuracy. A frontier LLM might score higher on the
same benchmark but cannot pass this contract.
Rename scope (semantics only, per ADR-0113):
status string "expert-demo" → "audit-passed"
predicate key predicates.expert_demo → predicates.audit_passed
reason key expert_demo_reason → audit_passed_reason
YAML key expert_demo_claims → audit_passed_claims
CLI command core demo expert → core demo audit-passed
output dir evals/expert_demos/ → evals/audit_passed/
artifact filenames expert_demo.{json,html} → audit_passed.{json,html}
HTML title CORE Expert-Demo: X → CORE Audit-Passed: X
Internal Python identifiers (module/file/function/class names like
`expert_demo.py`, `evaluate_expert_demo`, `ExpertDemoClaim`,
`expert_demo_claim_for`) are deliberately kept to minimize churn. ADR
file titles (ADR-0106..0112) preserved as historical record.
`expert` namespace reserved for ADR-0114+: an actual capability tier
above `audit-passed` backed by a public benchmark with a stated
threshold. ADR-0114 proposes the first such target — GSM8K-math —
laying out a falsifiable 7-phase arc (parser → solver → verifier →
stepped-realizer → eval lane → first `expert` ledger tier promotion).
Tests: 184 directly-affected tests green (140 capability/expert-demo
suite + 34 demo/audit-tour + 10 correction-cue). Smoke suite 67/67.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes the asymmetry between the `expert-demo` ledger status (audit
artifact only) and the actual `core demo` surface (runnable
walkthroughs producing HTML + JSON). Until this commit the word
"demo" in `expert-demo` was aspirational; now it corresponds to
something a reader can open.
What it does
- Reads the signed expert_demo_claims entry from docs/reviewers.yaml
- Loads latest on-disk result files for each attached lane × split
- Re-derives the evidence-bundle digest and asserts byte-for-byte
match against the signed claim_digest — this is the load-bearing
audit step, now exercised at two independent enforcement points
(ledger gate + showcase)
- Runs each lane's metrics through the ADR-0109 lane-shape registry
and surfaces the verdict
- Picks the first three cases from each split verbatim (deterministic
by file order) and renders them as HTML for inspection
- Emits expert_demo.json (canonical bytes, deterministic) + expert_demo.html
Surface
core demo expert --domain mathematics_logic
core demo expert --domain physics
# → evals/expert_demos/<domain>/latest/expert_demo.{json,html}
Read-only by construction: cannot mutate docs/reviewers.yaml or any
lane result file. Tested. Unpromoted domains raise ValueError —
no silent fallback, no "preview" mode that fakes a showcase.
Generated artifacts are gitignored — the inputs they derive from are
already committed, so duplicating the renders would just churn the
tree.
Tests: 16 new cases pinning all five ADR-0112 invariants. Smoke suite
still 67/67 green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Second worked promotion exercising the ADR-0106 + ADR-0109 contract
on a domain distinct from mathematics_logic. No contract change.
Evidence:
- foundational_physics_ood: accuracy=1.0 (117/117 public, 39/39 holdout)
- inference_closure: all_pass_rate=1.0 (shared with math, distinct digest via domain_id)
- fabrication_control: refused=n, fabricated=0 across all classes (shared)
Signed claim digest: a104cad136f3219df05dc7ce6a78437c02f7b5827cd3cdce568db3acda6a43ed
Bridge landed: cases_plaintext.jsonl dev-mode fallback for
foundational_physics_ood (matches ADR-0105 convention; analogous to the
math/inference bridges in ADR-0110). One small file, not a contract change.
Tests:
- tests/test_adr_0111_physics_expert_demo.py — 4 invariants, 6 cases
- tests/test_adr_0110_math_expert_demo.py — relaxed "only math promoted"
to "math stays promoted" (load-bearing for ADR-0110 is persistence)
- tests/test_capability_reports.py — physics row now expert-demo
Retires the "first promotion was math-specific" objection: the bridges
ADR-0110 landed were correctly scoped, and the contract holds across
two distinct domains using shared lane infrastructure with distinct
digests.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- README.md: fix broken evals/CLAIMS.md link to root CLAIMS.md;
add 'Evidence-Governed Domain Layer' section pointing to the
ADR-0091/0096/0106/0109 contract chain and current ledger state
(math at expert-demo, four other domains at reasoning-capable).
- docs/decisions/README.md: extend the 'fully accepted' slate from
0091..0105 to 0091..0110 with ADR-0106 through 0110 entries; extend
the 'Evidence-governed domain chain' chain-notes section to reflect
the expert-demo arc (refused-amended-succeeded narrative).
Replaces the cognition-shape-uniform threshold dispatch in
core/capability/expert_demo.py with an explicit LANE_SHAPE_REGISTRY
mapping 8 ratified lane ids to 5 shapes:
cognition -> cognition_shape
elementary_math_ood -> accuracy_shape
foundational_physics_ood -> accuracy_shape
symbolic_logic -> symbolic_logic_shape
hebrew_fluency -> accuracy_shape
koine_greek_fluency -> accuracy_shape
inference_closure -> inference_shape
fabrication_control -> refusal_shape
Each shape has a documented threshold checker. Unknown lane ids
fail-closed with a named reason. ADR-0106 \xc2\xa71.1/\xc2\xa71.3/\xc2\xa71.4/\xc2\xa71.5
unchanged; only \xc2\xa71.2 (threshold rules) dispatches by shape.
tests/test_lane_shape_thresholds.py pins all four ADR-0109 invariants
plus dead-shape and threshold-value gates (13 new tests).
tests/test_expert_demo_contract.py fixtures updated to provide
shape-appropriate metrics (no semantic change to those tests; same
12 cases still pin the ADR-0106 contract).
ADR-0109 status: Proposed -> Accepted. README sequencing updated
(ADR-0110 now only blocked by inference_closure, not by metric-shape
amendment).
Ledger: all five domains remain reasoning-capable, expert_demo=false.
Amends ADR-0106 \xc2\xa71.2 to dispatch threshold rules by lane shape rather
than imposing cognition-pack-shape metrics uniformly. ADR-0107
surfaced that every non-cognition lane was failing the gate by
absence-of-key, not by substance.
Status: Proposed. Ships five shapes covering every lane currently
attached to a ratified pack: cognition_shape, accuracy_shape,
inference_shape, refusal_shape, symbolic_logic_shape. Four invariants
pinned. Unknown lanes fail closed; new shapes require ADR amendment.
\xc2\xa71.1 (reasoning-capable prereq), \xc2\xa71.3 (signature scoping), \xc2\xa71.4
(domain-aware), \xc2\xa71.5 (replay byte-equality) all preserved. ADR-0106
status remains Accepted.
The ADR-0106 contract correctly refused promotion. ADR-0107 records the
deferral and reserves two follow-up ADRs:
- ADR-0109 (lane-shape-aware threshold amendment): ADR-0106 \xc2\xa71.2
prescribes cognition-pack-shape metrics uniformly, but math /
physics / systems / hebrew-greek lanes carry native shapes
(accuracy, passed_rate, all_pass_rate). Prerequisite for any future
expert-demo promotion.
- ADR-0110 (math re-attempt): conditional on ADR-0109 landing and
inference_closure substantively passing (currently all_pass_rate=0.4
on public).
tests/test_adr_0107_deferral.py pins adr_0107_no_silent_promotion: math
stays at reasoning-capable, has no expert_demo_claims entry, and the
ledger row carries a named refusal reason.
No change to core/capability/expert_demo.py or reporting.py -- the
contract is honored, not amended. README sequencing updated to reflect
ADR-0107 acceptance and the new ADR-0109/0110 prerequisites.
Closes ADR-0106 acceptance evidence:
- ExpertDemoClaim dataclass + additive expert_demo_claims block on
ReviewerRegistry (schema_version stays at 1; backward-compatible).
- New core/capability/expert_demo.py with derive_evidence_digest,
evaluate_expert_demo, collect_domain_lanes, materialise_lane_results.
- core/capability/reporting.py: replaces the cognition-lane-only
predicate (previous lines 418-433) with a domain-aware,
reviewer-signed gate; ledger rows now also carry
expert_demo_reason for operator legibility. Reviewer registry is
fail-closed: an unloadable registry yields zero claims, so a broken
registry never silently grants expert_demo=true.
- tests/test_expert_demo_contract.py covers all three ADR-0106
invariants: requires_signature, domain_aware, replay_byte_equality;
plus threshold + production-ledger-untouched gates. 12 new tests.
- tests/test_reviewer_registry.py extended with TestExpertDemoClaimsSchema
covering omitted block, valid parse, unknown signer rejection,
malformed digest rejection, duplicate domain rejection. 5 new tests.
- README index row + table preface updated to note expert_demo is
contract-gated. Frontier list trimmed (ADR-0106 has landed).
- ADR-0106 Status flipped Proposed -> Accepted.
No domain row's expert_demo field flips by this PR -- only the contract
changes. Promotion of any ratified domain requires a follow-up ADR
(ADR-0107 reserved for mathematics_logic) plus a signed claim.
- ADR-0108 Status: Proposed -> Accepted
- README index row updated to Accepted
- 'Current frontier' rewritten with the ranked Proposed-ADR list mandated
by ADR-0108 \xc2\xa7Decision; removes the now-false 'No ADR currently sits in
a "Proposed but unimplemented" state' sentence
- Open candidate directions (no-ADR-yet) section retained for the
multi-reviewer governance frontier item from ADR-0105
Makes the post-ADR-0105 sequencing of ADR-0080 / 0084 / 0087 / 0106
explicit, durable, and revisable. Status: Proposed. No content of the
four sequenced ADRs is modified — sequencing is meta, not content.
Defines a domain-aware, reviewer-signed expert_demo promotion gate to
replace the current cognition-lane-only predicate in
core/capability/reporting.py:418. Status: Proposed. This ADR does not
promote any domain — it defines the contract that a follow-up ADR (likely
mathematics_logic as ADR-0107) will consume as the first worked
promotion.
Sibling reconciliation PR to #104. The four ADRs explicitly called out as
the 'current implementation frontier' in PR #104 are already implemented
to the same evidence bar as the eight ADRs that PR accepted:
- ADR-0094: teaching/source.py + proposal schema widening + migration
script; tests/test_proposal_source.py green
- ADR-0095: teaching/from_miner.py + miner_loop_closure lane;
SHA-pinned in scripts/verify_lane_shas.py; tests/test_miner_proposals.py
green
- ADR-0098: core/demos/contract.py + adapter surface + demo_composition
lane; SHA-pinned; tests/test_demo_composition.py green
- ADR-0099: core/demos/showcase.py + public_demo lane;
SHA-pinned; tests/test_public_showcase.py green
Three of four lanes are SHA-pinned in CI (a stricter bar than several
already-accepted ADRs). Local pytest run: 85/85 passed across the four
tests/test_*.py files in 17s.
Also refreshes docs/decisions/README.md:
- flips the four table rows to Accepted (2026-05-22)
- rewrites the 'Current frontier' section now that no ADR-0091..0102
entry is unimplemented
- enumerates candidate next directions (curriculum proposals,
language-specific holdout splits, expert-demo ratification)
Docs-only change; no runtime code touched.
Two new intent shapes + composers turn the runtime's corpus
density into operator-visible articulation. Both consult the
cross-corpus aggregator from ADR-0064; no new ratification needed.
P3.3 — chat/narrative_surface.py + IntentTag.NARRATIVE.
Classifier patterns (registered BEFORE generic DEFINITION):
^tell\s+me\s+about\s+
^describe\s+
^what\s+(?:can|do)\s+you\s+(?:say|know)\s+about\s+
narrative_grounded_surface(subject, max_clauses=4) walks every
reviewed chain rooted on subject across all registered teaching
corpora. Dedupes by (connective, object) — cause + verification
carrying the same predicate emit one clause, not two. Sorts by
(intent, connective, object) for replay stability.
Surface format:
"{X} — narrative-grounded ({corpus_ids}): {dX1}; {dX2}.
{X} {conn1} {O1} ({dO1}); {X} {conn2} {O2} ({dO2}).
No session evidence yet."
Cross-corpus subjects (e.g. mother in relations_v2) emit
narrative-grounded (relations_chains_v2) tag; cognition subjects
emit cognition_chains_v1 tag. Multi-corpus subjects (when
applicable) emit composite "corpus_a + corpus_b" tag.
P3.4 — chat/example_surface.py + IntentTag.EXAMPLE.
Classifier patterns:
^(?:give|show)\s+(?:me\s+)?an?\s+(?:example|instance)\s+of\s+
^example\s+of\s+
example_grounded_surface(object_lemma, max_examples=3) walks chains
where the lemma is the OBJECT — inverts the typical subject-keyed
access pattern. Dedupes by subject; sorts by (intent, subject,
connective).
Surface format:
"{X} — example-grounded ({corpus_ids}): {dX1}.
Example: {subj1} {conn1} {X}; {subj2} {conn2} {X}.
No session evidence yet."
Cross-cutting:
- Both intents added to _OOV_INTENT_TAGS — fall through to OOV
invitation when subject is unknown (Phase 2 gradient discipline).
- Both tagged grounding_source="teaching" (same provenance tier
as the existing teaching_grounded_surface).
- No prose generation, no new mutation surface.
Live verification:
> Tell me about truth.
[teaching] truth — narrative-grounded (cognition_chains_v1):
cognition.truth; logos.core. truth grounds knowledge
(cognition.knowledge); truth requires evidence (cognition.evidence).
> Give me an example of knowledge.
[teaching] knowledge — example-grounded (cognition_chains_v1):
cognition.knowledge. Example: truth grounds knowledge;
understanding requires knowledge; evidence grounds knowledge.
> Tell me about mother.
[teaching] mother — narrative-grounded (relations_chains_v2):
kinship.parent.female. mother precedes daughter (kinship.child.female).
> Describe photosynthesis.
[oov] I haven't learned 'photosynthesis' yet (intent: narrative). ...
ADR-0066 (this commit completes the ADR). 30 new tests passed.
Full lane: 2067 passed, 2 skipped, 0 failed in 2:32.
Mirrors the chain-gap pipeline (Phase 1.1+1.2) for vocabulary gaps:
the OOV invitation surface (P2.1) now emits structured signals that
operators can aggregate, rank, and auto-promote into reviewed
PackMutationProposal candidates — closing the OOV loop the same way
Phase 1 closed the chain loop.
Three new modules + two new CLI surfaces:
teaching/oov_sink.py.
OOVCandidate dataclass mirroring teaching.discovery.DiscoveryCandidate.
OOVBufferSink (in-memory) + OOVMonthlyFileSink (append-only JSONL
under <root>/<YYYY>/<YYYY-MM>.jsonl — same layout as discovery sink
so the aggregator reuses the file-walk machinery).
hash_oov_candidate_id(token, intent, trace_hash) — deterministic
32-char hex id matching DiscoveryCandidate's replay invariant.
format_oov_candidate_jsonl — sorted-keys compact JSONL line.
teaching/oov_gaps.py.
aggregate_oov_gaps(root, since, sample_limit) groups emitted
candidates by token, tracks intent-shape union (a token asked under
multiple intents is a stronger curriculum signal), splits
boundary_clean from boundary_tainted counts, supports --since
YYYY-MM filtering via the sink's file naming convention.
Pure reader; never mutates the sink. Deterministic ordering:
(count desc, token asc).
teaching/oov_promotion.py.
promote_oov_gaps(gaps, threshold, include_tainted, suggested_packs)
lifts threshold-crossing tokens to OOVPromotion records.
- boundary_clean_count gates promotion by default (tainted-only
tokens may indicate the prompt hit a safety axis rather than a
vocab gap).
- --include-tainted flag for operator override.
- threshold < 1 raises.
- queue_id deterministic: ``oov:<token>@<threshold>`` — diffable
across runs.
- suggested_packs lists mounted packs but does NOT recommend one
— domain inference is out of scope (would require a stochastic
classifier). Operator picks the destination.
Runtime wiring:
ChatRuntime.attach_oov_sink(sink) mirrors attach_discovery_sink.
Runtime emits one OOVCandidate JSONL line per turn whose
grounding_source == "oov", no-op when no sink is attached.
Intent classifier is now invoked when EITHER sink is attached
(was: only discovery sink) — both downstream paths need it.
CLI:
core teaching oov-gaps [--top N] [--since YYYY-MM] [--root PATH]
[--sample-limit N] [--json]
core teaching oov-queue [--threshold N] [--include-tainted]
[--root PATH] [--since YYYY-MM] [--json]
ADR-0065 documents the full design (five-tier honesty gradient,
P2.1-P2.4 architecture). README.md updated with the ADR-0065
index entry.
Verification:
tests/test_oov_pipeline.py 24 passed
Operator workflow round-trip verified live:
> rt.attach_oov_sink(sink); rt.chat("What is photosynthesis?")
→ sink receives:
{"boundary_clean":true,"candidate_id":"f51bf8...",
"intent":"definition","token":"photosynthesis","trigger":"unresolved_subject",
"source_turn_trace":"","review_state":"unreviewed"}
> core teaching oov-gaps --root /tmp/oov_demo
→ ranked table by count, intent-set per token
> core teaching oov-queue --root /tmp/oov_demo --threshold 2
→ promoted tokens + suggested mounted packs
Full lane: 2005 passed, 2 skipped, 0 failed in 2:34 (xdist).
ADR-0064 is the corpus-layer sibling of ADR-0063. The teaching-grounded
surface composer was hardcoded to cognition_chains_v1, so kinship CAUSE/
VERIFICATION prompts fell through to the universal disclosure even though
en_core_relations_v1 was mounted on the live runtime (ADR-0063).
Architectural change in chat/teaching_grounding.py:
- New TeachingCorpusSpec dataclass (corpus_id, path, pack_id).
- TEACHING_CORPORA tuple registers every active corpus. Each
corpus is 1:1-bound to one lexicon pack — cross-domain triples
deferred per docs/teaching_order.md §5.
- _load_corpus(spec) loads one corpus with pack-residency scoped
to its declared pack.
- _all_chains_index() aggregates across all registered corpora
(first-match-wins; cognition first preserves byte-identity).
- _pack_for_corpus(corpus_id) → bound pack lexicon.
- clear_teaching_caches() atomic cache invalidation.
- TeachingChain gains corpus_id field → surface tag follows resolving corpus.
Wiring updates:
- teaching_grounded_surface + teaching_grounded_surface_composed
consult _all_chains_index; surface tag follows chain.corpus_id.
- teaching/discovery.py gate uses chat.pack_resolver.is_resolvable
(any mounted pack) + _all_chains_index (any registered corpus).
- teaching/replay.py _swap_corpus_path rewrites the registry path
+ clears all teaching caches during the gate's transient phase.
Active corpus bytes unchanged (replay invariant preserved).
- evals/learning_loop/run_demo.py scene-5 swap mirrors the new
pattern so the demo still grounds against transient corpora.
Back-compat preserved: _corpus_index, _CORPUS_PATH, TEACHING_CORPUS_ID
remain cognition-corpus-specific for audit/replay consumers.
Phase 1.4 — relations_chains_v1 seeded with 7 reviewed kinship chains:
cause_parent_precedes_child
cause_child_follows_parent
cause_ancestor_precedes_descendant
cause_descendant_follows_ancestor
cause_family_grounds_parent
verification_child_requires_parent
verification_descendant_requires_ancestor
5 of 8 relations lemmas covered. All connectives already humanised.
Strict pack-internal to en_core_relations_v1 (no cross-domain in v1).
Seed pattern matches cognition_chains_v1's original pre-ADR-0055 seed.
Live verification:
> Why does parent exist?
parent — teaching-grounded (relations_chains_v1):
kinship.ascendant.direct; kinship.parent.
parent precedes child (kinship.descendant.direct).
grounding_source = teaching
Cognition eval byte-identical to pre-ADR baseline:
public: intent 100% / surface 100% / term 91.7% / closure 100%
holdout: intent 100% / surface 100% / term 83.3% / closure 100%
Lanes green: smoke 67 / cognition 121 / teaching 17 / packs 6 /
runtime 19 / algebra 132 / full 1933 passed.
ADR-0063 closes the ADR-0048/0050/0053/0061 hardcoded-cognition-pack
asymmetry. New chat/pack_resolver.py provides resolve_lemma(lemma,
pack_ids) → (resolving_pack_id, semantic_domains) across an ordered
tuple of mounted lexicon packs (first-match-wins, lru_cache per-pack).
Surface composers in chat/pack_grounding.py now consult the resolver
instead of a hardcoded en_core_cognition_v1. en_core_relations_v1
joins RuntimeConfig.input_packs defaults; kinship lemmas now ground
on the live path:
> What is a parent?
parent — pack-grounded (en_core_relations_v1):
kinship.ascendant.direct; kinship.parent; biology.progenitor.
No session evidence yet.
Cross-pack comparison (knowledge × parent) renders composite tag
(en_core_cognition_v1 × en_core_relations_v1). Cognition lane
remains byte-identical: cognition is resolved first and the surface
format for cognition lemmas is unchanged.
Cognition eval (byte-identical to pre-ADR baseline):
public → intent 100% / surface 100% / term 91.7% / closure 100%
holdout → intent 100% / surface 100% / term 83.3% / closure 100%
Curated lanes green: smoke 67 / cognition 121 / teaching 17 /
packs 6 / runtime 19 / algebra 132.
New tests: test_pack_resolver.py (28) + test_cross_pack_grounding.py
(17). test_en_core_relations_v1_pack.py: default-input-packs guard
inverted. test_pack_grounding.py: two stale ADR-0048 tests rewritten
(premises invalidated by ADR-0052/0061; now use fully-out-of-pack
prompts).
chat/teaching_grounding.py UNCHANGED — cognition_chains_v1 corpus
stays cognition-only. Cross-pack teaching corpora are the natural
ADR-0064.
Pre-ADR-0062, the teaching-grounded composer emitted exactly one
reviewed chain per surface — "light reveals truth" — even when the
corpus already contained an immediate follow-up "truth grounds
knowledge". With 21 active chains after curriculum saturation v2,
many grounded prompts had a corpus-ratified follow-up the composer
silently dropped.
ADR-0062 adds the composed composer + an opt-in config flag:
flag OFF (default):
light — teaching-grounded (cognition_chains_v1): cognition.illumination;
logos.core. light reveals truth (cognition.truth). No session evidence yet.
flag ON:
light — teaching-grounded (cognition_chains_v1): cognition.illumination;
logos.core. light reveals truth (cognition.truth), which grounds
knowledge (cognition.knowledge). No session evidence yet.
Follow-up resolution:
- prefer cause; fall back to verification (deterministic preference)
- cycle guard: 1-step cycles (A→B, B→A) blocked
- pack-residency guard: follow-up's object must be pack-resident
- bounded depth: v1 follows exactly one hop
- degrades to single-chain BYTE-IDENTICALLY when no follow-up
survives the guards (drop-in replacement)
Trust-boundary invariants preserved:
- Every visible non-template token is lemma / pack-domain /
humanize_predicate connective / template constant. Only added
template constant: ", which "
- Deterministic: same chains → same surface bytes
- Default-False flag pattern mirrors ADR-0047/0058
- `versor_condition < 1e-6` invariant untouched (surface composition only)
Cognition lane null-drop invariant CI-pinned:
Composed mode emits a strictly LONGER surface (extra follow-up
clause); every expected_term passing flag-OFF must still pass flag-ON.
Asserted in test_cognition_lane_metrics_unchanged_with_composed_flag
for both public and holdout splits. If a future change drops tokens,
the test fails as a deliberate regression.
public flag OFF: intent 100% / surface 100% / term 91.7% / versor 100%
public flag ON : intent 100% / surface 100% / term 91.7% / versor 100% (identical)
holdout flag OFF: intent 100% / surface 100% / term 83.3% / versor 100%
holdout flag ON : intent 100% / surface 100% / term 83.3% / versor 100% (identical)
Live-prompt lift visible on ~12 of 21 active chains; the rest hit
cycle or pack-residency guards. Saturation v2's clusters were
authored partly with composition in mind (thought→meaning→
understanding, inference→evidence→knowledge, etc.).
- core/config.py — `RuntimeConfig.composed_surface: bool = False`
- chat/teaching_grounding.py — `teaching_grounded_surface_composed`
sibling to `teaching_grounded_surface`
- chat/runtime.py — dispatch branch in `_maybe_pack_grounded_surface`
selects composed vs single-chain based on config flag
- tests/test_composed_surface.py — 11 tests pin: function-level
(None on no chain / degrades when no follow-up / two-clause when
follow-up exists / includes intermediate + final domains /
deterministic / cycle guard / trust label preserved); runtime
integration (default single-chain / flag-on composed / frozen
config); cognition-lane null-drop invariant.
Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
composed-surface 11 — all green.
Pre-ADR-0061 every "How do I X?" question fell through to the
universal disclosure even when X was a pack-resident lemma. The
teaching corpus carries CAUSE/VERIFICATION chains only — procedural
knowledge is fundamentally different in kind from propositional
claims and deserves its own ratification path (deliberately out of
scope; a future parallel `procedure_chains_v1.jsonl` schema is
discussed in the ADR's non-goals).
ADR-0061 adds the honest cold-start fallback: ground the topic in
pack semantic_domains and note explicitly that ratified step-by-step
guidance does not exist yet.
Surface format:
"procedure-grounded ({pack_id}): {lemma} ({d1}; {d2}).
Step-by-step guidance for {lemma} is not yet ratified
in this session."
Selector — **last** pack-resident lemma in the verb-phrase subject:
"define a concept" → concept (object beats verb)
"verify a claim" → verify (verb wins when object is OOV)
"correct an error" → correct
"learn this" → learn
"do stuff" → None (falls through to universal disclosure)
Stopwords: only `be` and `have` (dialogue fillers). Procedure verbs
are deliberately NOT stopworded so the verb-as-fallback rule fires
when the object is OOV — keeps surface coverage.
Trust-boundary invariants:
- Every visible non-template token is lemma / pack-domain / template.
- Deterministic: same subject_text → same bytes.
- Returns None for fully-unknown utterances → universal disclosure
fires. Never fabricates surface from nothing (ADR-0053 contract).
- "not yet ratified" trust-label preserved.
Cognition lane lift:
public : intent 100% / surface 100% / term 91.7% / versor 100% (unchanged)
holdout : intent 100% / surface 94.7%→100.0% / term 79.2%→83.3% / versor 100%
Two cases fixed:
- procedure_define_010 ("How do I define a concept?") — surface +
term `concept` now captured.
- procedure_verify_034 ("How do I verify a claim?") — surface only
(case has no expected_terms; the verb fallback grounds it).
Combined effect: holdout `surface_groundedness` closes to 100%; 4 of
5 architectural holdout misses now resolved (this ADR + ADR-0060 +
the supersede from epistemology v1). Remaining 2 are UNKNOWN-intent
cases (unknown_spirit_041, unknown_word_018) — out of scope; deserve
their own ADR with distinct selector semantics.
- chat/pack_grounding.py — `_extract_procedure_topic_lemma` helper +
`pack_grounded_procedure_surface` composer.
- chat/runtime.py — import + dispatch branch for `IntentTag.PROCEDURE`.
- tests/test_procedure_surface.py — 15 tests pin: extraction
(last-wins / verb-by-elimination / be+have skipped / None on empty /
strips punctuation / case-insensitive); surface (contains lemma /
contains domains / pack_id / "not yet ratified" label / None for
no-pack-lemma / deterministic); end-to-end through ChatRuntime.
Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
procedure 15 — all green.
The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this ADR changes surface composition only.
ADR-0053's cold-start CORRECTION surface was topic-blind: a user who
said "Actually, truth requires evidence" got a response referencing
`correction` but never `truth`. The holdout case correction_truth_040
expected `term=['truth']` and missed — one of the architectural gaps
surfaced by the epistemology v1 curriculum unit.
ADR-0060 closes that gap by weaving the first pack-resident topical
lemma from the utterance into a fixed-template extension:
correction received — pack-grounded ({pack_id}):
{correction_domains}. Noted topic: {lemma} ({lemma_domains}).
No prior turn in this session to correct yet.
Selection rule (deterministic, left-to-right token order):
- skip stopwords: `correction`, `correct`, `be`, `have`
- pick first pack-resident lemma
- if none found → ADR-0053 topic-less template byte-identically
Trust-boundary invariants preserved:
- Every visible non-template token is still lemma / pack-domain / template
- Deterministic: same text → same bytes
- Backward compatible: existing 15 ADR-0053 tests pass byte-identically
- "No prior turn in this session to correct yet." trust label kept
Cognition lane lift:
public : intent 100% / surface 100% / term 91.7% / versor 100% (unchanged)
holdout : intent 100% / surface 94.7% / term 75.0%→79.2% / versor 100%
The +4.2pp matches the single-case fix exactly (correction_truth_040).
Remaining 3 holdout misses (procedure_define_010, unknown_spirit_041,
unknown_word_018) are out of scope for this ADR.
- chat/pack_grounding.py — `_extract_correction_topic_lemma` helper +
optional `text` parameter on `pack_grounded_correction_surface`.
- chat/runtime.py — single-line call-site change to pass `text` through.
- tests/test_correction_topic_lemma.py — 14 new tests pin:
extraction (first lemma / skips correction / skips fillers / None on
empty / strips punctuation / case-insensitive); surface (contains
corrected lemma / contains topic domains / degrades to ADR-0053
byte-identically / preserves trust label / deterministic / correct
pack_id); end-to-end (correction_truth_040 emits 'truth' / no-pack-
lemma still grounds).
Why text-level extraction, not intent.subject:
`intent.subject` after ADR-0049 head-noun extraction returns
", truth requires evidence" for the test prompt — the CORRECTION
intent's subject-extractor preserves the post-marker tail. Parsing
the raw text at the surface layer is cleaner; isolates the fix;
doesn't perturb upstream classification logic.
Lanes (regression): smoke 67 / cognition 121 / teaching 17 /
correction tests 29 (15 ADR-0053 backward-compat + 14 ADR-0060 new) —
all green.
The non-negotiable field invariant (versor_condition < 1e-6) is
unaffected: this ADR changes surface composition only.
`ChatRuntime.correct()` propagates a backward perturbation through the
session graph (per session/correction.py): each past turn whose output
versor has non-trivial CGA-alignment with the correction versor is
blended toward it (decayed by graph distance). The forward regen turn
that followed already emitted a TurnEvent — but the backward
perturbation itself was invisible to the telemetry sink.
ADR-0059 closes that gap with a discriminated event line.
- chat/telemetry.py — adds `serialize_correction_event` +
`format_correction_event_jsonl` emitting one JSONL line discriminated
by `"type": "correction"`. Payload: target_turn, records_count,
turns_skipped, turn_idxs_affected, max_delta_norm, mean_delta_norm,
SHA-256 correction_versor_digest, pack ids. No raw versor coordinates.
- chat/runtime.py — `_emit_correction_event` (mirrors
`_emit_turn_event`); called from `correct()` after the graph state
is updated but before the forward regen turn. No-op without sink.
- tests/test_correction_telemetry.py — 7 tests pin: no-op without
sink, emission with sink, payload shape (required keys + types +
ranges), SHA-256 digest shape, trust boundary (no versor
coordinates leaked), determinism (byte-identical lines across
runs), correction event and turn event coexist in the sink.
Trust boundary (per CLAUDE.md):
- Metadata-only: only L2 deltas + SHA-256 digest.
- No implicit wall-clock.
- Deterministic: same CorrectionResult → byte-identical line.
- Sink contract unchanged: `emit(line: str)`.
- `versor_condition < 1e-6` invariant: untouched (telemetry-only).
Verification: smoke 67 / runtime 19 / correction telemetry 7 — green.
ADR-0058 closes the ADR-0047 follow-up question ("should the
forward_graph_constraint flag become default-on or pack-opt-in?")
with the explicit answer: neither, yet.
The ADR-0047 A/B characterisation found that the flag is observably
inert on every public-cognition-lane metric — narrowing which tokens
the walk may visit did not change which surface gets emitted. That
finding scoped ADR-0048..0053, which closed the cognition lane to
100.0% surface_groundedness / 91.7% term_capture_rate via realizer /
surface-assembly work downstream of propagation.
This ADR makes three load-bearing decisions:
1. `forward_graph_constraint` remains opt-in with default `False`.
No identity pack (including precision_first_v1) opts in.
2. No `runtime_preferences` block is added to identity packs; no
path from pack JSON to RuntimeConfig is opened. Deferring the
pack-to-runtime composition layer until at least one such
preference has demonstrated lift avoids letting the wiring lead
the lift and locking in an abstraction at the wrong level.
3. The ADR-0047 null-lift finding is promoted from a historical
observation to a CI-enforced invariant. A new regression test
runs the public cognition split twice (flag OFF vs ON) and
asserts every watched metric is pair-wise identical. If
downstream realizer work later moves a metric on the flag flip,
the test fails as a deliberate transition rather than silent drift.
- docs/decisions/ADR-0058-forward-graph-constraint-status.md — ADR doc.
- docs/decisions/README.md — index entry.
- tests/test_forward_graph_constraint_null_lift.py — 2 tests:
null-lift invariant across the cognition lane, default-False contract.
Verification:
smoke 67 passed; flag tests 7 passed (5 wiring + 2 null-lift).
No runtime behaviour change; versor_condition < 1e-6 invariant unaffected.
The only path by which CORE extends its own active teaching corpus.
Closes ADR-0055 Phase C alongside ADR-0056's cognitive surface.
Three load-bearing calls (recorded in ADR-0057):
1. Replay-equivalence is a precondition, not a permission;
operator --accept remains required.
2. Eligibility = polarity in {affirms, falsifies} AND at least
one source='corpus' evidence pointer AND boundary_clean AND
claim_domain != evaluative (unless --allow-evaluative) AND
proposed_chain complete.
3. Append-only proposal log; corpus history append-only too.
Changes
- teaching/proposals.py — TeachingChainProposal, ReplayEvidence,
ProposalLog (event-sourced replay → current_state), eligibility
predicate, propose_from_candidate, accept/reject/withdraw,
append_chain_to_corpus (the sole corpus-write surface). Uses
TYPE_CHECKING guards to break the circular import with
chat.pack_grounding.
- teaching/replay.py — run_replay_equivalence; swaps _corpus_index
path to a tmp file, runs cognition lane on the active corpus
AND a transient copy with the proposed chain appended, returns
regressed-metrics list; trust-boundary assertion that the active
corpus bytes are byte-identical pre/post.
- teaching/discovery.py — moved chat.pack_grounding /
chat.teaching_grounding imports inside extract_discovery_candidates
to break the cycle (was masked when chat.runtime was the entry
point; surfaced by CLI entry).
- core/cli.py — three new subcommands:
core teaching propose <candidate-jsonl-path> [--allow-evaluative]
core teaching proposals [--state pending|accepted|rejected|withdrawn] [--json]
core teaching review <proposal_id> --accept --review-date YYYY-MM-DD
core teaching review <proposal_id> --reject [--note ...]
core teaching review <proposal_id> --withdraw [--note ...]
- tests/test_teaching_proposals.py — 16 tests covering: every
eligibility gate, proposal_id idempotency, append-only log,
replay-equivalent stays pending, regression auto-rejects with
named regressed metrics, --accept appends one line with typed
Provenance, --accept refused on non-equivalent, state-machine
blocks double-accept, real replay gate runs cognition lane
twice and asserts byte-clean active corpus pre/post.
Invariants preserved
- versor_condition(F) < 1e-6 — C2 touches no algebra path.
- Active corpus bytes byte-identical regardless of replay outcome.
- No clock-time reads, no LLM, no async.
- Proposal-only — accept_proposal is the sole corpus-write path.
Lanes: smoke 67 / cognition 121 / runtime 19 / teaching 17 /
new proposals 16. Cognition eval unchanged.
Open follow-ups (not in scope):
- supersession via operator review action
- cross-pack falsification arbitration (ADR-0056 Call 2 deferred)
- pack-data migration of frame-dependent connectives
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Splits ADR-0055 Phase C into:
- C1 (this ADR): cognitive contemplation loop — question
decomposition + polarity (affirms/falsifies/undetermined) +
claim_domain typing (factual/relational/evaluative)
- C2 (future ADR): review-and-apply — TeachingChainProposal,
replay-equivalence gate, corpus append-on-accept
Documents four load-bearing design calls with explicit reasoning
so future sessions can re-derive without re-arguing:
1. Stopping condition: record-the-gap-and-stop primary, bounded
depth failsafe; failsafe firing emits recursion_overflow audit
signal — never silent truncation.
2. Falsification evidence: reviewed-only, same pack family;
session-tier contests but does not falsify. Cross-pack
arbitration deferred.
3. Order: C1 before C2. Reversed instinct to land 'small thing
first' — C2 alone is useless without enriched input; C1
physically cannot mutate corpus until C2 wires the apply path.
4. Sync, not async. CORE hot path is deterministic; concurrency
overhead exceeds probe cost on local-only probes. Async
deferred to a future ADR if a blocking probe surface emerges.
Trust boundary: C1 never mutates the corpus. C1 reads pack,
corpus, vault, and most recent TurnEvent; writes only to the
existing Phase B discovery sink. Gap-recorded sub-questions
emit as new top-level candidates on the same sink — recursion
reified into the stream.
Maps directly onto user-stated framing recorded verbatim in the
ADR:
- 'contemplation always starts with a question' → candidate is
the posing; contemplate() is the answering
- 'truths and/or falsities' → polarity on the chain itself
- 'remain humble' → claim_domain with escalating evidence
thresholds, mandatory hedge for evaluative
Phased design for closing the inter-session learning loop without a
parallel learning path:
- Phase A: make today's 4-tier story load-bearing (audit CLI,
active-set view via superseded_by, typed provenance enum)
- Phase B: DiscoveryCandidate emission from the turn loop —
deterministic rule-firing on the audit trail, never writes the
corpus
- Phase C: TeachingChainProposal — sibling to PackMutationProposal,
proposal-only, replay-equivalence gate on dev+public
- Phase D: epistemic-tier guard (only COHERENT evidence promotes)
- Phase E: curriculum integration via formation review
Non-goals named explicitly: no embeddings, no DB storage, no
automatic identity/safety/ethics mutation, no opaque LLM step, no
removal of human reviewer.
Status Proposed; later ADRs land each phase against the verification
contracts named here.
