Shay 44e78aa438 feat(generalization): curriculum-grounded serving — exams answered from ratified curriculum (ADR-0262)

Phase 2 of the generalization arc, implementing the plan's §4
curriculum-entailment gold contract. "Does force cause acceleration?" is
answered from the ratified physics chain corpus and nothing else; "Does
gravity cause acceleration?" is declined because gravity is in no pack
CORE has been taught; "Does force cause motion?" is unsettled even though
the curriculum contains both links, because nothing ratified says
causation composes.

Path (flag-gated, default off): closed question grammar -> subject routing
by ratified vocabulary -> family-scoped premise compilation from reviewed,
pack-resident chains -> the SAME argument bands (ADR-0260/0261) -> the
ROBDD engine. Zero subject-specific decision code: physics differs from
philosophy only in which rows load.

Epistemology enforced mechanically, not by convention:
- gold is a function of (curriculum, question); cases pin chain ids and
  the runner FAILS a case whose pinned chain is absent or unratified;
- untaught => UNKNOWN, never "no" (open-world; silence is not denial);
- an independent oracle (own loader, ratification predicate, family table,
  agreement rules, verdict rule) re-derives every gold — it disagreed once,
  on "entropy reveals energy" vs "entropy causes energy", and the ORACLE
  was the side that was wrong;
- anti-recall probes are a lane GUARD: a split without >=3 true-but-untaught
  probes refuses to run.

Findings recorded rather than worked around (ADR-0262 §5):
- every curriculum band is UNEARNED and every answer is hedged. A band needs
  n>=657 with a real outcome mix; physics teaches 7 causal + 9 modal
  relations, so at most 16 questions in the subject can ever be ENTAILED.
  A balanced band needs ~219 taught relations per subject×family — a ~25x
  gap that only ratified curriculum content closes. Phase 2's blocker is
  curriculum volume, not machinery.
- REFUTED is unreachable from present corpora (every chain is positive).
- there is NO biology domain-chain corpus; the biology OOD lane measures
  fluency, not truth. The four subjects with ratified chains and mounted
  packs are physics, mathematics_logic, systems_software,
  philosophy_theology — the composer serves all four.

[Verification]: curriculum lane 32/32 wrong=0 with 5 anti-recall probes and
all three contract guards passing; tests/test_curriculum_serve.py 20 passed;
core test --suite deductive 252 passed; lane pinned as curriculum_serve_v1.

2026-07-24 14:38:08 -07:00

4.1 KiB

Raw Blame History

Curriculum-serve lane contract

What this lane scores

The production curriculum-grounded answering path — the exact pipeline chat/curriculum_surface.py::decide_curriculum_question runs on a core chat turn when curriculum_serving_enabled is on:

Does <term> <relation> <term>?      (closed exam-question grammar)
  → subject routing        (the domain whose ratified vocabulary holds BOTH terms)
  → relation family        (the connective, normalized by ADR-0260 agreement)
  → premise compilation    (that family's RATIFIED chains, and nothing else)
  → the argument bands     (ADR-0260 verb reading → ADR-0261 existential)
  → the ROBDD engine       (ADR-0201/0218)

There is no subject-specific decision code anywhere in the path. Physics differs from philosophy only in which curriculum rows load — which is the property that makes "add a subject" a data operation.

Gold vocabulary

Three classes are reachable: entailed, unknown, declined.

refuted is unreachable from a purely positive curriculum and that is not an omission. The curriculum is read OPEN-world: a relation it does not state is UNKNOWN, never "no". Refuting would require the curriculum to teach a negative ("no X causes Y"), which no ratified corpus does yet. See ADR-0262 §5.

The three lane guards (all run before any case is scored)

Failing a guard is a lane failure, not a case miss — the lane is unsound, not merely under-covered.

Provenance (plan §4.3) — every chain id a case pins must resolve in the subject's ratified curriculum. A case whose curriculum moved under it breaks loudly rather than quietly answering from what remains.
Corpus soundness (§4.4) — the INDEPENDENT oracle (evals/curriculum_serve/oracle.py, sharing no code with the serving path: own loader, own ratification predicate, own family table, own agreement normalization, own verdict rule) must re-derive every committed gold.
Anti-recall coverage (§4.7) — the split must carry ≥3 probes whose answer is true in the world and absent from the curriculum, and no probe may carry a committed gold. Without them the lane cannot show the system decodes rather than recalls, and it must not ship.

Splits

physics/ — 32 hand-authored cases over physics_chains_v1: 13 taught edges (both connectives of each family, incl. the row whose declared operator_family disagrees with its connective), 5 untaught compositions at depth 2–3, 4 reverse-direction and cross-relation near-misses, 5 anti-recall probes (3 with untaught vocabulary — gravity, current, pressure — and 2 with taught vocabulary but untaught relations), and 5 typed refusals.

wrong=0 discipline

Identical to the deduction-serve lane: wrong (a committed verdict that disagrees with gold) MUST stay 0; a decline where gold expected a verdict is a coverage miss, tracked in counts.declined, never conflated with a confabulation. The runner requires correct == n.

What the lane deliberately does NOT do

It does not compose chains. force causes acceleration and acceleration causes motion are both taught; force causes motion is UNKNOWN. Causal transitivity is a substantive claim about the world, and no ratified corpus teaches it. The oracle reports the shortest path length alongside its verdict precisely so the lane can assert that a reachable pair is still answered UNKNOWN — composition is provably not happening.
It does not read the corpus's operator_family field. The family comes from the CONNECTIVE, because a question carries a relation word and nothing else; deriving the family from a field the question cannot carry would let premise compilation and question routing disagree, and a taught edge could go missing from the premises compiled to decide it.

Reproduce

uv run python -m evals.curriculum_serve.runner                                    # human-facing
uv run python -m evals.curriculum_serve.runner --report evals/curriculum_serve/report.json

Pinned in scripts/verify_lane_shas.py as lane id curriculum_serve_v1. core test --suite deductive runs tests/test_curriculum_serve.py.

4.1 KiB Raw Blame History Unescape Escape