Merge pull request 'docs(arc): curriculum-license-loop plan of record + tier division of work' (#115) from docs/curriculum-license-loop-plan into main

Reviewed-on: #115
This commit is contained in:
Joshua Matthew-Catudio Shay 2026-07-25 22:17:47 +00:00
commit 6ada6f7a03
2 changed files with 444 additions and 0 deletions

View file

@ -0,0 +1,258 @@
# Division of Work — Curriculum License Loop
**Arc:** curriculum-license-loop-2026-07 · **Written:** 2026-07-25 · **By:** Opus 5
**Plan of record:** `docs/plans/curriculum-license-loop-2026-07-25.md` (binding)
**Read before any unit:** that plan §1§3, then `AGENTS.md`.
This file is the contract for *who does which unit and why*. It is written to be
read cold, by a tier that was not present for the analysis. If it conflicts with
a memory or a chat instruction, the plan of record and `AGENTS.md` win.
---
## 1. The tiering criterion — read this first, it is the whole rationale
Units are **not** assigned by difficulty, and not by model prestige. The evidence
does not support that split. In the 2026-07-24/25 window Sonnet 5 (Tier S)
produced the six-worktree `public_demo` archaeology, found the
`derived_close_proposals.py::DEFAULT_SINK` bug that wrote outside the repository,
and found an engine-refused misclassification while building the instrument meant
to measure something else. In the same window Opus 5 mis-diagnosed the CGA hot
path by multiplying a microbenchmark by a call count, and made three wrong
readings of this branch's own commits that survived only until they were checked.
Model tier was not the variable. **Verification discipline was.**
So the split is by one question:
> **If this unit is done wrong, does a gate catch it, or is the error silent?**
- **Gate-caught** — a test fails, an import breaks, a lane goes red, a hash
drifts. The work is safe to execute at the cheapest competent tier, because
being wrong is *loud*. Iterating against a red gate needs no design authority.
- **Silent** — the system stays green while asserting something false. No
downstream check can recover it, because the wrong thing has become the
reference. These units need the tier that can reason about what the gate cannot
see, and they must be finished before the units that depend on them.
Two units in this arc are silent-failure units *because they are themselves the
gate*. Those are the ones worth spending expensive tokens on. Everything else is
loud, and spending expensive tokens on it is waste.
**Corollary for whoever picks this up:** if you find yourself about to make a
judgment call that no test would catch, you have crossed a tier boundary. Stop
and escalate (§6). That is not a failure — it is the mechanism working.
---
## 2. Ownership at a glance
| Unit | Phase | Owner | Failure mode |
|---|---|---|---|
| Negative-curriculum epistemology ADR | A | **Opus** | silent |
| Volume-honesty invariant (tests first) | B | **Opus** | silent |
| R7 assertion — *what* to assert | E1 | **Opus** | silent |
| Articulation feasibility study | §6 of plan | **Opus** | silent |
| `assert_corpus_sound` rename | E2 | **Sonnet** | loud |
| Practice producer implementation | C | **Sonnet** | loud |
| `core proposal-queue reseal` verb | D | **Sonnet** | loud |
| R7 code move + suite registration | E1 | **Sonnet** | loud |
| Rust parity measurement | plan §6 | **Sonnet** | loud |
| ADR status vocabulary sweep | E3 | **Haiku** | loud |
| Content authorship (≈219 relations) | F | **Shay** | — |
| Capability-index scoring / stratification / `hash_surface` rulings | plan §5 | **Shay** | — |
| `curriculum_serving_enabled` flip | G | **Shay** | — |
---
## 3. Opus units — expectations
### A. Negative-curriculum epistemology (ADR) — **do this first**
**Why Opus.** This is the highest-risk unit in the arc. If the epistemology is
wrong, the corpus encodes falsehoods and **`wrong=0` still passes** — because the
lane's gold derives from the same corpus, and `evals/curriculum_serve/oracle.py`
is independent in *code* but reads the *same rows*. Corpus-level epistemology
errors are structurally unfalsifiable by every gate this repo has. Nothing
downstream notices; the whole arena inherits the error and certifies it at
volume.
**Binding constraints (from the plan, not re-litigable):**
- Absent edge ⇒ UNKNOWN, never "no". This is the open-world reading and it is
the reason curriculum serving is honest at all.
- Explicit negative edge ⇒ REFUTED. These are different states and the corpus
must distinguish them.
- Polarity is a **row-level field or `intent` value**, never a new
`operator_family`. Bands key on `(domain, operator_family)`
(`chat/curriculum_surface.py:184`), so a `modal_negative` family creates a new
band at n=0 rather than adding refuted volume to `curriculum_physics_modal`.
- The oracle must learn polarity **independently** of the serving path. Its
code-level independence is the entire evidentiary value; do not let the two
share a polarity helper.
**Done when:** ADR is Accepted, the row schema shape is fixed, and the
UNKNOWN-vs-REFUTED boundary is stated precisely enough that Phase B can enumerate
against it. **No code.**
### B. Volume-honesty invariant (tests, written before any producer)
**Why Opus.** This unit *is* the anti-gaming gate, so there is nothing above it
to catch a mistake. S6 already proved the exposure: `conservative_floor` cannot
distinguish "657 independent facts" from "16 facts asked 41 times each"
(`docs/research/curriculum-volume-quantification-2026-07-24.md` §1). An unbounded
generator therefore clears θ=0.99 at n≥657 **without earning anything**, and the
resulting license reads as identical to a real one. Getting the bounding rule
wrong silently invalidates all of Phase F's authoring effort.
**Expectations:**
- Tests come **first** and must fail for the right reason before any producer
code exists. The test file is the specification of the invariant; that is why
it is the deliverable and not a by-product.
- At most one committed case per distinct
`(relation, direction, polarity, question-shape)` tuple. No paraphrase padding.
- Assert the bound structurally: *N* ratified chains cannot yield more than a
bounded multiple of *N* committed cases.
- **Mutation-check it.** Pad the generator ⇒ red. A pin that cannot fail is
worthless; this is the same discipline `tests/test_adr_status_governance.py`
was held to.
**Done when:** tests exist, fail correctly, fail under mutation, and the module
interface they imply is unambiguous enough that Phase C needs no design decisions.
### E1 (Opus half). The R7 assertion
**Why Opus.** The code move is trivial and specified —
`docs/specs/runtime_contracts.md:448-456` already says "defer the single sink
emission to the pipeline serve boundary." What is *not* specified is what to
assert, and the default sink is `None`, so **nothing exercises this path**. A
wrong move is therefore silent. This is also the exact seam that regressed
silently for two days in the #96 register-axis incident, so it gets the careful
tier by history as well as by reasoning.
**Done when:** a written assertion exists — the sink receives the **post-override**
event, and its `trace_hash` matches a pipeline replay — precise enough for Sonnet
to implement against.
---
## 4. Sonnet units — expectations
Every unit here has a loud failure mode. You are expected to iterate against the
gate until it is green, not to re-derive design. If a unit seems to require a
design decision, that is the signal in §6.
**Order: E2 → C → D → E1(code) → parity.** E2 first because Phase C adds a third
`assert_corpus_sound` caller and renaming after that is strictly more work.
### E2. `assert_corpus_sound` rename
Two functions, one name: `evals/deduction_serve/practice/gold.py:1163` (no args,
practice corpus) and `evals/curriculum_serve/runner.py:71` (`domain, cases`, lane
contract). Not a missing contract — a name collision, flagged in
`BRIEF-CLOSE-assessment-verification-2026-07-25.md` §9.5. **Gate:** imports break.
### C. Practice producer
`evals/curriculum_serve/practice/{__init__,generator,runner}.py`. Mirror
`evals/deduction_serve/practice/` topology **exactly**: producer in `evals/`,
sealed artifact next to its reader in `chat/data/`.
**Reuse, do not rewrite** — this is why the unit is small:
- `evals/curriculum_serve/oracle.py` — already an independent decision procedure;
wrap as the gold tether.
- `core/learning_arena/engine.py::run_practice` — the ADR-0199 pure fold.
- `core/ratified_ledger.py::{seal_artifact, tally_dict, write_sealed_ledger}`
extracted by ADR-0263 for exactly this third instance.
- `core/reliability_gate::{Action, Ceilings, ClassTally, license_for}`.
- `chat/curriculum_surface.py` — the solver under test.
Determinism is required: no clock, no RNG, byte-identical sealed output across
runs, so the artifact is safe to commit and SHA-verify on load. Read
`evals/deduction_serve/practice/runner.py` docstring for the pattern.
**Gate:** Phase B tests; `wrong=0` on the physics lane.
### D. `core proposal-queue reseal`
`teaching/ratification.py`, `core/cli_proposal_queue.py`. Explicit verb —
**never** auto-reseal on `ratify` (doctrine forbids ratification automation, and
per-row reseal would churn the ledger's `content_sha256`). `ratify_chain` records
the enqueue; a human runs the seal. Follow the existing
`status|list|show|review|ratify` verb patterns.
**Gate:** CLI tests; the verb produces a SHA-sealed
`chat/data/curriculum_serve_ledger.json`, and `curriculum_serve_license()` reads
a real but still-unearned ledger instead of an absent one.
### E1 (code half). R7
Move the emission per Opus's written assertion. Do not improvise the assertion.
### Rust parity measurement
`PYO3_PYTHON=/opt/homebrew/bin/python3.12`, `cargo test` in `core-rs/`, then rerun
the hash-pinned lanes under `CORE_BACKEND=rust`. cargo 1.97.1 and the registry
cache are present locally; the cloud session was blocked only by sandbox network
policy. **Report the measurement; do not act on it.** A parity *failure* is a
silent-class finding about determinism — escalate rather than "fix."
### Suite registration (all units)
Anything you add goes into a curated suite in `core/cli_test.py`. Two files from
PR #113 landed in **no** suite and ran only under `full`, which gates nothing —
that was the fifth instance of this pattern in three arcs. Do not make it six.
---
## 5. Haiku unit
### E3. ADR status vocabulary normalization
27 unparseable status lines plus `draft`/`ratified`/`active`/`accepted.` variants
across 312 ADR files (inventory: `docs/research/adr-status-governance-2026-07-25.md`
§6). Normalize to the intended lifecycle, then widen
`tests/test_adr_status_governance.py` from its deliberately narrow scope to a
closed vocabulary.
Bulk mechanical with an objective acceptance criterion: **the widened pin goes
green.** Do not change any ADR's *decision* — only its status token. If a file's
intended status is genuinely ambiguous, list it and stop; do not guess.
---
## 6. Escalation — when a cheaper tier must stop
Stop and hand back up when any of these is true. Escalating is the mechanism
working, not a failure; guessing past one of these is the only real error
available here.
1. **You are about to make a call no test would catch.** By definition you have
crossed into a silent-failure unit.
2. **A gate is green but you do not believe it.** That is the signature of a
reference that has become wrong — exactly the ADR-0256 shape, where the record
asserted "not yet decided" about a live decision and nothing failed.
3. **The measurement contradicts the plan.** Say so with the numbers. Both prior
arcs improved by having their premises falsified in writing; a falsified plan
premise is a deliverable, not an interruption.
4. **A parity or determinism check fails.** Never "fix" a hash drift. Report it.
5. **Scope exceeds the unit as written.** Flag it in the PR/handoff rather than
improvising, per the standing Tier-S/O constraint.
---
## 7. Standing constraints (non-negotiable, all tiers)
- `wrong=0`. No flag flips — those are Shay's ratification only.
- Forgejo-primary; no `gh`, no `tea`. SSH:29222 often times out — push over the
authenticated HTTPS remote instead.
- Branch **and** worktree per unit; both deleted immediately after merge.
- Pre-push gate in-worktree on canonical **Python 3.12.13** with
`uv sync --locked`: `core test --suite smoke -q` then `--suite deductive -q`.
Baselines at arc start: **smoke 555**, **deductive 285**.
- `[Verification]:` line in every PR description, naming the interpreter.
- One PR per coherent unit. Compare-URL PRs; **Shay merges** — no merge
automation, and green PRs sit until explicit authorization.
- Never `scripts/verify_lane_shas.py --update` — surgical single-line pin edits
only. Three arcs depend on that rule.
- New tests go into a curated suite (§4).
## 8. At arc close
Fill in `docs/handoff/ARC-CLOSE-TEMPLATE.md` → this directory. The
**Falsified assumptions** section is the one that cannot be recovered from the
diff — state which parts of the plan of record turned out wrong, including your
own claims. Also record whether the articulation trigger fired, which the
previous arc left unrecorded.

View file

@ -0,0 +1,186 @@
# Curriculum License Loop — Plan of Record
**Date:** 2026-07-25 · **Status:** ACTIVE · **Base:** main @ `0a26787f`
(post ADR-governance pin, PR #114)
Successor to `generalization-arc-2026-07-24.md`, which closed its Tier F/O/S
arc, and to the assessment-verification arc (PR #113). Division of work across
tiers: `docs/handoff/curriculum-license-loop-2026-07/DIVISION-OF-WORK.md`
(binding — read it before starting any unit).
---
## 1. Why this arc exists
Two arcs closed on the same stated conclusion: *the binding constraint is
ratified curriculum volume, not machinery.* That conclusion is half right, and
the wrong half is load-bearing. Verified 2026-07-25 by reading code:
1. **`evals/curriculum_serve/practice/` does not exist.** The deduction analogue
(`evals/deduction_serve/practice/{gold,runner}.py`) is what folds
`run_practice` over a corpus into per-band `ClassTally`. Curriculum has no
producer.
2. **`seal_ledger` exists only for deduction**
(`evals/deduction_serve/practice/runner.py:96`).
`chat/curriculum_serve_license.py`'s own docstring names
`evals.curriculum_serve.practice.runner.seal_ledger` as "the only writer" of
the artifact it reads. **That writer was never built**, and
`chat/data/curriculum_serve_ledger.json` does not exist.
3. **The ceremony stops short.** `teaching/ratification.py::RatificationReceipt`
carries `pending_stages = ("arena_queue_entry", "ledger_reseal")` — it grows
the corpus and explicitly neither practices it nor reseals the ledger.
So the pipeline is `author → ratify chain → ??? → ??? → license → serve`, with
two unbuilt segments. **Authoring curriculum today cannot earn a license at any
volume.** Content is necessary and currently insufficient.
The build is nonetheless *modest*, because the expensive pieces already exist:
`evals/curriculum_serve/oracle.py` is already an independent decision procedure
usable as the gold tether, and ADR-0263 already extracted the sealers
(`core/ratified_ledger.py`) for precisely a third instance.
## 2. Two measured facts that shape the content work
- **`refuted` has never been produced.** The physics lane gold is
`entailed 14 / unknown 12 / declined 6` — zero refuted. The verdict exists in
`CurriculumDecision.verdict`; no corpus row can express it.
- **Bands key on `(domain, operator_family)`**
(`chat/curriculum_surface.py:184`). Negative content must therefore live
*inside* `operator_family: "modal"` with a row-level polarity marker. A
separate `modal_negative` family would create a **new band at n=0** instead of
adding refuted volume to the target band. This is the easiest available way to
waste the entire authoring effort.
## 3. Decisions taken (2026-07-25, Shay)
| Decision | Choice |
|---|---|
| Leading track | Close the license loop (not articulation, not substrate) |
| First content target | **physics · modal** — n=9, closest of eleven bands (24× short) |
| `refuted` representation | **Explicit negative edges**, polarity inside the existing family |
| Reseal trigger | **Explicit `core proposal-queue reseal` verb**, never automatic on `ratify` |
Reseal is explicit because `AGENTS.md` forbids ratification automation, and
because an auto-reseal per ratified row would churn the ledger's
`content_sha256` on every append.
## 4. Phases
### Phase A — Negative-curriculum epistemology *(design only; gates everything)*
ADR deciding how an explicitly-taught negative coexists with the open-world
reading. Binding constraints: absent edge ⇒ UNKNOWN (never "no"); explicit
negative edge ⇒ REFUTED; polarity is a row-level field or `intent` value, never
a new `operator_family`; `evals/curriculum_serve/oracle.py` must learn polarity
**independently** of the serving path, since that independence is the whole
evidentiary value.
Touches: `teaching/domain_chains/physics_chains_v1.jsonl` schema,
`teaching/curriculum_premises.py`,
`teaching/ratification.py::validate_admissible`,
`evals/curriculum_serve/oracle.py`, lane contract anti-recall probes.
**Done when:** ADR Accepted; schema shape fixed; no code written.
### Phase B — The volume-honesty invariant *(the anti-gaming gate)*
Failing tests written **before** any producer code. The generator must emit at
most one committed case per distinct `(relation, direction, polarity,
question-shape)` tuple and must never paraphrase-pad. S6 already proved
`conservative_floor` cannot distinguish "657 independent facts" from "16 facts
asked 41 times," so an unbounded generator would clear the Wilson floor without
earning anything.
**Done when:** tests exist, fail for the right reason, and fail under mutation
(pad the generator ⇒ red).
### Phase C — The producer *(implementation against Phase B)*
`evals/curriculum_serve/practice/{__init__,generator,runner}.py`, mirroring
`evals/deduction_serve/practice/` topology exactly — producer in `evals/`,
artifact next to its reader in `chat/data/`.
Reuse, do not rewrite: `evals/curriculum_serve/oracle.py` (gold tether),
`core/learning_arena/engine.py::run_practice`,
`core/ratified_ledger.py::{seal_artifact, tally_dict, write_sealed_ledger}`,
`core/reliability_gate::{Action, Ceilings, ClassTally, license_for}`,
`chat/curriculum_surface.py` (solver under test).
**Done when:** Phase B tests green; `wrong=0` on the physics lane.
### Phase D — Close the ceremony's `pending_stages`
`core proposal-queue reseal` verb; `ratify_chain` records the enqueue.
Files: `teaching/ratification.py`, `core/cli_proposal_queue.py`.
**Done when:** the verb produces a SHA-sealed
`chat/data/curriculum_serve_ledger.json`, and
`curriculum_serve_license()` reads a real (still-unearned) ledger rather than an
absent one.
### Phase E — Honesty seams *(parallel, no dependency on AD)*
- **E1 — audit-ledger R7.** Defer the single sink emission to the pipeline serve
boundary (`chat/runtime.py::finalize_turn_surface`; spec at
`docs/specs/runtime_contracts.md:448-456`). Repairs the stale surface and
stale `trace_hash` together. Default sink is `None`, so no deployment risk.
Independent of the `hash_surface`-on-`TurnEvent` ruling.
- **E2 — `assert_corpus_sound` name collision.** Two functions, one name:
`evals/deduction_serve/practice/gold.py:1163` (no args, practice corpus) vs
`evals/curriculum_serve/runner.py:71` (`domain, cases`, lane contract). Must
land **before** Phase C adds a third caller.
- **E3 — ADR status vocabulary normalization.** 27 unparseable status lines plus
`draft`/`ratified`/`active`/`accepted.` variants across 312 files; then widen
`tests/test_adr_status_governance.py` to a closed vocabulary.
### Phase F — Content *(the actual constraint)*
Author + ratify physics · modal volume: ≈219 distinct taught modal relations for
the entailed bucket, plus matching refuted/unknown. Measure with
`teaching.ratification._count_chains` and
`evals/vocab_trigger_instrument.py --compare` admissions delta.
Relations must be **true**. Drafting candidates is delegable; ratification is
Shay's.
### Phase G — The flag *(Shay only)*
Only after a band clears θ=0.99 at n≥657 with a genuine outcome mix: *propose*
flipping `curriculum_serving_enabled`. On earned evidence, never on proximity.
## 5. Deferred, decision-gated *(a ruling, not effort)*
- **Band-level capability index** — 25 near-1.0 entries would inflate
`coverage_geomean` against the index's own anti-gaming property. Needs a
scoring decision before any schema change.
- **Discovery-yield stratification** — blocked on new persisted per-source
counters in the `engine_state` manifest
(`teaching/discovery_yield.py:67-70` reads one monotonic `turn_count`).
- **`hash_surface` on `TurnEvent`** — self-verifying telemetry vs replay-only.
Current contract states recomputation is unsupported *and correct to be so*.
## 6. Off this arc's critical path
- **Articulation depth (Phase 5 / core_logos).** `EntailmentTrace` carries five
opaque BDD node keys; `generate/proof_chain/entail.py:125-190` decides via
`is_tautology`. There are no intermediate steps, so nothing is dropped and the
stated trigger describes an event that cannot occur. CORE's decision procedure
is **complete but not explanatory.** Next deliverable is a *feasibility study
for proof-term extraction over the ROBDD* — a derivation calculus with a real
soundness surface, not a data-structure refactor. Highest intellectual value,
least schedulable.
- **Substrate.** `CORE_BACKEND=rust` bit-exact parity is now answerable locally
(cargo 1.97.1 + registry cache present; `core-rs/tests/test_crdt_hash_parity.rs`
exists; needs `PYO3_PYTHON=/opt/homebrew/bin/python3.12`). Then Rust typestate
(`UnverifiedClaim` → `VersorClaim`) — zero determinism cost, greenfield. The
real hot path is `cga_inner``geometric_product`, 33,986 calls/turn, ~73% of
turn time. **Not** the diagonal shortcut — not bit-exact (954/4000 in f32).
**Refuted, do not revisit:** `.metal` kernel, MLX fusion, bf16.
- **Math.** Phase 4.2 case-first is falsified on a count — cases
0000/0001/0148/0082 affect 1 and 2 cases out of 500. Only remaining lever is
the `q:complex` decomposition study (101 cases, one label). Low expected value.
## 7. Dependencies
- **A gates B** (polarity shapes the generator's enumeration — building C before
A means building it twice).
- **B gates C** (tests first; the invariant *is* the specification).
- **E2 lands before C.** E1/E3 fully parallel.
- **C + D gate F's measurement**, and F gates G.
- Phase 6 items independent and parallel throughout.
## 8. Non-goals
- Flipping any flag without explicit ratification.
- Adding band entries to the capability index before the scoring decision.
- Paraphrase-padding practice volume to reach the Wilson floor.
- Renumbering or overwriting Accepted ADRs.