From 717eaf6ed7e5fb4f450c9989723f8416678ec562 Mon Sep 17 00:00:00 2001 From: Shay Date: Sun, 17 May 2026 18:48:44 -0700 Subject: [PATCH] docs: teaching-order doctrine + refresh formation/roadmap status - Add docs/teaching_order.md as durable reference for curriculum ordering. Five-layer rule (identity axes -> atomic definitions -> binary relations -> composed relations -> domain expansion), grounded in ratify.py G3, MasteredCoursesIndex, and exact CGA distance. Linked from README.md. - Mark formation_pipeline_plan.md as IMPLEMENTED (back half); enumerate the open items not closed by Phases 1-7 (additional templates, first formation- routed curriculum, G2 activation). - Add 2026-05-17 status block to capability_roadmap.md covering the FSC chain, epistemic schema closure, formation pipeline back half, FSC v3 proof matrix, cost benchmark, and the pulse import fix. --- README.md | 8 ++++ docs/capability_roadmap.md | 33 +++++++++++++- docs/formation_pipeline_plan.md | 28 +++++++++++- docs/teaching_order.md | 80 +++++++++++++++++++++++++++++++++ 4 files changed, 146 insertions(+), 3 deletions(-) create mode 100644 docs/teaching_order.md diff --git a/README.md b/README.md index e716da76..13a298d8 100644 --- a/README.md +++ b/README.md @@ -133,6 +133,14 @@ Full evidence: --- +## Teaching Order + +CORE's manifold is built by ratified relations under a strict prerequisite DAG — not by absorbing a corpus. The "elementary → college" intuition is right at the macro level (simple before composed, anchored before novel) and wrong at the literal level (don't import a K–12 corpus). Five-layer ordering: **identity axes → atomic definitions → binary relations → composed relations → domain expansion**, re-applied inside every new domain. + +Full doctrine, decision rules, and curriculum-platform locations: [`docs/teaching_order.md`](docs/teaching_order.md). + +--- + ## Architecture ``` diff --git a/docs/capability_roadmap.md b/docs/capability_roadmap.md index d4469cf6..f445c6b8 100644 --- a/docs/capability_roadmap.md +++ b/docs/capability_roadmap.md @@ -2,7 +2,38 @@ **Status:** Draft, derived from `docs/sessions/SESSION-2026-05-15-capability-gates.md` **Owner:** Joshua Shay -**Last updated:** 2026-05-15 +**Last updated:** 2026-05-17 + +## Status update — 2026-05-17 + +Work landed since the 2026-05-15 draft (each item is a roadmap input, not yet +a roadmap rewrite — that pass is queued): + +* **Forward Semantic Control chain** — ADR-0022 through ADR-0026 accepted; + inner-loop admissibility, rotor/frame admissibility, and ranked-with-margin + gates implemented and CI-enforced. See README §"Forward Semantic Control". +* **Epistemic schema closure** — three leaks closed, four new lanes, three new + architectural invariants (INV-21/22/23). Realizer-side refusal calibration + and contradiction-coherence checker landed. +* **Formation pipeline (back half)** — Phases 1–7 implemented; 138/138 + formation tests pass. See `docs/formation_pipeline_plan.md`. +* **FSC v3 50-case threshold proof matrix** — landed on + `feat/fsc-proof-suite-demo-rebased`; PR pending. +* **Cost benchmark** — `bench cost` lane reports $/1000 turns + latency with + disclosed assumptions; current measurement is 48–149× cheaper per turn than + frontier LLMs. +* **`core pulse` import fix** — 2026-05-17 hex-literal bug in + `language_packs/en_seeder.py` repaired; pulse path is operational again. + +Closed gates (vs the Phase 1 "Foundational Triple"): + +* `identity-divergence` — lane runner, axes, 93-event shared curriculum, dev / + public / holdout splits all present. +* `grammatical-coverage` — partial; v1 work in flight, no formal contract yet. +* `zero-code-domain-acquisition` — not started. + +A full rewrite of the phase exit-criteria against current state is queued for +the next planning pass. This document walks CORE from its present state through the gating framework defined in the 2026-05-15 session. It is organized into six phases. Each phase has entry criteria, work items, exit criteria, and a benchmark discipline contract. diff --git a/docs/formation_pipeline_plan.md b/docs/formation_pipeline_plan.md index 7193e5cc..82b3ce92 100644 --- a/docs/formation_pipeline_plan.md +++ b/docs/formation_pipeline_plan.md @@ -1,9 +1,33 @@ # Formation Pipeline — Implementation Plan -Status: **DRAFT — awaiting confirmation** +Status: **IMPLEMENTED (back half) — 2026-05-17** Author: planner pass, 2026-05-16 Companion doc: `docs/decisions/ADR-0021-epistemic-grade-policy.md`, -`docs/runtime_contracts.md` +`docs/runtime_contracts.md`, `docs/teaching_order.md` + +## Implementation status + +Phases 1–7 (the back half — substrate through reviewed-promote bridge) are +landed on `main`. `core test --suite formation -q` passes 138 / 138. + +| Phase | Target modules | Status | +|---|---|---| +| 1 | `course.py`, `hashing.py`, `cache.py`, `candidate.py` | ✅ implemented, tested | +| 2 | `forge.py` (sole text→manifold trust boundary) | ✅ implemented, tested | +| 3 | `compose.py` + `templates/definition.py` | ✅ implemented (only `definition` template exists; composed-relation / procedural / falsification / identity-anchor templates are still TODO) | +| 4 | `compiler.py` + `runner.py` | ✅ implemented, tested | +| 5 | `ratify.py` + `mastery.py` | ✅ implemented; G2 (prior-course regression) intentionally records `deferred:no_prior_courses` until `MasteredCoursesIndex` accumulates history | +| 6 | CLI wiring (`core formation`) + micro-course e2e | ✅ implemented, tested | +| 7 | `promote.py` + `index.py` (sole `SPECULATIVE→COHERENT` bridge through `teaching/review.py`) | ✅ implemented, tested | +| 8–11 | `smelter.py`, `miner.py`, adapters | smelter exists; LLM miner adapter remains gated behind `--enable-llm-source` per item #11 below | + +Open items not yet closed by the back-half landing: + +* Additional course templates beyond `definition` (see Phase 3). +* First production curriculum routed through Forge→Promote (current + `evals/identity_divergence/curriculum/teaching.jsonl` predates the pipeline + and still flows through the older runner path). +* G2 activation once a real `MasteredCoursesIndex` history exists. ## 0. Purpose diff --git a/docs/teaching_order.md b/docs/teaching_order.md new file mode 100644 index 00000000..d26c255e --- /dev/null +++ b/docs/teaching_order.md @@ -0,0 +1,80 @@ +# Teaching Order — How to Curriculum CORE + +**Status:** Reference doctrine. Update only when the formation pipeline's gate semantics change. +**Last updated:** 2026-05-17 +**Companion docs:** [`formation_pipeline_plan.md`](formation_pipeline_plan.md), [`capability_roadmap.md`](capability_roadmap.md), [`sessions/SESSION-2026-05-15-capability-gates.md`](sessions/SESSION-2026-05-15-capability-gates.md) + +--- + +## TL;DR + +Teach in **prerequisite-topological order**, not in pedagogical grade order. The "elementary → college" intuition is correct at the macro level (simple before composed, anchored before novel) and wrong at the literal level (do not start with a 3rd-grade language-arts corpus). Start with the smallest set of CORE-grade primitives the formation pipeline can ratify, then expand the prerequisite DAG outward. + +--- + +## Why order matters in CORE specifically + +CORE is not a transformer absorbing a corpus. Three structural reasons force a strict ordering: + +1. **`formation/ratify.py` G3** checks that every relation's `head` and `tail` concepts already exist as mastered or in-scope predecessors. Teach a triple whose endpoints aren't anchored and the gate hard-fails. +2. **`formation/index.py::MasteredCoursesIndex`** is built specifically so the next course's P3 (prerequisites) can read prior mastery. The system is curriculum-DAG-aware by construction; there is nowhere to hide an out-of-order triple. +3. **CGA recall is exact algebraic distance**, not a fuzzy embedding. A concept's position in the manifold is determined by the relations it participates in. Teaching `gravity ENTAILS mass` before `mass` and `gravity` are seeded produces an under-constrained point — the equivalent of writing equations with undefined variables. + +A sampling architecture absorbs corpora regardless of order because the loss surface averages everything out. CORE has no loss surface. Order is part of the construction. + +--- + +## The Five-Layer Ordering Rule + +Always teach in this order, both globally and re-applied within every new domain: + +1. **Identity axes and refusal probes.** Seeded first so identity is load-bearing before any content lands. Already covered in `formation/templates/definition.py::_IDENTITY_OVERRIDE_PROBES` and `evals/identity_divergence/axes/`. Adversarial probes must be defined before the concepts they protect. + +2. **Atomic definitions.** Concepts with no internal structure — `is_a`, `kind_of`, `instance_of` only. These are the leaf nodes of the prerequisite DAG. No relation in a step-2 course references a concept defined later in the same course. + +3. **Binary relations between defined concepts.** `entails`, `composes`, `contrasts`, `causes`, `precedes`, etc. — exactly the 17 predicates currently in the `en_core_cognition_v1` pack, applied only to concepts ratified in step 2. + +4. **Composed relations.** Chains that reuse step-3 predicates (`A causes B AND B entails C ⇒ A licenses C`). These exercise `compose_relations` and the realizer's chain-handling — the work that lifted the discourse_paragraph lane from 68.8% → 100%. + +5. **Domain expansion.** Pick *one* commercial domain and run the full 1→4 progression *inside* that domain before opening a second domain. Cross-domain triples come last and only after both domains have ratified their own internal DAG. + +The failure mode this rule prevents: a triple lands whose `tail` concept is "scheduled for next week" — `ratify.py` G3 fails, the course is rejected, and you've wasted the LLM mining cost on a course that can't be promoted. + +--- + +## What "elementary → college" gets right and wrong + +**Right:** Build simple before composed. Anchor before extend. Teach `parent_of` before `grandparent_of`, then `ancestor_of`. Teach `red` and `crimson` before `red is_a color and crimson is_a red`. + +**Wrong:** Don't import a K–12 ELA corpus. Most of that material is *generated content* (essays, stories, vocabulary lists) — the proposition graphs are mostly implicit, the relations are mostly stylistic, and the ratification gates will reject the bulk of it. CORE doesn't need stories; it needs ratified relations. A 200-triple curated kinship lane gets you further than 200,000 lines of children's literature. + +The right "elementary" for CORE is the smallest closed set of primitives that exercises every gate end-to-end. That's the kinship + color + spatial + uncertainty + modal mix already prototyped in `evals/identity_divergence/curriculum/teaching.jsonl`. + +--- + +## Where the curriculum platform lives today + +| Artifact | Path | Role | +|---|---|---| +| Shared 93-event teaching corpus | `evals/identity_divergence/curriculum/teaching.jsonl` | Reference curriculum for identity-divergence lane | +| Curriculum generator | `scripts/generate_identity_curriculum.py` | Produces teaching events across kinship / color / spatial / logical / uncertainty / modal domains | +| Axis profiles | `evals/identity_divergence/axes/{axis_a,axis_b}.yaml` | Precision-first vs Generosity-first identity profiles | +| Lane contract | `evals/identity_divergence/contract.md` | What the lane measures, scoring rubric, pass thresholds | +| Lane runner | `evals/identity_divergence/runner.py` | Deterministic scorer | +| First course template | `formation/templates/definition.py` | "Every relation = definitional edge" Course YAML template | +| Pack-template artifact | `packs/common/anchors/trilingual-anchor-template.json` | Trilingual anchor template (separate concern from formation/templates/) | + +**Known gap (as of 2026-05-17):** the identity-divergence curriculum predates the formation pipeline. Its triples currently flow through `runner.py`, not through `Forge → Compose → Ratify → Promote`. Routing existing curriculum events through the formation gates is a closing item — see [`formation_pipeline_plan.md`](formation_pipeline_plan.md). + +--- + +## Decision rule for "what to teach next" + +Before adding any course to the queue, answer: + +1. Are all `head` and `tail` concepts in this course already in `MasteredCoursesIndex` or earlier in this batch? +2. Does the course's template type exist? (Today: only `definition`. Composed-relation, procedural, falsification, and identity-anchor templates are not yet implemented.) +3. Is at least one identity-override probe present? +4. Would `ratify.py`'s six gates (G1–G6) succeed against this material? + +If any answer is no, the course is out of order. Move it later in the queue, or add the missing prerequisite course first.