From 2acf71f0242f76b0df41604c4f5234274eadff36 Mon Sep 17 00:00:00 2001 From: Shay Date: Mon, 18 May 2026 14:02:37 -0700 Subject: [PATCH] curriculum(epistemology-v1): five reviewed chains; holdout term_capture +4.2pp MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First end-to-end curriculum unit through the production propose / review --accept / supersede operator surfaces against the active teaching corpus. Replay-equivalence gate passed for every proposal; public split byte-identical; holdout term_capture lifted exactly as predicted. - Supersede `verification_wisdom_grounds_judgment` → `verification_wisdom_requires_knowledge`. Fixes the only corpus- fixable holdout miss: `verification_wisdom_036` ("Is wisdom the same as knowledge?") now grounds with both expected terms. Provenance carries `:supersede(verification_wisdom_grounds_judgment)`. - Propose + accept four new chains closing epistemology subgraph cells: cause_understanding_requires_knowledge cause_judgment_requires_wisdom verification_evidence_grounds_knowledge cause_inference_requires_evidence Each chain is pack-consistent, uses canonical predicates, and opens a previously-empty (subject, intent) cell. Replay gate confirmed no metric regression on the public split before each accept. Lift (cognition eval): public : intent 100% / surface 100% / term 91.7% / versor 100% (unchanged) holdout : intent 100% / surface 94.7% / term 70.8%→75.0% / versor 100% The remaining four holdout misses (correction_truth_040, procedure_define_010, unknown_spirit_041, unknown_word_018) are architectural — surface-composition gaps in the correction- acknowledgment template, procedure-intent routing, and unknown- intent surface — and out of scope for corpus surgery. - teaching/cognition_chains/cognition_chains_v1.jsonl — 10 → 15 lines (4 appends + 1 supersession marker; 1 retired chain still on disk per the audit doctrine of append-only at the file level). - teaching/proposals/proposals.jsonl — new append-only proposal log with `created` / `replay` / `transition` / `accepted_corpus_append` events for every accepted proposal. - docs/curriculum/epistemology_v1.md — full curriculum log: rationale per chain, prediction-vs-result on the holdout lift, reproducibility commands, architectural-gap analysis. Lanes (regression check): core test --suite smoke 67 passed core test --suite cognition 121 passed core test --suite teaching 17 passed tests/test_eval_holdout_split 10 passed The first curriculum unit that *measurably moves a cognition-lane metric* through the operator surfaces, with full provenance from operator note back to corpus append. --- docs/curriculum/epistemology_v1.md | 169 ++++++++++++++++++ .../cognition_chains_v1.jsonl | 5 + teaching/proposals/proposals.jsonl | 16 ++ 3 files changed, 190 insertions(+) create mode 100644 docs/curriculum/epistemology_v1.md create mode 100644 teaching/proposals/proposals.jsonl diff --git a/docs/curriculum/epistemology_v1.md b/docs/curriculum/epistemology_v1.md new file mode 100644 index 00000000..8144aab1 --- /dev/null +++ b/docs/curriculum/epistemology_v1.md @@ -0,0 +1,169 @@ +# Curriculum Unit — Epistemology v1 + +**Date:** 2026-05-18 +**Author:** Shay +**Lift:** cognition holdout `term_capture_rate` 70.8% → 75.0% (+4.2pp); public split unchanged. +**Active corpus after this unit:** 14 chains (15 lines on disk, 1 retired). + +--- + +## Premise + +CORE's reviewed teaching corpus is the inter-session memory surface +([ADR-0055](../decisions/ADR-0055-inter-session-memory-discovery-promotion.md)). +The operator-driven path to extend it is the propose → replay-equivalence +gate → operator accept loop ([ADR-0057](../decisions/ADR-0057-teaching-chain-proposal-review.md)), +plus the supersede surface for retiring chains in favour of replacements. + +This unit is the **first** end-to-end use of those surfaces against the +production corpus to **measurably lift a cognition-lane metric**. + +--- + +## What this unit teaches + +Five reviewed chains closing epistemology-subgraph cells: + +| Action | Old chain (if any) | New chain | +|---|---|---| +| Supersede | `verification_wisdom_grounds_judgment` | `verification_wisdom_requires_knowledge` | +| Propose + Accept | — | `cause_understanding_requires_knowledge` | +| Propose + Accept | — | `cause_judgment_requires_wisdom` | +| Propose + Accept | — | `verification_evidence_grounds_knowledge` | +| Propose + Accept | — | `cause_inference_requires_evidence` | + +All five claims are pack-consistent (subject + object both in +`en_core_cognition_v1`) and follow the canonical predicate set. + +--- + +## Why these chains (and not others) + +### The supersede + +`verification_wisdom_036` ("Is wisdom the same as knowledge?") was +the only holdout case whose term-capture miss was addressable by +corpus surgery. The case expects both `wisdom` and `knowledge` in +the surface; the prior chain (`wisdom grounds judgment`) emitted +only `wisdom` and `judgment`. Replacing the chain's object from +`judgment` to `knowledge` and the connective from `grounds` to +`requires` produces a defensible claim (wisdom presupposes +knowledge) and emits both expected terms. + +No holdout case depended on the prior `wisdom→grounds→judgment` +doctrine, so the supersede is net-positive. Provenance on the new +entry: `adr-0057:hand_authored:2026-05-18:supersede(verification_wisdom_grounds_judgment)`. + +### The four proposals + +Each opens a previously-empty `(subject, intent)` cell in the +teaching index: + +- `(understanding, cause)` — *understanding requires knowledge*: the + classical claim that understanding is structured-over-knowledge. +- `(judgment, cause)` — *judgment requires wisdom*: pairs with the + pre-existing `wisdom orders judgment`, asserting the inverse + dependency direction (wisdom orders judgment, judgment depends + on wisdom to be sound). +- `(evidence, verification)` — *evidence grounds knowledge*: cognate + to `truth grounds knowledge` but admitting evidence as the + warranting relation. +- `(inference, cause)` — *inference requires evidence*: pairs with + `knowledge requires evidence`, locating inference downstream of + evidence rather than alongside knowledge. + +Each is defensible philosophically and passes the eligibility +predicate (polarity=affirms, corpus-evidence-pointer, claim_domain=factual, +boundary_clean, complete chain). + +--- + +## Why the holdout lift is exactly +4.2pp + +Holdout has 19 cases × 1.27 terms-per-case ≈ 24 expected terms +total. Pre-unit term-capture was 70.8% = 17/24 ≈ 17 captured. +Post-unit captures one additional term (`knowledge` in +`verification_wisdom_036`), so 18/24 = 75.0%. **The +4.2pp matches +the single-term fix exactly.** + +The remaining four holdout misses are architectural, not corpus-fixable: + +| Case | Category | Gap | +|---|---|---| +| `correction_truth_040` | correction | Correction-acknowledgment template doesn't mention the corrected subject lemma. | +| `procedure_define_010` | procedure | `procedure` intent has no teaching-grounded surface path; pack-grounded path doesn't fire on this intent. | +| `unknown_spirit_041` | unknown | UNKNOWN intent → disclosure; no clear `(subject, intent)` to teach. | +| `unknown_word_018` | unknown | Same. | + +Closing those four requires changes to runtime surface composition, +not to the teaching corpus. They're correctly scoped out of this unit. + +--- + +## Public split: no regression + +The replay-equivalence gate ran the full public cognition split twice +per proposal (active corpus vs. transient-with-append) and reported +`replay_equivalent=True` and `regressed_metrics=[]` for every one +of the four proposals. After all four accepts plus the supersede, +public split metrics are byte-identical to pre-unit: + +``` +intent_accuracy 100.0% (unchanged) +term_capture_rate 91.7% (unchanged) +surface_groundedness 100.0% (unchanged) +versor_closure_rate 100.0% (unchanged) +``` + +The supersede was operator-direct (no replay gate by design — operator +explicitly accepts the change). It also did not regress any public +metric. + +--- + +## Reproducibility + +Every action is replayable from the proposal log + corpus file alone: + +```bash +# Inspect the trail +core teaching audit # 15 lines on disk, 14 active, 1 dropped +core teaching supersessions # wisdom→grounds→judgment retired by →requires→knowledge +core teaching proposals --state accepted # the four new chains' proposal_ids + +# Confirm the lift +core eval cognition # public unchanged +core eval cognition --split holdout # holdout term_capture 75.0% +``` + +The proposal-log JSONL is append-only at `teaching/proposals/proposals.jsonl`. +Every state transition (`created` → `replay` → `transition` → +`accepted_corpus_append`) is one line; replaying the log reconstructs +the active view deterministically. + +--- + +## What this unit demonstrates + +1. **The propose/review/accept loop works end-to-end on the production + corpus** — not just in the demo trilogy. +2. **The supersede surface produces a clean retirement trail** — + `core teaching supersessions` shows the retired chain paired with + its replacement and the supersession provenance string. +3. **A four-chain batch + one supersession costs ~10 seconds of + operator wall-time** — propose is ~2s each (replay gate runs the + cognition lane), accept is sub-second. +4. **The replay-equivalence gate's no-regression guarantee held + for every proposal** — public split metrics did not move. +5. **Curriculum work can target specific measurable misses** + (the `verification_wisdom_036` case) and the lift matches the + prediction exactly. + +--- + +## Cross-References + +- [ADR-0055](../decisions/ADR-0055-inter-session-memory-discovery-promotion.md) — the inter-session-memory architecture this unit consumes. +- [ADR-0056](../decisions/ADR-0056-contemplation-loop-c1.md) — DiscoveryCandidate contemplation; the candidate JSONL in this unit was operator-authored (hand-augmented with corpus-evidence pointers) rather than emitted by the runtime, but the structure is the same. +- [ADR-0057](../decisions/ADR-0057-teaching-chain-proposal-review.md) — the proposal + replay gate + accept surface this unit uses; the supersede CLI is the follow-up at `8d2c84a`. +- [teaching_order.md](../teaching_order.md) — the prerequisite-topological doctrine that scoped this unit's chain selection. diff --git a/teaching/cognition_chains/cognition_chains_v1.jsonl b/teaching/cognition_chains/cognition_chains_v1.jsonl index 21462ff4..35eaf156 100644 --- a/teaching/cognition_chains/cognition_chains_v1.jsonl +++ b/teaching/cognition_chains/cognition_chains_v1.jsonl @@ -8,3 +8,8 @@ {"chain_id":"cause_wisdom_orders_judgment","subject":"wisdom","intent":"cause","connective":"orders","object":"judgment","domains_subject_k":2,"domains_object_k":1,"provenance":"adr-0053:reviewed:2026-05-18"} {"chain_id":"verification_truth_requires_evidence","subject":"truth","intent":"verification","connective":"requires","object":"evidence","domains_subject_k":2,"domains_object_k":1,"provenance":"adr-0053:reviewed:2026-05-18"} {"chain_id":"verification_wisdom_grounds_judgment","subject":"wisdom","intent":"verification","connective":"grounds","object":"judgment","domains_subject_k":2,"domains_object_k":1,"provenance":"adr-0053:reviewed:2026-05-18"} +{"chain_id":"verification_wisdom_requires_knowledge","connective":"requires","domains_object_k":1,"domains_subject_k":2,"intent":"verification","object":"knowledge","provenance":"adr-0057:hand_authored:2026-05-18:supersede(verification_wisdom_grounds_judgment)","subject":"wisdom","superseded_by":"verification_wisdom_grounds_judgment"} +{"chain_id":"cause_understanding_requires_knowledge","connective":"requires","domains_object_k":1,"domains_subject_k":2,"intent":"cause","object":"knowledge","provenance":"adr-0057:discovery_promoted:2026-05-18","subject":"understanding"} +{"chain_id":"cause_judgment_requires_wisdom","connective":"requires","domains_object_k":1,"domains_subject_k":2,"intent":"cause","object":"wisdom","provenance":"adr-0057:discovery_promoted:2026-05-18","subject":"judgment"} +{"chain_id":"verification_evidence_grounds_knowledge","connective":"grounds","domains_object_k":1,"domains_subject_k":2,"intent":"verification","object":"knowledge","provenance":"adr-0057:discovery_promoted:2026-05-18","subject":"evidence"} +{"chain_id":"cause_inference_requires_evidence","connective":"requires","domains_object_k":1,"domains_subject_k":2,"intent":"cause","object":"evidence","provenance":"adr-0057:discovery_promoted:2026-05-18","subject":"inference"} diff --git a/teaching/proposals/proposals.jsonl b/teaching/proposals/proposals.jsonl new file mode 100644 index 00000000..78651f7d --- /dev/null +++ b/teaching/proposals/proposals.jsonl @@ -0,0 +1,16 @@ +{"event":"created","proposal":{"claim_domain":"factual","evidence":[{"epistemic_status":"coherent","polarity":"affirms","ref":"cause_truth_grounds_knowledge","source":"corpus"}],"operator_note":"","polarity":"affirms","proposal_id":"39d09297759f5bfa54e037d089a9fd11","proposed_chain":{"connective":"requires","intent":"cause","object":"knowledge","subject":"understanding"},"provenance":null,"replay_evidence":null,"review_state":"pending","source_candidate_id":"a199f13dba127a34cb8fa4bf28728020bec932e6b55c24c93ee1ef1be8d51063"}} +{"event":"replay","proposal_id":"39d09297759f5bfa54e037d089a9fd11","replay_evidence":{"baseline":{"intent_accuracy":1.0,"surface_groundedness":1.0,"term_capture_rate":0.9167,"versor_closure_rate":1.0},"candidate":{"intent_accuracy":1.0,"surface_groundedness":1.0,"term_capture_rate":0.9167,"versor_closure_rate":1.0},"regressed_metrics":[],"replay_equivalent":true}} +{"event":"created","proposal":{"claim_domain":"factual","evidence":[{"epistemic_status":"coherent","polarity":"affirms","ref":"cause_wisdom_orders_judgment","source":"corpus"}],"operator_note":"","polarity":"affirms","proposal_id":"944c6c5180819c9194765f32dbc31714","proposed_chain":{"connective":"requires","intent":"cause","object":"wisdom","subject":"judgment"},"provenance":null,"replay_evidence":null,"review_state":"pending","source_candidate_id":"1f1a1387a4cd1cecb27dc6fb37af0e3322afd8bc23b6abeb9c9c0e5e606cf91a"}} +{"event":"replay","proposal_id":"944c6c5180819c9194765f32dbc31714","replay_evidence":{"baseline":{"intent_accuracy":1.0,"surface_groundedness":1.0,"term_capture_rate":0.9167,"versor_closure_rate":1.0},"candidate":{"intent_accuracy":1.0,"surface_groundedness":1.0,"term_capture_rate":0.9167,"versor_closure_rate":1.0},"regressed_metrics":[],"replay_equivalent":true}} +{"event":"created","proposal":{"claim_domain":"factual","evidence":[{"epistemic_status":"coherent","polarity":"affirms","ref":"cause_truth_grounds_knowledge","source":"corpus"}],"operator_note":"","polarity":"affirms","proposal_id":"a5647f07a0227d1853f926d439603f28","proposed_chain":{"connective":"grounds","intent":"verification","object":"knowledge","subject":"evidence"},"provenance":null,"replay_evidence":null,"review_state":"pending","source_candidate_id":"b568e12273ddac1c0bf6e611ae331febcffc9f4df7a47df7f575416277d288f1"}} +{"event":"replay","proposal_id":"a5647f07a0227d1853f926d439603f28","replay_evidence":{"baseline":{"intent_accuracy":1.0,"surface_groundedness":1.0,"term_capture_rate":0.9167,"versor_closure_rate":1.0},"candidate":{"intent_accuracy":1.0,"surface_groundedness":1.0,"term_capture_rate":0.9167,"versor_closure_rate":1.0},"regressed_metrics":[],"replay_equivalent":true}} +{"event":"created","proposal":{"claim_domain":"factual","evidence":[{"epistemic_status":"coherent","polarity":"affirms","ref":"cause_knowledge_requires_evidence","source":"corpus"}],"operator_note":"","polarity":"affirms","proposal_id":"bb01f8acd6f5db144bdac21a6f754dc7","proposed_chain":{"connective":"requires","intent":"cause","object":"evidence","subject":"inference"},"provenance":null,"replay_evidence":null,"review_state":"pending","source_candidate_id":"a4e1560e3e65f32d2b77496d9fe819038a9c0df2d7c8112fa8aaf6347b7b7081"}} +{"event":"replay","proposal_id":"bb01f8acd6f5db144bdac21a6f754dc7","replay_evidence":{"baseline":{"intent_accuracy":1.0,"surface_groundedness":1.0,"term_capture_rate":0.9167,"versor_closure_rate":1.0},"candidate":{"intent_accuracy":1.0,"surface_groundedness":1.0,"term_capture_rate":0.9167,"versor_closure_rate":1.0},"regressed_metrics":[],"replay_equivalent":true}} +{"event":"transition","note":"Epistemology v1: understanding requires knowledge","proposal_id":"39d09297759f5bfa54e037d089a9fd11","to":"accepted"} +{"chain_id":"cause_understanding_requires_knowledge","event":"accepted_corpus_append","proposal_id":"39d09297759f5bfa54e037d089a9fd11","provenance":{"adr_id":"adr-0057","raw":"adr-0057:discovery_promoted:2026-05-18","review_date":"2026-05-18","source":"discovery_promoted"}} +{"event":"transition","note":"Epistemology v1: judgment requires wisdom","proposal_id":"944c6c5180819c9194765f32dbc31714","to":"accepted"} +{"chain_id":"cause_judgment_requires_wisdom","event":"accepted_corpus_append","proposal_id":"944c6c5180819c9194765f32dbc31714","provenance":{"adr_id":"adr-0057","raw":"adr-0057:discovery_promoted:2026-05-18","review_date":"2026-05-18","source":"discovery_promoted"}} +{"event":"transition","note":"Epistemology v1: evidence grounds knowledge","proposal_id":"a5647f07a0227d1853f926d439603f28","to":"accepted"} +{"chain_id":"verification_evidence_grounds_knowledge","event":"accepted_corpus_append","proposal_id":"a5647f07a0227d1853f926d439603f28","provenance":{"adr_id":"adr-0057","raw":"adr-0057:discovery_promoted:2026-05-18","review_date":"2026-05-18","source":"discovery_promoted"}} +{"event":"transition","note":"Epistemology v1: inference requires evidence","proposal_id":"bb01f8acd6f5db144bdac21a6f754dc7","to":"accepted"} +{"chain_id":"cause_inference_requires_evidence","event":"accepted_corpus_append","proposal_id":"bb01f8acd6f5db144bdac21a6f754dc7","provenance":{"adr_id":"adr-0057","raw":"adr-0057:discovery_promoted:2026-05-18","review_date":"2026-05-18","source":"discovery_promoted"}}