632a69db40
Phase 2's second lane: after N teaching cycles in unrelated domains,
competence on previously-taught domains must not regress. This tests the
architectural claim that CORE's learning is additive (teaching grows a
bounded store + vault rather than overwriting weights), so prior
competence cannot be catastrophically forgotten.
Protocol per split:
cycle 0: probe all domains (baseline)
cycle 1..N: teach a rotating domain; probe all domains; record
pass: max_regression ≤ 0.05, floor_score ≥ 0.80, cycle_count ≥ 10
Components:
- evals/monotonic_learning/{contract.md, runner.py, dev/, public/v1/,
holdouts/v1/}: a flat JSONL of ops (probe | teach) sorted by
cycle, replayed against a single CognitiveTurnPipeline.
- scripts/generate_monotonic_cases.py: regenerates the cycle/probe
corpora deterministically per split.
Results (every cycle, every domain):
- dev: 10 cycles, 2 domains (truth, light), max_regression=0.00,
floor_score=1.00.
- public/v1: 12 cycles, 3 domains (truth, light, wisdom),
max_regression=0.00, floor_score=1.00.
- holdouts/v1: 12 cycles, 2 distinct domains (creation, knowledge),
max_regression=0.00, floor_score=1.00.
Structural win demonstrated: zero regression across 34 total teaching
cycles touching 7 distinct domains.
PROGRESS.md updated to mark monotonic-learning v1 complete.
54 lines
6.4 KiB
JSON
54 lines
6.4 KiB
JSON
{"cycle": 0, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 0, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 0, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 0, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 1, "op": "teach", "domain": "truth", "prime": ["What is truth?"], "prompt": "Actually truth is more than that."}
|
|
{"cycle": 1, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 1, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 1, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 1, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 2, "op": "teach", "domain": "light", "prime": ["What is light?"], "prompt": "Actually light is more than that."}
|
|
{"cycle": 2, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 2, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 2, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 2, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 3, "op": "teach", "domain": "truth", "prime": ["What is truth?"], "prompt": "No, truth requires deeper understanding."}
|
|
{"cycle": 3, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 3, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 3, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 3, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 4, "op": "teach", "domain": "light", "prime": ["What is light?"], "prompt": "No, light requires deeper understanding."}
|
|
{"cycle": 4, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 4, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 4, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 4, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 5, "op": "teach", "domain": "truth", "prime": ["What is truth?"], "prompt": "Actually truth relates to coherence."}
|
|
{"cycle": 5, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 5, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 5, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 5, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 6, "op": "teach", "domain": "light", "prime": ["What is light?"], "prompt": "Actually light relates to coherence."}
|
|
{"cycle": 6, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 6, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 6, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 6, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 7, "op": "teach", "domain": "truth", "prime": ["What is truth?"], "prompt": "Actually truth is more than that."}
|
|
{"cycle": 7, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 7, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 7, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 7, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 8, "op": "teach", "domain": "light", "prime": ["What is light?"], "prompt": "Actually light is more than that."}
|
|
{"cycle": 8, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 8, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 8, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 8, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 9, "op": "teach", "domain": "truth", "prime": ["What is truth?"], "prompt": "No, truth requires deeper understanding."}
|
|
{"cycle": 9, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 9, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 9, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 9, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|
|
{"cycle": 10, "op": "teach", "domain": "light", "prime": ["What is light?"], "prompt": "No, light requires deeper understanding."}
|
|
{"cycle": 10, "op": "probe", "domain": "truth", "id": "PT-1", "prompt": "What is truth?", "expected_terms": ["truth"]}
|
|
{"cycle": 10, "op": "probe", "domain": "truth", "id": "PT-2", "prompt": "Is truth coherent?", "expected_terms": ["truth"]}
|
|
{"cycle": 10, "op": "probe", "domain": "light", "id": "PL-1", "prompt": "What is light?", "expected_terms": ["light"]}
|
|
{"cycle": 10, "op": "probe", "domain": "light", "id": "PL-2", "prompt": "Why does light reveal?", "expected_terms": ["light"]}
|