Adds the fourth Phase 2 lane. v1 measures the structural foundations
on which a future inference engine would be built:
M1. premise_recall — probe vault_hits >= min after chain teaching
M2. replay_determinism — same chain + probe → same trace_hash
M3. proposal_storage — correction premises store as PackMutationProposals
Patterns covered: modus_ponens_chain, modus_tollens_chain, syllogism,
negation, chain_recall (up to 4-hop chains).
v1 results across 38 cases (8 dev + 18 public + 12 holdouts):
premise_recall=1.0, replay_determinism=1.0, proposal_storage=1.0.
Each case runs twice on fresh CognitiveTurnPipelines to verify the
trace_hash matches — confirming deterministic replay over premise chains.
Architectural finding logged in evals/symbolic_logic/gaps.md:
CORE has no first-class inference operator. Chain "inference" today is
emergent from teaching-store commits + cumulative vault recall, not a
named-rule symbolic engine. v1 honestly tests what CORE deterministically
*does* (store, replay, recall chains) without overclaiming that CORE
reasons symbolically. v2 would assert specific transitive recall
contents in the probe surface, which requires either a
PropositionGraph traversal operator or pack-axiom rules — both filed
as suggested follow-up work.
Adds the third Phase 2 lane: calibration measures whether CORE's runtime
emits distinguishable, typed evidence for three cognitive states:
no_grounding vault_hits == 0 (gate fired, no recall)
coherent vault_hits > 0 (vault recall fired)
correction_proposed pack_mutation_proposal is not None
Each case runs on its own fresh CognitiveTurnPipeline to avoid
cross-case field-state drift (the gate's geometric recall score is
sensitive to vault content drift across turns).
v1 results: dev 12/12, public/v1 24/24, holdouts/v1 18/18 — all classes
score 1.0 across all splits.
Architectural findings logged in evals/calibration/gaps.md:
1. The ingest gate fires on a *geometric* CGA-recall score, not on
semantic OOD. 6/42 hand-chosen OOD prompts fire the gate with a
warmed vault; the other 36 land geometrically near in-pack
versors after morphological grounding. v1 measures the reliable
recall/correction signals, not semantic OOD detection.
2. CognitiveTurnPipeline.run() unconditionally overrides the
runtime's gate-safety surface with the realizer surface. The OOD
marker survives in walk_surface but not in surface. v1 classifies
on vault_hits (preserved) rather than surface (overridden).
Both findings are filed as suggested follow-up work, not v1 blockers.
Phase 2's second lane: after N teaching cycles in unrelated domains,
competence on previously-taught domains must not regress. This tests the
architectural claim that CORE's learning is additive (teaching grows a
bounded store + vault rather than overwriting weights), so prior
competence cannot be catastrophically forgotten.
Protocol per split:
cycle 0: probe all domains (baseline)
cycle 1..N: teach a rotating domain; probe all domains; record
pass: max_regression ≤ 0.05, floor_score ≥ 0.80, cycle_count ≥ 10
Components:
- evals/monotonic_learning/{contract.md, runner.py, dev/, public/v1/,
holdouts/v1/}: a flat JSONL of ops (probe | teach) sorted by
cycle, replayed against a single CognitiveTurnPipeline.
- scripts/generate_monotonic_cases.py: regenerates the cycle/probe
corpora deterministically per split.
Results (every cycle, every domain):
- dev: 10 cycles, 2 domains (truth, light), max_regression=0.00,
floor_score=1.00.
- public/v1: 12 cycles, 3 domains (truth, light, wisdom),
max_regression=0.00, floor_score=1.00.
- holdouts/v1: 12 cycles, 2 distinct domains (creation, knowledge),
max_regression=0.00, floor_score=1.00.
Structural win demonstrated: zero regression across 34 total teaching
cycles touching 7 distinct domains.
PROGRESS.md updated to mark monotonic-learning v1 complete.
- grammatical-coverage holdout v1: 52 cases across all 13 constructions, 100% pass
- zero-code-domain-acquisition lane: contract + 3 surprise domains (kinship,
calendar, color) with vocabulary, relations, axioms, teaching examples,
and dev prompts; pack closure verified for all three domains
- he_core_cognition_v1: 20 entries in Hebrew script with morphology decomposition
(triliteral roots, binyanim, aspect/person/gender/number); depth_root role
with fail_closed OOV policy
- grc_logos_cognition_v1: 20 entries in polytonic Greek with morphology
decomposition (stems, prefix/suffix chains, declension class, tense/voice/
mood/person); depth_relation role with fail_closed OOV policy
Establish the grammatical-coverage eval lane with 13 English v1
constructions (simple declarative, negation, conjunction, disjunction,
embedded clause, relative clause, quantification, tense, aspect).
- contract.md with scoring rubric and pass thresholds
- runner.py conforming to framework interface
- dev set: 41 cases (baseline: 24.4%, only C01/C10 pass)
- public v1: 36 cases (baseline: 19.4%, only C01/C10 pass)
- holdout and realizer engineering are next
The realizer currently handles only simple present-tense SVO declaratives.
Negation, conjunction, embedding, quantification, tense, and aspect all
need engineering work.
The top-level --version flag (bool) collided with eval's --version argument
(string). Rename the top-level dest to print_version so both coexist.
Also mark Phase 0 exit gate as complete in PROGRESS.md:
- v1 public: 13/13 (100% all metrics)
- holdout: 19/19 (unsealed plaintext, encryption deferred)
- baseline: scaffold with pluggable BaselineModel protocol
Implement the eval infrastructure defined in ADR-0016 before building new
eval lanes. This establishes the discipline that governs the entire
capability roadmap.
- Generic eval framework (evals/framework.py): lane discovery, versioned
scoring, result persistence
- Cognition lane retrofitted into new convention: 45 cases split into
stratified dev (13) / public v1 (13) / holdout (19) sets with contract,
runner, and recorded results
- Generalized `core eval <lane>` CLI: dynamic lane discovery, --list,
--version, --split, --save, --json flags
- Holdout runner scaffold: plaintext fallback, encryption interface ready
- Baseline runner scaffold: pluggable frontier model interface
- Fix: CognitiveTurnPipeline.run() crashed on turn_log[-1] when the
unknown-domain gate returned a stub without appending to turn_log
- ADR-0016, eval_methodology.md, PROGRESS.md, capability gates session log
Phase 0 exit audit found two methodology issues:
1. Pipeline turn_log crash (fixed here)
2. Versor drift in multi-turn sessions (pre-existing, under investigation)
- docs/decisions/ADR-0008-allocation-physics.md
Formalizes salience, attention, inhibition, and coherence-budget
as the allocation physics of cognition. Replaces attention-as-weights
with attention-as-field-curvature over the versor manifold.
- docs/decisions/ADR-0009-compositional-physics.md
Defines temporal binding, digest cycles, reasoning trajectories,
and articulation planning as the compositional physics layer —
how CORE assembles pressure into structured thought and output.
- docs/decisions/ADR-0010-identity-physics.md
Establishes IdentityManifold, DriveGradientMap, ExertionMeter,
and CharacterProfile as structural identity primitives. Identity
is a field over the geometry, not a prompt veneer. Grounded in
John 1:1–2 and the Logos theology that anchors the architecture.
- docs/architecture/MIND-PHYSICS-BLUEPRINT.md
Integration blueprint showing how allocation → compositional →
identity physics layers compose into the full cognitive cycle.
- core/physics/ (11 Python interface stubs)
SalienceOperator, AttentionOperator, InhibitionOperator,
BindingFrame, DigestCycle, ReasoningTrajectory,
ArticulationPlanner, DriveGradientMap, ExertionMeter,
IdentityManifold, CharacterProfile — all typed, all frozen
where stateless, all carrying explicit field contracts.
Third Door: no off-the-shelf cognitive architecture borrowed.
All operators defined from the geometry up.
