core/evals
Shay 9e1add43a1 feat(phase4): long-context-cost lane + ADR-0019 Stage 1 vault recall vectorisation
Phase 4 lane #2 (long_context_cost) measured vault.recall latency
as a function of vault size N. The pre-vectorisation curve was
median 875 ms at N=1k, ~9 s at N=10k — unfit for runtime use.

ADR-0019 Stage 1 replaces the per-element Python dispatch loop in
algebra/backend.py::vault_recall with a vectorised exact scan over
the diagonal Cl(4,1) CGA inner-product metric. Per-versor serial
component reduction order is preserved, so scores are bit-identical
to the scalar cga_inner path. CLAUDE.md exactness is preserved; no
approximate recall is introduced.

Post-vectorisation: 0.217 ms at N=1k, 20.795 ms at N=100k. Slope
0.99 (linear). ~4,000-5,000x speedup at every probed N. Smoke,
algebra, and runtime suites all green.

Stages 2 (norm-bucketed exact pre-filter) and 3 (layered store
with deterministic promotion) are documented in ADR-0019 but
deferred — Stage 1 has dissolved the bottleneck at the scales
relevant to current curriculum work.
2026-05-16 16:39:30 -07:00
..
adversarial_identity docs(identity): empirical finding — fix #3 needs upstream ingest-gate work 2026-05-16 14:23:20 -07:00
calibration feat(evals): v2 lanes for calibration and symbolic-logic 2026-05-16 13:17:41 -07:00
cognition feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
compositionality feat(phase3): compositionality, multi-step-reasoning, introspection, cross-domain-transfer v1 2026-05-16 14:48:36 -07:00
cross_domain_transfer feat(phase3): compositionality, multi-step-reasoning, introspection, cross-domain-transfer v1 2026-05-16 14:48:36 -07:00
grammatical_coverage feat(evals): grammatical-coverage v2 cases - 36 cases with deeper nesting and rarer vocabulary (100% pass) 2026-05-16 06:40:55 -07:00
identity_divergence feat(evals): identity-divergence lane v1 - 93 curriculum events, two axis profiles (Precision/Generosity), divergence/coherence/causal metrics (all pass) 2026-05-16 06:48:13 -07:00
inference_closure feat(phase3): inference-closure lane v1 — foundation OK, no operator 2026-05-16 14:33:08 -07:00
introspection feat(phase3): core/cognition/explain.py — close Gap 3 introspection 2026-05-16 15:09:48 -07:00
long_context_cost feat(phase4): long-context-cost lane + ADR-0019 Stage 1 vault recall vectorisation 2026-05-16 16:39:30 -07:00
monotonic_learning feat(evals): v3 lanes — monotonic-learning passes, adversarial-identity reveals gap 2026-05-16 13:42:47 -07:00
multi_step_reasoning feat(phase3): compositionality, multi-step-reasoning, introspection, cross-domain-transfer v1 2026-05-16 14:48:36 -07:00
provenance feat(evals): parallel runner + adversarial-identity v2 2026-05-16 13:10:26 -07:00
reports
sample_efficiency feat(phase4): sample-efficiency v1 — first quantitative-curve lane 2026-05-16 15:39:28 -07:00
symbolic_logic feat(evals): v2 lanes for calibration and symbolic-logic 2026-05-16 13:17:41 -07:00
zero_code_domain_acquisition feat began creation of zero code domain acquisition. did not complete yet. 2026-05-16 06:31:01 -07:00
__init__.py
baseline_runner.py feat(evals): frontier structural-zero baselines for Phase 2 v1 lanes 2026-05-16 12:45:28 -07:00
cognition_cases.jsonl
framework.py feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
holdout_runner.py feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
metrics.py
parallel.py feat(evals): parallel runner + adversarial-identity v2 2026-05-16 13:10:26 -07:00
run_cognition_eval.py