core-labs/core
core-labs/core
1
0
Fork 0
Code Issues 11 Pull requests 2 Projects Releases Packages Wiki Activity Actions
core/evals
History
Shay 6b25069da8 feat(adr-0054): vault recall indexing/batching + holdout split wired 
Two doctrine-aligned CLAUDE.md items closed together.

Part 1 — vault indexing + batching (item #4):
- VaultStore lazy _matrix_cache (invalidated on store / reproject /
  eviction); vault_recall(prebuilt_matrix=...) skips deque→ndarray
  rebuild on hot path
- New vault_recall_batch + VaultStore.recall_batch — B queries
  scored in one component-serial sweep, bit-identical to per-query
  vault_recall (3 seeds × 7 queries × N=137 parity test)
- No approximation, no hot-path repair, scoring arithmetic
  unchanged

Part 2 — holdout split wired:
- LaneInfo.holdout_cases_path resolves plaintext holdouts in fixed
  priority; sealed (.age) holdouts stay in holdout_runner
- framework.run_lane(split="holdout") + argparse --split choices
- First official cognition holdout numbers: 19 cases, intent 100%,
  surface 94.7%, term_capture 70.8%, versor 100% — single miss is
  predicted correction_truth_040 (ADR-0053 scope-limit)

Tests: 21 new vault tests + 10 new framework tests. Lanes: smoke
67, cognition 121, runtime 19, teaching 17, packs 6, algebra 132 —
all green. versor_condition < 1e-6 invariant preserved.
2026-05-18 07:58:57 -07:00
..
adversarial_identity
articulation_of_status feat(epistemic): realizer-side closure — refusal_calibration + articulation_of_status graduate 2026-05-17 10:12:59 -07:00
audit_tour feat(adr-0042): audit-tour demo — pack-layer story in four scenes 2026-05-17 22:06:45 -07:00
calibration fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
classical_literature_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
cognition feat(adr-0053): cognition lane closure — corpus expansion + CORRECTION acknowledgement 2026-05-18 07:43:39 -07:00
compositionality feat(compositionality): compose_relations operator lifts lane 68.8% → 100% 2026-05-16 22:44:06 -07:00
contradiction_detection feat(epistemic): contradiction coherence checker — CONTESTED transitions wired, last Tier 4.5 row closes 2026-05-17 10:36:48 -07:00
cross_domain_transfer feat(algebra): null-preserving versor_apply path + un-skip 2 invariant tests 2026-05-16 21:40:37 -07:00
discourse_paragraph feat(compositionality): compose_relations operator lifts lane 68.8% → 100% 2026-05-16 22:44:06 -07:00
elementary_mathematics_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
english_fluency_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
forward_semantic_control docs(cli): self-explanatory demos — preambles + per-directory READMEs 2026-05-17 16:39:50 -07:00
foundational_biology_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
foundational_physics_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
grammatical_coverage fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
hebrew_fluency
identity_divergence feat(adr-0043): Phase-2 pack measurements — claims → numbers 2026-05-17 22:19:24 -07:00
industry_demos fix(adr-0046): make forward-graph-constraint branch mergeable 2026-05-18 05:57:46 -07:00
inference_closure feat(algebra): null-preserving versor_apply path + un-skip 2 invariant tests 2026-05-16 21:40:37 -07:00
introspection
koine_greek_fluency
long_context_cost feat(adr-0044, adr-0045): domain ethics pack + long-context comparison 2026-05-17 22:31:47 -07:00
monotonic_learning
multi_agent_composition fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
multi_step_reasoning feat(algebra): null-preserving versor_apply path + un-skip 2 invariant tests 2026-05-16 21:40:37 -07:00
provenance
refusal_calibration feat(adr-0043): Phase-2 pack measurements — claims → numbers 2026-05-17 22:19:24 -07:00
reports feat(adr-0023): Forward Semantic Control proof evidence — Accepted 2026-05-17 12:55:19 -07:00
results feat(adr-0043): Phase-2 pack measurements — claims → numbers 2026-05-17 22:19:24 -07:00
sample_efficiency
symbolic_logic
teaching_injection_resistance feat(epistemic): truth-seeking schema audit — 3 leaks closed, 4 new lanes, 3 new invariants 2026-05-17 07:27:41 -07:00
zero_code_domain_acquisition
__init__.py
baseline_runner.py
CLAIMS.md feat(bench): bench cost — $/1000 turns + latency, with disclosed assumptions 2026-05-17 10:53:08 -07:00
cognition_cases.jsonl
framework.py feat(adr-0054): vault recall indexing/batching + holdout split wired 2026-05-18 07:58:57 -07:00
holdout_runner.py
metrics.py
parallel.py
run_cognition_eval.py