core/evals
Shay ffe439c889
chore(ci): re-pin drifted lane SHAs + refresh canonical reports (#229)
Three lane SHA pins drifted because intentional surface/serialization
changes shipped without re-running scripts/verify_lane_shas.py --update.

Bisect attributing the drift:
- demo_composition + public_demo broke at 5cad0a4 (#118 ADR-0110
  mathematics_logic → expert_demo) — the demos enumerate the expert set.
- demo_composition drifted a second time at ab4c7cb (#220 Phase 3
  state tagging spine) — additional epistemic fields shifted the surface.
- domain_contract_validation broke at a45eab1 (#219 Phase 2 epistemic
  bug repairs) — normative/epistemic field shape changed.

The in-tree canonical report for fabrication_control_summary was also
stale vs. its (correct) pin; refreshed here for byte-alignment.

After this commit: 7/7 lanes match pinned SHAs; verify_lane_shas.py
runs green locally and in CI.

Followup (separate PR): hook/template guard so future PRs that touch
core/cognition/result.py, chat/runtime.py, or capability registries
re-run --update before merge.
2026-05-24 14:25:11 -07:00
..
adversarial_identity
anchor_lens_tour
anti_regression feat(epistemic): Phase 3 state tagging spine (#220) 2026-05-24 11:26:06 -07:00
articulation
articulation_of_status
audit_tour
calibration
classical_literature_ood
cognition
cold_start_grounding
compositionality
compound_intent_decomposition
contradiction_detection
conversation
conversational_thread_coherence
cross_domain_transfer
curriculum_loop_closure
demo_composition chore(ci): re-pin drifted lane SHAs + refresh canonical reports (#229) 2026-05-24 14:25:11 -07:00
deterministic_fluency
discourse_paragraph
domain_contract_validation chore(ci): re-pin drifted lane SHAs + refresh canonical reports (#229) 2026-05-24 14:25:11 -07:00
elementary_mathematics_ood
english_fluency_ood
fabrication_control chore(ci): re-pin drifted lane SHAs + refresh canonical reports (#229) 2026-05-24 14:25:11 -07:00
forward_semantic_control
foundational_biology_ood
foundational_physics_ood feat: ADR-0111 physics expert-demo promotion (second successful) 2026-05-22 14:37:36 -07:00
frontier_compare
grammatical_coverage
gsm8k_math fix(epistemic): Phase 2 known bug repairs (#219) 2026-05-24 11:20:32 -07:00
gsm8k_parser_dev feat: add ADR-0125 perturbation suite 2026-05-22 17:12:33 -07:00
hebrew_fluency
identity_divergence
industry_demos
inference_closure feat: ADR-0122 systems_software audit-passed deferred (lane-shape mismatch) 2026-05-22 16:31:59 -07:00
introspection
koine_greek_fluency
lab
learning_loop
long_context_cost
math_bounded_grammar/v1 feat(ADR-0131.3): bounded-grammar word-problem benchmark — lane PASSED 50/50 (#180) 2026-05-23 11:27:04 -07:00
math_capability_axes feat(ADR-0136.S.4): novel-initial-form parser extension + rescan v4 (#210) 2026-05-23 22:34:51 -07:00
math_expert_claims/v1 feat(ADR-0120 math, ledger flip): mathematics_logic → expert tier (first-ever) (#195) 2026-05-23 18:55:34 -07:00
math_symbolic_equivalence feat(ADR-0131.1.F): frontier-baseline comparison harness for B1 (#178) 2026-05-23 12:14:06 -07:00
math_teaching_corpus/v1 feat(ADR-0131.2.B): B2 teaching-corpus enrichment — load-bearing gate (#177) 2026-05-23 11:29:48 -07:00
miner_loop_closure
monotonic_learning
multi_agent_composition
multi_sentence_response
multi_step_reasoning
obligation_2_ood_ratio feat(ADR-0114a.2): OOD-ratio auditor — Obligation #2 wired for B3, ratio=1.00 (#193) 2026-05-23 16:25:28 -07:00
obligation_5_perturbation feat(ADR-0114a.5): reasoning-isolation perturbation suite — Obligation #5 wired for B3, PASSING 130/130 preserving, 68/68 breaking (#191) 2026-05-23 16:07:59 -07:00
obligation_6_depth_curve feat(ADR-0114a.6): depth-curve auditor — Obligation #6 wired for B3 (assertion holds, coverage gap named) (#190) 2026-05-23 16:19:58 -07:00
obligation_8_adversarial feat(ADR-0114a.8): adversarial auditor — Obligation #8 wired, PASSING; surfaces 2 known parser-layer gaps (#192) 2026-05-23 16:11:37 -07:00
obligation_10_pack_provenance feat(ADR-0114a.10): pack-provenance auditor — Obligation #10 wired for B3, PASSING 2026-05-23 15:44:53 -07:00
orthogonality_tour
prompt_diversity
provenance
public_demo chore(ci): re-pin drifted lane SHAs + refresh canonical reports (#229) 2026-05-24 14:25:11 -07:00
realizer_guard
refusal_calibration
register_diagnostics
register_tour
reports
results
reviewer_registry
sample_efficiency
self_consistency_over_time
symbolic_logic feat: ADR-0122 systems_software audit-passed deferred (lane-shape mismatch) 2026-05-22 16:31:59 -07:00
teaching_injection_resistance
walkthrough_chain
warmed_session_consistency
zero_code_domain_acquisition
__init__.py
_parallel.py
baseline_runner.py
CLAIMS.md
cognition_cases.jsonl
framework.py
holdout_runner.py
metrics.py
parallel.py
run_cognition_eval.py