core/evals
Shay ac77b88864
chore(ratify): accept four Phase C round-2 recognizers (round 2) (#309)
* chore(ratify): accept four Phase C round-2 recognizers (round 2)

Operator ratification of the four Phase B round-2 proposals per
ADR-0163:

- 8c7645b4 — discrete_count_statement
- 03627f6f — multiplicative_aggregation
- 00547671 — currency_amount
- 4d47a247 — temporal_aggregation (v2 widening)

All four passed Phase C's admissibility replay gate at propose-time:
replay_equivalent=True, wrong_count_delta=0.  Each acceptance also
appends the synthetic admissibility chain to teaching/cognition_chains.

Post-ratification empirical signal (verified by running the
train_sample lane):
- correct: 3 (unchanged)
- refused: 47 (unchanged)
- wrong: 0 (unchanged — invariant holds)

The case-level lift did not materialize because the architectural
bottleneck migrated from STATEMENT admission to QUESTION admission.
44 of 47 cases now refuse on a QUESTION (vs 7 pre-ratification).
The four new recognizers' matchers fire on 36 of 47 first-failed
sentences, but the cases then refuse on a different (later)
sentence — typically the question itself.

The unlock for this round is Phase D.3 (conditional-prefix question
recovery, PR #308) + a follow-up parser-grammar extension to handle
mass nouns (how much), modal verbs (will be able to), and pronoun
entity resolution.  Those touch grammar surface, not admission
wiring; separate ADR.

This PR commits the ratification audit trail.  The lift composes
when Phase D.3 lands and the grammar layer follows.

wrong=0 invariant: preserved by Phase D's skip-only construction.
Statement-level recognizer matches contribute zero math state to
the Cartesian product; no recognizer can introduce a wrong answer
under skip-only semantics.

Cross-references: ADR-0163, Phase A PR #297, Phase B round 1 PR
#298, Phase C PR #301, Phase D PR #302, ratify round-1 PR #304,
docs PR #305, Phase B round 2 PR #306, Phase C round-2 extension
PR #307, Phase D.3 PR #308.

* chore(ratify): re-pin public_demo lane SHA after round-2 ratification

The four round-2 ratifications appended synthetic admissibility
chains to teaching/cognition_chains/cognition_chains_v1.jsonl,
which is consumed by the public_demo lane.  The lane's deterministic
output SHA changed accordingly — drift confirmed by CI on origin
PR #309 (`✗ public_demo  e323adb35ea17987..  expected 888ddd0d12635d70..`).

Re-pin per the standard remediation:

  python scripts/verify_lane_shas.py --update
  python scripts/generate_claims.py

This is the expected corpus-mutation cycle following ratification.
No code change, no test change.  The new public_demo SHA reflects
the engine's new admissibility surface; the lane runner's output
is byte-stable under the new corpus.

Cross-references: ratify round-2 PR #309 (this branch), Phase D
PR #302, Phase C PR #301.
2026-05-26 16:03:01 -07:00
..
adversarial_identity
anchor_lens_tour
anti_regression feat(epistemic): Phase 3 state tagging spine (#220) 2026-05-24 11:26:06 -07:00
articulation
articulation_of_status
audit_tour
calibration
classical_literature_ood
cognition
cold_start_grounding fix(quarantine): drain all 60 quarantined tests — QUARANTINE=∅ (#267) 2026-05-25 11:22:12 -07:00
compositionality
compound_intent_decomposition
contemplation_quality feat(W-025): contemplation quality eval lane (ADR-0159) (#286) 2026-05-25 20:38:52 -07:00
contradiction_detection
conversation
conversational_thread_coherence
cross_domain_transfer
curriculum_loop_closure feat(evals): ADR-0105 — sealed holdout encryption via age (#108) 2026-05-22 10:09:43 -07:00
demo_composition fix(W-015): replace _slerp_toward with rotor-geodesic anchor pull (#255) 2026-05-24 20:05:25 -07:00
deterministic_fluency
discourse_paragraph
domain_contract_validation chore(ci): re-pin drifted lane SHAs + refresh canonical reports (#229) 2026-05-24 14:25:11 -07:00
elementary_mathematics_ood feat(capability): ADR-0110 promote mathematics_logic to expert_demo (#118) 2026-05-22 12:59:23 -07:00
english_fluency_ood
fabrication_control chore(ci): re-pin drifted lane SHAs + refresh canonical reports (#229) 2026-05-24 14:25:11 -07:00
forward_semantic_control
foundational_biology_ood
foundational_physics_ood feat: ADR-0111 physics expert-demo promotion (second successful) 2026-05-22 14:37:36 -07:00
frontier_compare feat(contemplation): land ADR-0080 phase 1 (#119) 2026-05-22 13:10:03 -07:00
grammatical_coverage
gsm8k_math fix(quarantine): drain all 60 quarantined tests — QUARANTINE=∅ (#267) 2026-05-25 11:22:12 -07:00
gsm8k_parser_dev feat: add ADR-0125 perturbation suite 2026-05-22 17:12:33 -07:00
hebrew_fluency feat(packs): ADR-0103 — attach hebrew_fluency + koine_greek_fluency lanes to ADR-0102 (#106) 2026-05-22 09:43:46 -07:00
identity_divergence
industry_demos evals/industry_demos: add run_all.py suite runner 2026-05-21 08:23:29 -07:00
inference_closure feat: ADR-0122 systems_software audit-passed deferred (lane-shape mismatch) 2026-05-22 16:31:59 -07:00
introspection
koine_greek_fluency feat(packs): ADR-0103 — attach hebrew_fluency + koine_greek_fluency lanes to ADR-0102 (#106) 2026-05-22 09:43:46 -07:00
lab
learning_arc fix(W-025): polish contemplation-quality eval lane follow-ups (#290) 2026-05-26 09:39:18 -07:00
learning_loop
long_context_cost
math_bounded_grammar/v1 feat(ADR-0131.3): bounded-grammar word-problem benchmark — lane PASSED 50/50 (#180) 2026-05-23 11:27:04 -07:00
math_capability_axes feat(ADR-0136.S.4): novel-initial-form parser extension + rescan v4 (#210) 2026-05-23 22:34:51 -07:00
math_expert_claims/v1 fix(quarantine): drain all 60 quarantined tests — QUARANTINE=∅ (#267) 2026-05-25 11:22:12 -07:00
math_symbolic_equivalence feat(ADR-0131.1.F): frontier-baseline comparison harness for B1 (#178) 2026-05-23 12:14:06 -07:00
math_teaching_corpus/v1 feat(ADR-0131.2.B): B2 teaching-corpus enrichment — load-bearing gate (#177) 2026-05-23 11:29:48 -07:00
miner_loop_closure feat(teaching): implement ADR-0095 — Miner-Sourced Teaching Proposals 2026-05-21 18:18:51 -07:00
monotonic_learning
multi_agent_composition
multi_sentence_response
multi_step_reasoning
obligation_2_ood_ratio feat(ADR-0114a.2): OOD-ratio auditor — Obligation #2 wired for B3, ratio=1.00 (#193) 2026-05-23 16:25:28 -07:00
obligation_5_perturbation feat(ADR-0114a.5): reasoning-isolation perturbation suite — Obligation #5 wired for B3, PASSING 130/130 preserving, 68/68 breaking (#191) 2026-05-23 16:07:59 -07:00
obligation_6_depth_curve feat(ADR-0114a.6): depth-curve auditor — Obligation #6 wired for B3 (assertion holds, coverage gap named) (#190) 2026-05-23 16:19:58 -07:00
obligation_8_adversarial feat(ADR-0114a.8): adversarial auditor — Obligation #8 wired, PASSING; surfaces 2 known parser-layer gaps (#192) 2026-05-23 16:11:37 -07:00
obligation_10_pack_provenance feat(ADR-0114a.10): pack-provenance auditor — Obligation #10 wired for B3, PASSING 2026-05-23 15:44:53 -07:00
orthogonality_tour
prompt_diversity feat(adr-0085): gloss-aware CAUSE composer — explanation frame from glosses (#70) 2026-05-20 15:55:08 -07:00
provenance
public_demo chore(ratify): accept four Phase C round-2 recognizers (round 2) (#309) 2026-05-26 16:03:01 -07:00
realizer_guard
refusal_calibration
refusal_taxonomy feat(ADR-0163.B.2): four new exemplar corpora — discrete_count_statement, multiplicative_aggregation, currency_amount, plus temporal_aggregation v2 widening (#306) 2026-05-26 14:36:59 -07:00
register_diagnostics feat(evals): per-intent register-firing diagnostic + CI gate + tests (#103) 2026-05-21 07:05:23 -07:00
register_tour
reports
results feat: ADR-0086 + ADR-0087 + 100-register catalog — cognition lane closure 2026-05-21 00:08:12 -07:00
reviewer_registry feat(capability): implement ADR-0092 — Reviewer Registry v1 2026-05-21 18:01:24 -07:00
sample_efficiency
self_consistency_over_time
symbolic_logic feat: ADR-0122 systems_software audit-passed deferred (lane-shape mismatch) 2026-05-22 16:31:59 -07:00
teaching_injection_resistance
walkthrough_chain
warmed_session_consistency
zero_code_domain_acquisition
__init__.py
_parallel.py
baseline_runner.py
CLAIMS.md
cognition_cases.jsonl
framework.py feat(evals): ADR-0105 — sealed holdout encryption via age (#108) 2026-05-22 10:09:43 -07:00
holdout_runner.py feat(evals): ADR-0105 — sealed holdout encryption via age (#108) 2026-05-22 10:09:43 -07:00
metrics.py
parallel.py
run_cognition_eval.py feat(W-008): L10 Shape B hybrid engine-state persistence (#271) 2026-05-25 11:45:54 -07:00