core/evals
Shay 5cad0a4b72
feat(capability): ADR-0110 promote mathematics_logic to expert_demo (#118)
First worked expert-demo promotion under the ADR-0106 + ADR-0109
contract. Math is now the first domain at expert_demo=true.

Signed claim (docs/reviewers.yaml):
  domain_id: mathematics_logic
  evidence_lanes: [elementary_mathematics_ood, inference_closure,
                   fabrication_control]
  evidence_revision: adr-0110:reviewed:2026-05-22
  signed_by: shay-j
  claim_digest: 94d74781e103854230c1a71590e4df2287f5d2e87832f1c29b8ec4618853c04b

Evidence (all three lanes, public + holdout):
  elementary_mathematics_ood: accuracy=1.0 (117/117 public, 39/39 holdout)
  inference_closure: all_pass_rate=1.0, replay_determinism=1.0,
                     overall_pass=True (20 public, 12 holdout)
  fabrication_control: by-class refusals 3/3/3, fabricated=0
                       (9 public, 9 holdout)

Infrastructure bridges (not contract changes):
- cases_plaintext.jsonl dev-mode fallback files for
  elementary_mathematics_ood + inference_closure (ADR-0105 pattern)
- 9 new holdout cases for fabrication_control across all three
  refusal classes (phantom_endpoint / cross_pack_non_bridge /
  sibling_collapse)
- core/capability/reporting.py: _fetch_lane_split folds top-level
  by_class into metrics so refusal_shape sees a canonical layout

Tests:
- tests/test_adr_0110_math_expert_demo.py: 4 invariant tests
  (math_expert_demo_holds, signed_claim_present, replay_digest_
  byte_equality, other_domains_unaffected)
- tests/test_adr_0107_deferral.py retired (deferral resolved)
- tests/test_expert_demo_contract.py: production-ledger test
  rewritten as 'every promoted domain has signed claim' (load-
  bearing invariant preserved)
- tests/test_capability_reports.py: math row asserted at
  expert-demo (was reasoning-capable)

Ledger state:
  systems_software: reasoning-capable
  mathematics_logic: EXPERT-DEMO   <- new
  physics: reasoning-capable
  hebrew_greek_textual_reasoning: reasoning-capable
  philosophy_theology: reasoning-capable

README updated. ADR-0107 referenced as resolved by this ADR.
CLAIMS.md regenerated. ADR-0106 / ADR-0109 contract unchanged.
2026-05-22 12:59:23 -07:00
..
adversarial_identity
anchor_lens_tour parallel eval runner (#46) 2026-05-19 23:51:59 -07:00
anti_regression chore(evals): contracts + bench json + Lane B viewer + chart + audit + demo schema (#62) 2026-05-20 13:53:13 -07:00
articulation
articulation_of_status
audit_tour
calibration
classical_literature_ood
cognition parallel eval runner (#46) 2026-05-19 23:51:59 -07:00
cold_start_grounding
compositionality
compound_intent_decomposition
contradiction_detection
conversation
conversational_thread_coherence
cross_domain_transfer
curriculum_loop_closure feat(evals): ADR-0105 — sealed holdout encryption via age (#108) 2026-05-22 10:09:43 -07:00
demo_composition feat(demos): implement ADR-0098 — Demo Composition Contract 2026-05-21 19:02:29 -07:00
deterministic_fluency
discourse_paragraph
domain_contract_validation feat(capability): implement ADR-0093 — Domain Pack Contract v1 wired in 2026-05-21 18:33:23 -07:00
elementary_mathematics_ood feat(capability): ADR-0110 promote mathematics_logic to expert_demo (#118) 2026-05-22 12:59:23 -07:00
english_fluency_ood
fabrication_control feat(capability): ADR-0110 promote mathematics_logic to expert_demo (#118) 2026-05-22 12:59:23 -07:00
forward_semantic_control
foundational_biology_ood
foundational_physics_ood
frontier_compare feat(frontier): add replay variability suite and token-cost telemetry (#66) 2026-05-20 15:04:34 -07:00
grammatical_coverage
hebrew_fluency feat(packs): ADR-0103 — attach hebrew_fluency + koine_greek_fluency lanes to ADR-0102 (#106) 2026-05-22 09:43:46 -07:00
identity_divergence
industry_demos evals/industry_demos: add run_all.py suite runner 2026-05-21 08:23:29 -07:00
inference_closure feat(capability): ADR-0110 promote mathematics_logic to expert_demo (#118) 2026-05-22 12:59:23 -07:00
introspection
koine_greek_fluency feat(packs): ADR-0103 — attach hebrew_fluency + koine_greek_fluency lanes to ADR-0102 (#106) 2026-05-22 09:43:46 -07:00
lab research(evals): phi separation probe for ADR-0081 follow-up (#57) 2026-05-20 12:34:59 -07:00
learning_loop chore(evals): contracts + bench json + Lane B viewer + chart + audit + demo schema (#62) 2026-05-20 13:53:13 -07:00
long_context_cost chore(evals): contracts + bench json + Lane B viewer + chart + audit + demo schema (#62) 2026-05-20 13:53:13 -07:00
miner_loop_closure feat(teaching): implement ADR-0095 — Miner-Sourced Teaching Proposals 2026-05-21 18:18:51 -07:00
monotonic_learning
multi_agent_composition
multi_sentence_response
multi_step_reasoning
orthogonality_tour parallel eval runner (#46) 2026-05-19 23:51:59 -07:00
prompt_diversity feat(adr-0085): gloss-aware CAUSE composer — explanation frame from glosses (#70) 2026-05-20 15:55:08 -07:00
provenance
public_demo fix(capability): correct discourse_planner flag catalog + commit-independent public_demo pin 2026-05-21 20:53:15 -07:00
realizer_guard chore(evals): contracts + bench json + Lane B viewer + chart + audit + demo schema (#62) 2026-05-20 13:53:13 -07:00
refusal_calibration
register_diagnostics feat(evals): per-intent register-firing diagnostic + CI gate + tests (#103) 2026-05-21 07:05:23 -07:00
register_tour feat(register): ADR-0077 — substantive register knobs + layering boundary (R6) 2026-05-19 23:39:11 -07:00
reports
results feat: ADR-0086 + ADR-0087 + 100-register catalog — cognition lane closure 2026-05-21 00:08:12 -07:00
reviewer_registry feat(capability): implement ADR-0092 — Reviewer Registry v1 2026-05-21 18:01:24 -07:00
sample_efficiency
self_consistency_over_time
symbolic_logic
teaching_injection_resistance
walkthrough_chain
warmed_session_consistency
zero_code_domain_acquisition
__init__.py
_parallel.py parallel eval runner (#46) 2026-05-19 23:51:59 -07:00
baseline_runner.py
CLAIMS.md chore(evals): contracts + bench json + Lane B viewer + chart + audit + demo schema (#62) 2026-05-20 13:53:13 -07:00
cognition_cases.jsonl
framework.py feat(evals): ADR-0105 — sealed holdout encryption via age (#108) 2026-05-22 10:09:43 -07:00
holdout_runner.py feat(evals): ADR-0105 — sealed holdout encryption via age (#108) 2026-05-22 10:09:43 -07:00
metrics.py
parallel.py parallel eval runner (#46) 2026-05-19 23:51:59 -07:00
run_cognition_eval.py parallel eval runner (#46) 2026-05-19 23:51:59 -07:00