4f473a5aa8
First worked attempt at promoting a domain under the ADR-0120
expert promotion contract. The contract refuses honestly.
Gate evaluation against live state:
ADR-0114a obligations: 10 of 10 pass
ADR-0120 contract-level gates:
audit_passed_holds ✓
correct_rate (public) ✓ 150/150 = 1.0
correct_rate (sealed) ✗ 0/1319 = 0.0 < 0.60 floor
signed_expert_claim ✗ (no entry, downstream of correct_rate)
Decision: mathematics_logic NOT promoted; stays at audit-passed.
Substantive blocker: parser grammar covers 0/1319 of real GSM8K.
What this proves
- The contract is genuinely falsifiable. ADR-0120 §"Threshold
rationale" deliberately set the floor above current measurement
so the first attempt would defer honestly. Same load-bearing
pattern as ADR-0107 → ADR-0110 for audit-passed.
- Wrong-zero discipline holds against real GSM8K (the load-
bearing positive claim). CORE refuses every problem outside
its grammar without confabulating on a single one.
What unlocks the promotion
Multi-ADR parser-expansion arc lifting sealed-GSM8K correct_rate
from 0.0 to ≥ 0.60. Each construction class (rate/comparison/
percentage/time-modal/etc.) ships as its own scoped ADR with:
- parser+solver+verifier+realizer extensions
- re-measurement on sealed holdout
- ADR-0118a OOD re-measurement (no surface-feature regression)
- ADR-0125 perturbation re-measurement (no invariance regression)
- ADR-0119.5 adversarial re-measurement (no new misparses)
Honest-fitting discipline: every lift is graded on the anti-
overfit obligations BEFORE the correct_rate change counts.
Tests: 6/6 with CORE_HOLDOUT_KEY; 4/6 + 2 skipped without (matches
ADR-0119.7 seal discipline).
This deferral demonstrates the expert tier's promotion machinery
is load-bearing — the gate has refused at least once before any
domain reaches it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| adr | ||
| architecture | ||
| curriculum | ||
| decisions | ||
| evals | ||
| handoff | ||
| sessions | ||
| capability_roadmap.md | ||
| ethics_packs.md | ||
| EVAL_AUDIT_2026-05-20.md | ||
| eval_methodology.md | ||
| formation_pipeline_plan.md | ||
| frontier_baselines.md | ||
| gaps.md | ||
| holdout_recipients.txt | ||
| identity_packs.md | ||
| pack_inventory_2026-05-21.md | ||
| PROGRESS.md | ||
| reviewers.yaml | ||
| runtime_contracts.md | ||
| RUST.md | ||
| safety_packs.md | ||
| teaching_order.md | ||
| truth_seeking_schema.md | ||
| Whitepaper.md | ||
| Yellowpaper.md | ||