Commit graph

7 commits

Author SHA1 Message Date
Shay
78312b3151 chore: ADR-0119.4 + ADR-0119.6 cleanup — typed refusals + numeric/freshness asserts
Audit follow-ups from #145/#146 merge review. Five small fixes; no
behavior change on the green path, but failure modes are now explicit
rather than silent.

ADR-0119.6 depth_curve.py
  - Add DepthCurveError typed exception
  - Raise on case_id missing from lane_report (was: silent → "refused")
  - Raise on depth >= 9 (was: silent new bucket key)
  - Two new tests pin both refusals
  - Removed stale sys.path hack at module top

ADR-0119.4 frontier-baseline tests
  - Assert comparison_v1.json's core_measurement reports wrong == 0
    (the load-bearing differentiator named in the disclaimer; a
    tampered file with wrong > 0 was previously syntactically valid
    and would have passed all old assertions)
  - Assert frontier citations are dated 2023 or later (freshness
    guard; older citations should be refreshed before ADR-0120
    gates anything for `expert` promotion)

Tests
  - tests/test_adr_0119_6_depth_curve.py: 7 → 9
  - tests/test_adr_0119_4_frontier_baseline.py: 5 → 7
  - 29/29 across runner + depth-curve + frontier suites; 67/67 smoke

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:47:42 -07:00
Shay
0ffbd5c40a merge origin/main and resolve README conflicts 2026-05-22 17:38:21 -07:00
Shay
9288688640
feat: ADR-0119.3 — gsm8k_math lane runner (Phase 5.3) (#145)
Composes the Phases 1-4 pipeline (parser → solver → verifier →
realizer) into a per-case scoring decision: correct / wrong /
refused.

Outcome categorization (ADR-0114a Obligation #4):
  parser ParseError       → refused
  solver SolveError       → refused
  verifier verdict failed → wrong
  realizer error          → wrong
  answer/unit mismatch    → wrong
  all match               → correct

`wrong == 0` is the load-bearing gate. The lane's overall_pass
holds only if wrong == 0 AND correct + refused == total.

Initial measurement on the Phase 5.2 corpus:
  dev    (50)  : 50 correct, 0 wrong, 0 refused, overall_pass=True
  public (150) : 150 correct, 0 wrong, 0 refused, overall_pass=True

Every correct case carries a trace_hash (64-char SHA-256) and
realized prose — full audit trail per case, consumable by ADR-0119.4
(frontier comparison), ADR-0119.6 (depth curve), and ADR-0120
(eventual expert-tier gate).

Tests: 13/13 green; 443 total green across runner + realizer +
solver + verifier; 67/67 smoke green.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:37:54 -07:00
Shay
a65040cb73 merge origin/main and resolve conflicts 2026-05-22 17:37:13 -07:00
Shay
51d3a73589 feat: ADR-0119.6 — depth-curve measurement harness (ADR-0114a Obligation #6) 2026-05-22 17:33:58 -07:00
Shay
c21068ed3e feat: ADR-0119.4 — frontier-baseline comparison (ADR-0114a Obligation #7) 2026-05-22 17:33:28 -07:00
Shay
1a37f97c4f feat: ADR-0119.2 — author 200 grade-school math problems for the GSM8K eval lane (dev + public) 2026-05-22 17:28:00 -07:00