From beafeb5ad236ac62610e91f69766e24573f93eb8 Mon Sep 17 00:00:00 2001 From: Shay Date: Sat, 23 May 2026 20:21:32 -0700 Subject: [PATCH] docs(ADR-0136): statement-layer corridor + S.0 taxonomy + resolve math-file conflicts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add ADR-0136-statement-layer-corridor.md: corridor overview, S.0-S.4 phase table, taxonomy summary (23 context-filler, 4 rate-class, 5 compound, 17 long-tail), standing invariants (admitted_wrong==0, context-filler safety rail, honest delta). - Restore generate/math_*.py to HEAD (main) — stash-pop conflict markers were cosmetic comment rewording; upstream version is authoritative. - Stage docs/reviewers.yaml math_expert_claims addition (already authored). --- .../ADR-0136-statement-layer-corridor.md | 122 ++++++++++++++++++ docs/reviewers.yaml | 11 ++ 2 files changed, 133 insertions(+) create mode 100644 docs/decisions/ADR-0136-statement-layer-corridor.md diff --git a/docs/decisions/ADR-0136-statement-layer-corridor.md b/docs/decisions/ADR-0136-statement-layer-corridor.md new file mode 100644 index 00000000..1308c93c --- /dev/null +++ b/docs/decisions/ADR-0136-statement-layer-corridor.md @@ -0,0 +1,122 @@ +# ADR-0136 — Statement-Layer Corridor: Graduated GSM8K Admission via Parser Extension + +**Status:** Active +**Date:** 2026-05-23 +**Author:** CORE agents + reviewers +**Parent:** [ADR-0131.G](./ADR-0131.G-gsm8k-coverage-probe.md) +**Depends on:** +[ADR-0115](./ADR-0115-math-problem-parser-and-graph.md), +[ADR-0116](./ADR-0116-deterministic-solver.md), +[ADR-0117](./ADR-0117-solution-trace-verifier.md) + +--- + +## Context + +ADR-0131.G pinned the GSM8K coverage probe at **0/50 admission** and established +`admitted_wrong == 0` as a standing architectural invariant. The G.x capability +axes (G.1–G.5) have landed on main; as of ADR-0131.5 (probe retirement), the +GSM8K probe is no longer a per-iteration gate — it activates only when a new +iteration claims Δadmission_rate ≥ 0.02. + +A taxonomy pass over all 50 refused cases (S.0) produced the following breakdown: + +| Primary barrier | Count | Notes | +|---|---|---| +| `context_filler` | 23 | Narrative scene-setters; parser's refusal is correct | +| `compound_statement` | 5 | Two operations in one sentence | +| rate/capacity/price class | 4 | Direct targets of S.1 | +| `distributive_multiply` | 1 (+5 secondary) | N bags × M items each | +| diverse long-tail | 17 | Age anchors, goal statements, multi-step chains, etc. | + +The 23 context-filler cases will not be addressed by statement-layer work alone — +they need semantic classification of scene-setting sentences, which is a separate +architectural concern. **The safety rail stands: if a sentence cannot parse into +a numeric initial-state candidate, the problem is refused.** + +The G.x micro-extension approach has been retired. Each G.x axis targeted one +regex shape in isolation; the return per axis was tiny and the axes were not +composable into a coherent parser narrative. The Statement-Layer Corridor replaces +G.x with a phased taxonomy-driven program that: + +1. Classifies all refusals before writing any code. +2. Targets the highest-signal unlockable barriers first. +3. Ships each phase as a self-contained unit with a curated axis lane, `wrong == 0` + gate, and honest GSM8K delta. +4. Never attempts to bypass the context-filler safety rail. + +--- + +## Decision + +Organize parser extension work as a **corridor of phases** rooted at ADR-0136: + +| Phase | ADR | Scope | Primary barrier targeted | +|---|---|---|---| +| S.0 | (this doc) | Taxonomy — deterministic classification of 50 refused cases | — | +| S.1 | ADR-0136.S.1 | Rate/event statements — capacity-rate + earnings-rate shapes | rate class (≤4 cases) | +| S.2 | ADR-0136.S.2 | Temporal statements — time anchors, duration expressions | time/age long-tail | +| S.3 | ADR-0136.S.3 | Compound statements — two operations in one sentence | `compound_statement` (5 cases) | +| S.4 | ADR-0136.S.4 | Coreference — pronoun + ellipsis resolution across sentences | cross-sentence barriers | + +Phases S.2–S.4 are deferred; scope and sequencing will be revisited after S.1 lands. + +--- + +## Taxonomy (S.0) + +Stored at: `evals/gsm8k_math/train_sample/v1/refusal_taxonomy.json` + +Schema v1. Each record carries: +- `case_id` — GSM8K case identifier +- `primary_barrier` — the single barrier that causes refusal even if all others were resolved +- `secondary_barriers` — additional barriers present in the problem +- `notes` — free-text rationale + +**Key finding.** The case with the shallowest barrier is `gsm8k-0014`: + +``` +Bob can shuck 10 oysters in 5 minutes. +How many oysters can he shuck in 2 hours? +``` + +This is a single-statement capacity-rate problem with a pronoun question. It is the +proof case for S.1: it must admit with `answer == 240.0`. + +--- + +## Invariants + +These invariants are non-negotiable across all corridor phases: + +- **`admitted_wrong == 0`** — no GSM8K case is admitted with a wrong answer. +- **Context-filler safety rail** — sentences without parseable numeric initial state + are refused; no soft-fail or skip-and-continue. +- **Honest delta** — each phase's PR body states the exact pre/post GSM8K admission + count; no rounding, no "approximately". +- **No solver/graph/verifier changes** — rate-path short-circuits and new extractors + live in `math_candidate_parser.py` and `math_candidate_graph.py` only. + +--- + +## Consequences + +- The G.x capability axis namespace is closed. New axes use the S.x naming convention. +- Each S.x phase ships its own curated axis lane at + `evals/math_capability_axes/S_/v1/cases.jsonl`. +- The GSM8K probe re-activates when a phase claims ≥1 new admission + (Δadmission_rate ≥ 0.02); otherwise it stays retired per ADR-0131.5. +- S.1 is the only phase that can honestly claim unlocking the context-filler-gated + cases once sentence-semantic classification lands (out of S.x scope). + +--- + +## Deferred + +- Context-filler gated problems (23 cases) — requires semantic classification of + narrative scene-setting sentences; architecturally separate from statement parsing. +- Conditional branching (`if she works more than 8 hours`) — needs branching semantics + in the solver. +- Percentage/interest rates — needs decimal arithmetic extension. +- Multi-statement rate problems (duration in separate sentence from capacity) — needs + coreference, addressed in S.4. diff --git a/docs/reviewers.yaml b/docs/reviewers.yaml index eec4e090..b2d9c27f 100644 --- a/docs/reviewers.yaml +++ b/docs/reviewers.yaml @@ -51,3 +51,14 @@ audit_passed_claims: evidence_revision: "adr-0124:reviewed:2026-05-22" signed_by: shay-j claim_digest: "17e24436b6875b89f6d1a5c2992557413c7ef456250f549d463159f54438c407" +math_expert_claims: + - domain_id: mathematics_logic + evidence_lanes: + - math_symbolic_equivalence/v1 (public) + - math_symbolic_equivalence/v1 (sealed) + - math_teaching_corpus/v1 + - math_bounded_grammar/v1 + evidence_revision: "adr-0120-math:reviewed:2026-05-23" + signed_by: shay-j + claim_digest: "94149794e8c19896851e062cf1f921cfa9ba04770b674bc3b4c33023f7c7331b" + \ No newline at end of file