diff --git a/docs/research/compare-increment-funnel-2026-07-18.md b/docs/research/compare-increment-funnel-2026-07-18.md index 6f4881ff..ada1139f 100644 --- a/docs/research/compare-increment-funnel-2026-07-18.md +++ b/docs/research/compare-increment-funnel-2026-07-18.md @@ -68,3 +68,46 @@ Each fix is developed against tune only, to `wrong=0` on tune; the measure split at the end. Refusal-by-reason on the measure split (frequency / nested / anaphora / temporal / inverse-undeterminable) is reported with the delta as the fail-closed evidence + the practice-lane curriculum. + +## Shared-layer reality — the whole-tune refusal histogram (pinned baseline) + +Across ALL 261 tune cases (not just compare-marked), the reader's refusal-reason histogram — pinned +**before** any shared-layer edit, asserted **stable-or-explained** after: + +| Refusal reason | Cases | +| :--- | ---: | +| **no injection** | **196** | +| no admissible candidate for statement | 48 | +| expected exactly one question | 9 | +| no admissible candidate for question | 4 | +| **PARSED** (all compare) | **3** | +| no branch produced a solvable | 1 | + +**"no injection" is 196/261 (75%) — a *shared* bottleneck across every family, not a compare defect.** +The reader recognizes statements but can't inject candidates for three-quarters of real GSM8K. A +guard-lift that silently changed *why* other cases refuse would corrupt the practice-lane curriculum +even with zero parse breakage, so the histogram is pinned; the only sanctioned post-edit delta is +compare cases moving from "no injection" → PARSED (or to a downstream layer), disclosed explicitly. + +## Funnel-conditioned escalation rule (PRE-COMMITTED, before the number exists) + +X = 15 was ratified assuming *family capability* was the wall. The baseline shows the wall may be the +**shared pipeline** (injection / enumeration / admissibility / round-trip) that every family crosses. +So a miss on X is read through the funnel: + +- **Mass dying in compare-specific logic** → the family's design well is dry → **practice-lane ruling** + (as designed). +- **Mass dying in a shared layer** → the next increment is a **shared-layer design increment**, *not* + a practice escalation. X is re-applied after the shared layer is fixed. + +Load-bearing reason: **the practice lane cannot learn past a deterministic wall.** A practice loop +generates better parse *candidates*, but they cross the same enumeration/admissibility/round-trip +pipeline; if a hard-coded shared layer refuses everything, practice fails identically — escalating into +a mechanism that can't help. Shared-layer bottlenecks are design work regardless of what X says. + +## Reframe (recorded) + +Compare is the **first slice of a much larger reader project**: the reader parses ~1% of real GSM8K +(3/261 tune), and "no injection" (75%) is a shared wall in front of every family. This is not scope +creep — it is the arc seeing its true size. The increments stay small and measured *precisely because* +the project is large.