core-labs/core
core-labs/core
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

docs(reader-arc): compare increment per-layer attrition funnel (diagnostic)

Browse source 
Instrument-first (PR #76 ruling): before touching the parser, measure where the
61 compare-marked TUNE cases die. Funnel: 57/61 die pre-enumeration; of those,
**45 at the injection layer** ('recognizer matched but produced no injection'),
8 'no admissible candidate for statement', 3 'expected exactly one question
sentence'; 3 solve today.

Finding: the leverage is INJECTION (45/61), not the candidate regexes (which
fire on clean clauses but the real multi-clause sentences never reach them) nor
segmentation writ large. Injection is an EMITTER, so the sanctioned fix is
making it inject compare candidates the recognizer already matches — never
loosening the round-trip gate. Fix order by measured mass: injection (45) ->
statement-admissibility (8) -> question-sentence (3). Tune-only development;
single measure run at the end with refusal-by-reason + delta.
This commit is contained in:
Shay 2026-07-18 17:25:26 -07:00
parent 260e56ca5c
commit 4a5d0f460d
1 changed files with 51 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

51
docs/research/compare-increment-funnel-2026-07-18.md Normal file
View file

@ -0,0 +1,51 @@
# compare_multiplicative Increment — Per-Layer Attrition Funnel (diagnostic artifact)
**Status**: DIAGNOSTIC — instrument-first, decides the fix order (PR #76 ruling)
**Date**: 2026-07-18
**Scope**: the 61 compare-multiplicative-marked cases in the **tune** split of official `holdout_dev/v1`
(measure split untouched). Signals are the reader's own: `branches_enumerated`, `refusal_reason`,
`selected_graph`, then the corridor compiler.
---
## The funnel
Reader pipeline stages: recognizer-match → **injection** → enumeration → admissibility → graph → solve.
| Layer | Cases (of 61) |
| :--- | ---: |
| Died **pre-enumeration** | **57** |
| Enumerated | 4 |
| …enumerated but no graph | 1 |
| …graph has a compare op | 3 |
| **Solved today** | **3** |
Pre-enumeration refusal reasons (the 57):
| Reason | Cases |
| :--- | ---: |
| **"recognizer matched but produced no injection"** | **45** |
| "no admissible candidate for statement" | 8 |
| "expected exactly one question sentence" | 3 |
| "no admissible candidate for question" | 1 |
## The finding
**The leverage is the injection layer: 45 / 61.** The recognizer already *matches* these compare
statements (e.g. 0000: "Aria has twice as many … as Emily, who has twice the number … as Spencer");
the **injection** step then produces no candidate — the multi-clause / compare shapes don't inject.
This is where the mass is — not the candidate regexes (they fire on clean clauses but the real
sentences never reach them), and not segmentation writ large.
## Consequence for the fix (emitter-fix, not gate-loosening)
Injection is an **emitter** (it produces candidates), not a validator — so making it inject
compare candidates for statements the recognizer already matches is the sanctioned kind of change
(fix emitters; never loosen the round-trip gate). The remaining piles are secondary and sequenced
after: "no admissible candidate for statement" (8), the one-question-sentence constraint (3).
Fix order, by measured mass: **injection (45) → statement-admissibility (8) → question-sentence (3)**.
Each fix is developed against tune only, to `wrong=0` on tune; the measure split gets its single run
at the end. Refusal-by-reason on the measure split (frequency / nested / anaphora / temporal /
inverse-undeterminable) is reported with the delta as the fail-closed evidence + the practice-lane
curriculum.