6e11145282
PR #204 (ADR-0137 retrospective binding, rescoped to subsumption-only) was drafted then closed without merging. The deferral reason is structural: the DeferredCandidate/BindingProof apparatus is the right shape for true retrospective binding (first-pass candidate has open slots awaiting later evidence), but re-examination showed neither S.1 nor S.2 short-circuit fits that definition — both join two fully grounded candidates. With no kind that has open slots in the v2 ledger, the machinery would degenerate to a join everywhere, which is speculative infrastructure ahead of a forcing function (CLAUDE.md warns against this pattern). Update appended to the post-rescan notes so the deferral lives in the repo, not just in a closed PR thread. Includes a reopen criterion: ADR-0137 (or successor) may return when a case appears whose first-pass candidate has genuinely open slots only later-sentence evidence can close. Until then the unlock vehicle is S.x. The S.1/S.2 short-circuits remain as tactical bridges; the canonical runner staleness is better fixed in the runner than in a binding layer.
191 lines
7.9 KiB
Markdown
191 lines
7.9 KiB
Markdown
# ADR-0136.S.2-post-rescan — Refusal Rescan v2: Barrier-Shift Ledger
|
||
|
||
**Status:** Accepted
|
||
**Parent:** ADR-0136 (Statement Layer Corridor)
|
||
**Date:** 2026-05-23
|
||
|
||
## Context
|
||
|
||
The v1 refusal taxonomy (ADR-0136.S.0) was written before S.1 and S.2 landed.
|
||
This rescan re-runs `parse_and_solve` on all 50 GSM8K train-sample cases against
|
||
the current parser (post-S.1 rate/event parsing, post-S.2 conditional-op
|
||
questions) and produces a barrier-shift ledger so ADR-0137 inherits measured
|
||
truth, not stale classification.
|
||
|
||
## Summary
|
||
|
||
| Metric | Value |
|
||
|---|---|
|
||
| Total cases | 50 |
|
||
| Admitted (correct) | 3 |
|
||
| Wrong | 0 |
|
||
| Refused | 47 |
|
||
| Barrier shifted (v1 → v2) | 27 |
|
||
|
||
## Current Admissions
|
||
|
||
| Case | Answer | Short-circuit path | Landed in |
|
||
|---|---|---|---|
|
||
| gsm8k-0014 | 240.0 | S.1 capacity rate | ADR-0136.S.1 |
|
||
| gsm8k-0018 | 16.0 | S.2 capacity rate (context filler resolved) | ADR-0136.S.2 |
|
||
| gsm8k-0042 | 30.0 | S.2 conditional-op question | ADR-0136.S.2 |
|
||
|
||
## Current Primary Barrier Distribution
|
||
|
||
| Primary barrier | Count | Notes |
|
||
|---|---|---|
|
||
| compound_statement | 6 | Two operations in one sentence |
|
||
| novel_initial_form | 5 | Existential/impersonal/quantifier sentence shapes |
|
||
| novel_initial_verb | 4 | Verb not in any closed set (donated, collected, gained, bakes) |
|
||
| admitted | 3 | Successfully parsed and solved |
|
||
| compound_comparative | 3 | N times as much/greater comparisons |
|
||
| conditional_question | 3 | If-clause in question sentence |
|
||
| context_filler | 3 | Narrative scene-setter still blocking (down from 23) |
|
||
| fraction_operand | 3 | Half/quarter/3-4/75% operands |
|
||
| distributive_multiply | 2 | N bags × M items |
|
||
| multi_entity_initial | 2 | Multiple entities in one sentence |
|
||
| rate_price | 2 | Price embedded in relative clause |
|
||
| temporal_frequency | 2 | Every N days / every other day |
|
||
| capacity_rate | 1 | "within" preposition instead of "in" |
|
||
| complex_question | 1 | Modal + infinitive chain in question |
|
||
| compound_multi_event | 1 | Multiple transport legs in one sentence |
|
||
| conditional_branch | 1 | Or-choice pricing |
|
||
| distributive_each_actor | 1 | "each saved" = per-actor distribution |
|
||
| goal_statement | 1 | "wants to lose" = intent, not state |
|
||
| multi_attribute_accumulation | 1 | Multiple platform counts in one sentence |
|
||
| multi_day_accumulation | 1 | Values across Mon–Thu in one sentence |
|
||
| partition_divide | 1 | Splits into sections |
|
||
| percentage_rate | 1 | 10% simple interest |
|
||
| rate_earnings | 1 | Multi-statement earnings with conditional |
|
||
| temporal_age_anchor | 1 | Age-conditioned initial state |
|
||
|
||
## Barrier-Shifted Cases (v1 → v2)
|
||
|
||
27 cases changed primary barrier classification. The dominant pattern: 18 of 23
|
||
v1 `context_filler` cases now parse their scene-setting sentence, exposing the
|
||
next structural barrier. Only 3 `context_filler` cases remain blocking (0022,
|
||
0031, 0049 — all have rate/intent embedded in complex relative clauses).
|
||
|
||
| Case | v1 barrier | v2 barrier |
|
||
|---|---|---|
|
||
| 0003 | context_filler | novel_initial_verb |
|
||
| 0004 | indefinite_quantity | fraction_operand |
|
||
| 0005 | compound_statement | fraction_operand |
|
||
| 0007 | context_filler | conditional_question |
|
||
| 0008 | context_filler | conditional_question |
|
||
| 0009 | compound_comparative | conditional_question |
|
||
| 0013 | context_filler | compound_statement |
|
||
| 0014 | capacity_rate | **admitted** |
|
||
| 0017 | context_filler | conditional_branch |
|
||
| 0018 | context_filler | **admitted** |
|
||
| 0019 | context_filler | rate_price |
|
||
| 0021 | context_filler | distributive_multiply |
|
||
| 0023 | multi_step_complex | novel_initial_verb |
|
||
| 0025 | context_filler | distributive_multiply |
|
||
| 0028 | context_filler | novel_initial_form |
|
||
| 0029 | context_filler | compound_comparative |
|
||
| 0030 | context_filler | novel_initial_form |
|
||
| 0032 | context_filler | compound_statement |
|
||
| 0034 | context_filler | capacity_rate |
|
||
| 0035 | coreference_pronoun | complex_question |
|
||
| 0036 | context_filler | compound_comparative |
|
||
| 0039 | context_filler | novel_initial_verb |
|
||
| 0040 | context_filler | multi_entity_initial |
|
||
| 0042 | distributive_multiply | **admitted** |
|
||
| 0043 | context_filler | compound_comparative |
|
||
| 0045 | context_filler | novel_initial_form |
|
||
| 0048 | context_filler | temporal_frequency |
|
||
|
||
## Subsumption Directive
|
||
|
||
> S.1/S.2 short-circuit paths are tactical bridges. ADR-0137 SHALL subsume them:
|
||
> every current short-circuit admission must be re-derivable as a
|
||
> (DeferredCandidate, evidence, BindingProof) triple. Short-circuits are kept
|
||
> only as regression fixtures.
|
||
|
||
### Update 2026-05-23: ADR-0137 attempted and deferred
|
||
|
||
ADR-0137 was drafted and rescoped against this ledger (PR #204), then **closed
|
||
without merging** on self-review. The reason is structural, not procedural:
|
||
|
||
The DeferredCandidate / BindingProof apparatus is the right shape for *true*
|
||
retrospective binding — where a first-pass candidate has open slots that only
|
||
later-sentence evidence can fill. Re-examining S.1/S.2 against that definition
|
||
showed neither short-circuit fits it:
|
||
|
||
- **0014**: `CandidateCapacity` (sentence 1) and `CandidateCapacityQuestion`
|
||
(sentence 2) are both fully grounded. What's needed is a **join rule**
|
||
pairing them into a derived `InitialPossession`, not a deferred-binding pass.
|
||
- **0042**: `CandidateInitial` (sentence 1) and `CandidateConditionalOpQuestion`
|
||
(sentence 2) are both fully grounded — same shape, same join.
|
||
|
||
With no kind that has genuinely open slots in the v2 ledger, the binding
|
||
machinery would degenerate to a join everywhere it ran. Building it now would
|
||
be speculative infrastructure ahead of a forcing function — explicitly the
|
||
pattern CLAUDE.md warns against.
|
||
|
||
**What stays.** The S.1/S.2 short-circuits in `parse_and_solve` remain as
|
||
tactical bridges. `wrong == 0` is preserved. The canonical
|
||
`evals/gsm8k_math/runner.py` staleness on short-circuit admissions (it asserts
|
||
`selected_graph is not None`) is a real annoyance but better fixed in the
|
||
runner — teach it to score short-circuit admissions — than by inventing a
|
||
binding layer to give every admission a graph.
|
||
|
||
**Reopen criterion.** ADR-0137 (or its successor) may be revisited when a
|
||
case appears whose first-pass candidate has genuinely open slots that only
|
||
later-sentence evidence can close. Until then, the unlock vehicle is S.x.
|
||
|
||
## ADR-0137 Probe Cases
|
||
|
||
The following cases are required in the ADR-0137 probe to verify subsumption
|
||
and test the next wave of unlocks:
|
||
|
||
1. **gsm8k-0014** — Bob shucks oysters (capacity rate, currently admitted via S.1)
|
||
2. **gsm8k-0018** — Xavier scores goals (capacity rate, currently admitted via S.2)
|
||
3. **gsm8k-0042** — Ella sells apples (conditional-op, currently admitted via S.2)
|
||
4. **gsm8k-0034** — Georgie runs yards (capacity rate, blocked by "within" preposition)
|
||
5. **gsm8k-0021** — John bench presses (distributive multiply, 15×10×3)
|
||
6. **gsm8k-0003** — Bookstore erasers (novel verb "donated", 48 boxes × 24 each)
|
||
7. **gsm8k-0029** — Mouse and keyboard (compound comparative, 3× cost ratio)
|
||
|
||
Cases 1–3 test subsumption (must remain admitted). Cases 4–7 are the
|
||
shallowest remaining barriers that ADR-0137's binding graph should unlock.
|
||
|
||
## Closed Barrier Vocabulary (v2)
|
||
|
||
The `primary_barrier` field in `refusal_taxonomy_v2.json` uses exactly this
|
||
enum (24 values):
|
||
|
||
```
|
||
admitted
|
||
capacity_rate
|
||
complex_question
|
||
compound_comparative
|
||
compound_multi_event
|
||
compound_statement
|
||
conditional_branch
|
||
conditional_question
|
||
context_filler
|
||
distributive_each_actor
|
||
distributive_multiply
|
||
fraction_operand
|
||
goal_statement
|
||
multi_attribute_accumulation
|
||
multi_day_accumulation
|
||
multi_entity_initial
|
||
novel_initial_form
|
||
novel_initial_verb
|
||
partition_divide
|
||
percentage_rate
|
||
rate_earnings
|
||
rate_price
|
||
temporal_age_anchor
|
||
temporal_frequency
|
||
```
|
||
|
||
## Evidence
|
||
|
||
- Rescan ledger: `evals/gsm8k_math/train_sample/v1/refusal_rescan_v2.json`
|
||
- Taxonomy v2: `evals/gsm8k_math/train_sample/v1/refusal_taxonomy_v2.json`
|
||
- Script: `evals/gsm8k_math/train_sample/v1/rescan_v2.py`
|
||
- Tests: `tests/test_rescan_v2_invariants.py`
|