e7a1ffb72e
Adds CandidateConditionalOpQuestion + extractor for the closed shape: "If <Entity> <verb> <N> <unit>, how many <unit2> does <Entity2> <aux> [<qualifier>]?" In parse_and_solve, when the question yields exactly one such candidate and exactly one matching InitialPossession exists by (entity, unit) across all statement sentences, computes initial_value ± operand (verb polarity) and emits when answer >= 0; refuses otherwise. Structurally identical to S.1 capacity/earnings short-circuits. GSM8K probe: 2/50 → 3/50 (+0042, answer=30.0), wrong stays 0. - generate/math_candidate_parser.py: _COND_SUBTRACT_VERBS / _COND_ADD_VERBS closed sets; _COND_OP_Q_RE; extract_conditional_op_question_candidates - generate/math_candidate_graph.py: short-circuit after earnings path - tests/test_adr_0136_S2_conditional_op.py: 25 tests (extractor unit tests, end-to-end short-circuit, B3 + S.1 regression guards, post-S.2 honest admission count) - docs/decisions/ADR-0136.S.2-conditional-op-question.md
106 lines
4 KiB
Markdown
106 lines
4 KiB
Markdown
# ADR-0136.S.2 — Conditional-Op Question (Statement-Layer Corridor)
|
|
|
|
**Status:** Active
|
|
**Date:** 2026-05-23
|
|
**Parent:** [ADR-0136](./ADR-0136-statement-layer-corridor.md)
|
|
|
|
---
|
|
|
|
## Context
|
|
|
|
After S.0 (context-sentence classifier) and S.1 (rate/event statement parsing)
|
|
landed, the GSM8K train-sample probe sat at **2/50** admitted (`gsm8k-0014`,
|
|
`gsm8k-0018`), `wrong == 0`.
|
|
|
|
A taxonomy pass over the remaining 48 refused cases identified `gsm8k-0042`
|
|
as the next single-barrier unlock:
|
|
|
|
```
|
|
Ella has 4 bags with 20 apples in each bag and six bags with 25 apples in
|
|
each bag. If Ella sells 200 apples, how many apples does Ella has left?
|
|
```
|
|
|
|
The initial-state sentence already parses via `_CONJ_EMBEDDED_RE` (ADR-0131.G.4
|
|
embedded quantifier) to `InitialPossession(entity="Ella", value=230,
|
|
unit="apples")`. The **only** barrier is the question form, which neither
|
|
`_Q_ENTITY_RE` nor `_Q_TOTAL_RE` cover: the `If <Entity> <verb> <N> <unit>,
|
|
how many <unit2> does <Entity2> <aux> [left|...]?` shape combines a
|
|
conditional-action operand with the entity-recall question.
|
|
|
|
---
|
|
|
|
## Decision
|
|
|
|
Add a **conditional-op question** extractor and a corresponding short-circuit
|
|
in `parse_and_solve` that:
|
|
|
|
1. Matches the closed shape `If <Entity> <verb> <N> <unit>, how many <unit2>
|
|
does <Entity2> <aux> [<qualifier>]?`
|
|
2. Classifies `<verb>` against two closed sets:
|
|
- `_COND_SUBTRACT_VERBS` (sell/sells/sold, give/gives/gave, eat/eats/ate,
|
|
use, lose, spend, donate, remove, take, send, pay, drop, throw)
|
|
- `_COND_ADD_VERBS` (buy/buys/bought, get/gets/got, receive, find, add,
|
|
collect, pick, earn, gain)
|
|
3. Refuses on any of: unknown verb, unit mismatch (`<unit>` vs `<unit2>`
|
|
after canonicalization), entity mismatch (`<Entity>` vs `<Entity2>`
|
|
case-insensitively), `N <= 0`.
|
|
4. In `parse_and_solve`: if the question yields exactly one
|
|
`CandidateConditionalOpQuestion`, collect all `extract_initial_candidates`
|
|
from every statement sentence and look for **exactly one** matching IC
|
|
by `(entity, unit)`. If found, compute `initial_value ± operand` by verb
|
|
polarity; emit only when `answer >= 0`. Refuses otherwise.
|
|
|
|
The short-circuit is structurally identical to the S.1 capacity/earnings
|
|
paths: it bypasses graph construction, returns `selected_graph=None`, and
|
|
preserves `wrong == 0` by refusing rather than guessing on any ambiguity.
|
|
|
|
---
|
|
|
|
## Invariants
|
|
|
|
- **`admitted_wrong == 0`** preserved by:
|
|
- Single-match (entity, unit) requirement before emission
|
|
- Non-negative answer gate
|
|
- Closed verb sets — no wildcards
|
|
- **Context-filler safety rail** unchanged (S.0)
|
|
- **No solver/graph/verifier changes** — extractor lives in
|
|
`math_candidate_parser.py`; short-circuit lives in `math_candidate_graph.py`
|
|
|
|
---
|
|
|
|
## Honest GSM8K delta
|
|
|
|
| Stage | Admitted | Wrong |
|
|
|---|---|---|
|
|
| Pre-S.0 | 0/50 | 0 |
|
|
| Post-S.1 | 1/50 (`0014`) | 0 |
|
|
| Post-S.0 classifier | 2/50 (`+0018`) | 0 |
|
|
| **Post-S.2** | **3/50** (`+0042`) | **0** |
|
|
|
|
`gsm8k-0042` admits with `answer == 30.0` (expected: 30).
|
|
|
|
---
|
|
|
|
## Consequences
|
|
|
|
- The S.x corridor now spans `parser` (S.1 capacity/earnings, S.2 question
|
|
shape) and `pre-pass classifier` (S.0). Future phases (S.3 compound
|
|
statements, S.4 coreference) extend this pattern.
|
|
- The canonical GSM8K runner (`evals/gsm8k_math/runner.py`) still asserts
|
|
`selected_graph is not None` on admission and therefore cannot score the
|
|
short-circuit admissions. The `report.json` artifact remains stale
|
|
(0/50/0); the honest count is asserted via direct `parse_and_solve` test
|
|
(`test_gsm8k_post_s2_admission_honest`). Aligning the canonical runner
|
|
with the short-circuit paths is deferred (out of S.2 scope).
|
|
|
|
---
|
|
|
|
## Deferred
|
|
|
|
- Conditional-op question forms with **more than one** operand (e.g.
|
|
"If she gives away 3 and buys 5, …") — needs two-op composition.
|
|
- Question forms without `does <Entity>` aux (e.g. "how many apples
|
|
remain?") — needs a sibling regex.
|
|
- Conditional questions where the conditional and the question reference
|
|
**different** units that are unit-related (e.g. dollars↔cents) — needs
|
|
unit-relation taxonomy.
|