core/docs/decisions/ADR-0136.S.2-conditional-op-question.md
Shay e7a1ffb72e
feat(ADR-0136.S.2): conditional-op question — gsm8k-0042 admits, wrong==0 (#203)
Adds CandidateConditionalOpQuestion + extractor for the closed shape:
  "If <Entity> <verb> <N> <unit>, how many <unit2> does <Entity2> <aux> [<qualifier>]?"

In parse_and_solve, when the question yields exactly one such candidate
and exactly one matching InitialPossession exists by (entity, unit) across
all statement sentences, computes initial_value ± operand (verb polarity)
and emits when answer >= 0; refuses otherwise. Structurally identical to
S.1 capacity/earnings short-circuits.

GSM8K probe: 2/50 → 3/50 (+0042, answer=30.0), wrong stays 0.

- generate/math_candidate_parser.py: _COND_SUBTRACT_VERBS / _COND_ADD_VERBS
  closed sets; _COND_OP_Q_RE; extract_conditional_op_question_candidates
- generate/math_candidate_graph.py: short-circuit after earnings path
- tests/test_adr_0136_S2_conditional_op.py: 25 tests (extractor unit tests,
  end-to-end short-circuit, B3 + S.1 regression guards, post-S.2 honest
  admission count)
- docs/decisions/ADR-0136.S.2-conditional-op-question.md
2026-05-23 21:20:52 -07:00

4 KiB

ADR-0136.S.2 — Conditional-Op Question (Statement-Layer Corridor)

Status: Active Date: 2026-05-23 Parent: ADR-0136


Context

After S.0 (context-sentence classifier) and S.1 (rate/event statement parsing) landed, the GSM8K train-sample probe sat at 2/50 admitted (gsm8k-0014, gsm8k-0018), wrong == 0.

A taxonomy pass over the remaining 48 refused cases identified gsm8k-0042 as the next single-barrier unlock:

Ella has 4 bags with 20 apples in each bag and six bags with 25 apples in
each bag.  If Ella sells 200 apples, how many apples does Ella has left?

The initial-state sentence already parses via _CONJ_EMBEDDED_RE (ADR-0131.G.4 embedded quantifier) to InitialPossession(entity="Ella", value=230, unit="apples"). The only barrier is the question form, which neither _Q_ENTITY_RE nor _Q_TOTAL_RE cover: the If <Entity> <verb> <N> <unit>, how many <unit2> does <Entity2> <aux> [left|...]? shape combines a conditional-action operand with the entity-recall question.


Decision

Add a conditional-op question extractor and a corresponding short-circuit in parse_and_solve that:

  1. Matches the closed shape If <Entity> <verb> <N> <unit>, how many <unit2> does <Entity2> <aux> [<qualifier>]?
  2. Classifies <verb> against two closed sets:
    • _COND_SUBTRACT_VERBS (sell/sells/sold, give/gives/gave, eat/eats/ate, use, lose, spend, donate, remove, take, send, pay, drop, throw)
    • _COND_ADD_VERBS (buy/buys/bought, get/gets/got, receive, find, add, collect, pick, earn, gain)
  3. Refuses on any of: unknown verb, unit mismatch (<unit> vs <unit2> after canonicalization), entity mismatch (<Entity> vs <Entity2> case-insensitively), N <= 0.
  4. In parse_and_solve: if the question yields exactly one CandidateConditionalOpQuestion, collect all extract_initial_candidates from every statement sentence and look for exactly one matching IC by (entity, unit). If found, compute initial_value ± operand by verb polarity; emit only when answer >= 0. Refuses otherwise.

The short-circuit is structurally identical to the S.1 capacity/earnings paths: it bypasses graph construction, returns selected_graph=None, and preserves wrong == 0 by refusing rather than guessing on any ambiguity.


Invariants

  • admitted_wrong == 0 preserved by:
    • Single-match (entity, unit) requirement before emission
    • Non-negative answer gate
    • Closed verb sets — no wildcards
  • Context-filler safety rail unchanged (S.0)
  • No solver/graph/verifier changes — extractor lives in math_candidate_parser.py; short-circuit lives in math_candidate_graph.py

Honest GSM8K delta

Stage Admitted Wrong
Pre-S.0 0/50 0
Post-S.1 1/50 (0014) 0
Post-S.0 classifier 2/50 (+0018) 0
Post-S.2 3/50 (+0042) 0

gsm8k-0042 admits with answer == 30.0 (expected: 30).


Consequences

  • The S.x corridor now spans parser (S.1 capacity/earnings, S.2 question shape) and pre-pass classifier (S.0). Future phases (S.3 compound statements, S.4 coreference) extend this pattern.
  • The canonical GSM8K runner (evals/gsm8k_math/runner.py) still asserts selected_graph is not None on admission and therefore cannot score the short-circuit admissions. The report.json artifact remains stale (0/50/0); the honest count is asserted via direct parse_and_solve test (test_gsm8k_post_s2_admission_honest). Aligning the canonical runner with the short-circuit paths is deferred (out of S.2 scope).

Deferred

  • Conditional-op question forms with more than one operand (e.g. "If she gives away 3 and buys 5, …") — needs two-op composition.
  • Question forms without does <Entity> aux (e.g. "how many apples remain?") — needs a sibling regex.
  • Conditional questions where the conditional and the question reference different units that are unit-related (e.g. dollars↔cents) — needs unit-relation taxonomy.