core/docs/decisions/ADR-0131.G.4-multi-clause.md
Shay de26d7f792 feat(ADR-0131.G.4): multi-clause composition (conj subjects + conj objects + embedded quantifiers + conj embedded) — admission 0/50 (Δ0), multi-clause refusals 2→1
Highest-risk axis of the ADR-0131.G capability iteration: within-
sentence multi-clause composition. Four extractors land in the
candidate-emitting parser; no graph-side or solver changes.

Parser extension (generate/math_candidate_parser.py)
- _conj_subject_each_candidates: '<A> and [his/her/their <kin>] <B>
  each <verb> <N> <unit>' → 2 CandidateInitial (one per actor).
- _conj_object_candidates: '<E> has <N1> <unit1> and <N2> <unit2>' →
  2 CandidateInitial for the same entity; same-unit conjuncts refuse
  (would silently collide under solver overwrite-on-collision).
- _embedded_quantifier_candidates: '<E> has <N> <container> with <M>
  <unit> in each [<container>]' → 1 derived CandidateInitial
  (value=N*M).
- _embedded_quantifier_candidates (conj branch): '... <N1> <C> with
  <M1> <U> in each ... and <N2> <C> with <M2> <U> in each ...' → 1
  SUM CandidateInitial (value=N1*M1+N2*M2); mixed-unit refuses.
- CandidateInitial anchor whitelist widened to include
  saved/earned/got/received/bought/made/paid (and inflections) —
  narrow widening needed for the conjoined-subject-each shape.

Closed-set discipline
- Distributive 'each' only — 'each ... together/altogether' refuses.
- Two-way conjunction only — 3-way refuses by non-match.
- Cross-sentence coreference stays refused (within-sentence axis).
- Ambiguous 'each' scope refuses (container2 must agree).

Curated axis lane (32 cases)
- evals/math_capability_axes/G4_multi_clause/v1/cases.jsonl:
  conj_subject_each ×6, conj_object ×6, embedded_quantifier ×6,
  conj_embedded ×6, refusal ×8.
- evals/math_capability_axes/G4_multi_clause/v1/runner.py +
  report.json: deterministic; wrong==0 gate; byte-equal across runs.

Tests (26 new)
- tests/test_adr_0131_G4_multi_clause.py: per-shape emission,
  refusal probes (parametric), distributive-only policy,
  cross-sentence refusal, runner byte-equality, GSM8K-probe gate.

GSM8K-probe gate (chosen: multi-clause refusals ↓)
- evals/gsm8k_math/train_sample/v1/report.json (candidate-graph
  probe): multi-clause statement-refusal count 2 → 1. Case 0042
  ('Ella has 4 bags with 20 apples in each bag and six bags with 25
  apples in each bag.') moves from statement-clause refusal to
  question-layer refusal. Case 0026 ('Aaron and his brother Carson
  each saved up $40') stays refused on the '$' value slot
  (deferred to G.3 numeric-literals axis).
- evals/gsm8k_math/train_sample/v1/train_sample_coverage_report.json
  (legacy probe): refreshed, byte-identical (legacy parser
  untouched).

B3 + candidate-graph + GSM8K probe lanes all pass (95/95
regression). wrong==0 preserved everywhere — load-bearing for the
highest-risk axis.
2026-05-23 14:43:16 -07:00

264 lines
12 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# ADR-0131.G.4 — Capability axis: multi-clause composition (conjoined subjects, conjoined objects, embedded quantifiers)
**Status:** Proposed
**Date:** 2026-05-23
**Author:** CORE agents + reviewers
**Parent:** [ADR-0131.G](./ADR-0131.G-gsm8k-coverage-probe.md)
**Depends on:**
[ADR-0126](./ADR-0126-candidate-graph-parser.md) (candidate graph),
[ADR-0127](./ADR-0127-en-units-v1.md) (substance qualifier policy),
[ADR-0131.3](./ADR-0131.3-bounded-grammar.md),
[ADR-0132](./ADR-0132-binding-graph-data-model.md)..[ADR-0135](./ADR-0135-question-target-binding.md)
---
## Context
GSM8K paragraphs frequently introduce starting state via within-sentence
composition the per-statement candidate parser refuses today:
| Baseline refusal sentence | Capability missing |
|---|---|
| `Aaron and his brother Carson each saved up $40 ...` | conjoined subject + distributive `each` |
| `Francine has five full boxes of crayons and 5 loose crayons` | conjoined object NPs sharing a verb |
| `Ella has 4 bags with 20 apples in each bag and six bags with 25 apples in each bag` | embedded quantifier + conjunction |
This is the **highest-risk axis** of ADR-0131.G's four near-term
capability extensions. Multi-clause emission means the round-trip
filter does more work; multi-candidate ambiguity makes confabulation
risk higher. Refusal-first stays paramount: admission gains must be
small and load-bearing.
---
## Decision
Land four within-sentence multi-clause extractors in
`generate/math_candidate_parser.py`. All emit
`CandidateInitial` records (initial state, not operations — the
shapes here introduce starting holdings, they do not mutate state):
| Extractor | Shape (closed set) | Emission |
|---|---|---|
| `_conj_subject_each_candidates` | `<A> and [his/her/their <kin>] <B> each <verb> <N> <unit>` | **two** `CandidateInitial` (one per actor), same `(N, unit)` |
| `_conj_object_candidates` | `<E> has <N1> <unit1> and <N2> <unit2>` | **two** `CandidateInitial` for the same entity; same-unit conjuncts refuse |
| `_embedded_quantifier_candidates` | `<E> has <N> <container> with <M> <unit> in each [<container>]` | **one** derived `CandidateInitial` with `value = N*M, unit = <unit>` |
| `_embedded_quantifier_candidates` (conj branch) | `<E> has <N1> <C> with <M1> <U> in each ... and <N2> <C> with <M2> <U> in each ...` | **one** SUM `CandidateInitial` with `value = N1*M1 + N2*M2, unit = <U>`; mixed-unit conjuncts refuse |
Wired into the existing `extract_initial_candidates` public entry
point — the binding graph (`math_candidate_graph._filtered_statement_choices`)
already consumes through that path; no graph-side edit required (the
read-only audit concluded composed candidates are reachable through
existing edges).
### Closed-set discipline
- **Distributive `each` only.** Surface markers `together`, `in
total`, `altogether` immediately abort emission (explicit
contradiction with distributive reading). Pinned by
`test_refuses_each_with_together` /
`test_collective_without_each_refuses`.
- **Two-way conjunction only.** Three-way `A and B and C each ...`
is out of closed-set shape and refuses by non-match.
- **Same-unit conjoined object refuses.** Two same-unit conjuncts
on the same entity (`Sam has 5 dimes and 3 dimes`) would silently
collide under the solver's `state[(entity, unit)]` overwrite
semantics (`math_solver.py:206`); refusing keeps `wrong == 0`.
- **Ambiguous `each` scope refuses.** `Ella has 4 bags with 20
apples in each box` — `box``bags` ⇒ refuse.
- **Mixed-unit conjoined embedded refuses.** Apples + pears cannot
be summed.
- **No cross-sentence state.** Multi-sentence inputs are processed
per-sentence; pronouns / coreference across sentences stay
refused (out of within-sentence axis scope).
### CandidateInitial anchor widening
`CandidateInitial.__post_init__` whitelists a narrow set of
initial-state-introducing verbs needed for the conjoined-subject-each
shape (`saved`, `earned`, `got`, `received`, `bought`, `made`,
`paid`, plus their inflected variants). The widening is keyed on
lowercase tokens; the `_token_in` check in
`math_candidate_graph._initial_admissible` confirms the anchor word
appears in source. Verb-class widening for the *general* parser
remains G.1's scope; G.4 widens only what conjoined-subject-each
needs.
### Derived-value provenance
Embedded-quantifier and conjoined-embedded emissions carry a
*derived* value (`N*M` and `N1*M1 + N2*M2` respectively) that does
not appear as a single source token. The round-trip filter's
"value grounds in source" check (`_value_grounds`) is satisfied by
anchoring `matched_value_token` on the **per-container `M`** (or
**first per-container `M1`** for the sum). This is a deliberate,
documented widening of the source-grounding spirit: the *components*
of the derived value all appear in the source, and the parser
commits to the canonical arithmetic composition. Refusing on
component-mismatch (mixed units, wrong container scope) and refusing
on indefinite quantifiers in any value slot together keep the
derivation honest. The alternative — emitting two flat candidates
for conjoined-embedded — was rejected: under the solver's
overwrite-on-collision semantics it would silently drop one
conjunct's contribution, breaching `wrong == 0`.
### No graph-side edits
`math_candidate_graph.py` is unchanged. Multi-candidate emission
flows through the existing per-sentence choice space + Cartesian
product; conjoined-subject-each and conjoined-object emissions land
in `_filtered_statement_choices` like any other initial candidate.
The solver's `state[(entity, unit)]` model naturally accommodates
distinct entities (each-shape) and distinct units (object-shape);
collision-prone shapes refuse at the parser. This decision is the
"read-only audit only edit if composed candidate is unreachable"
posture from the brief.
### Curated coverage cases (G.4 axis lane)
`evals/math_capability_axes/G4_multi_clause/v1/cases.jsonl`
**32 cases**:
| Category | Cases | Notes |
|------------------------|-------|-------|
| `conj_subject_each` | 6 | incl. kin-appositive, word-form value |
| `conj_object` | 6 | distinct-unit conjuncts only |
| `embedded_quantifier` | 6 | with + without explicit `container2` |
| `conj_embedded` | 6 | same-unit only |
| `refusal` | 8 | together / altogether / 3-way / cross-entity / scope-mismatch / mixed-unit / cross-sentence / same-unit collision |
Runner emits a deterministic `report.json`; `wrong == 0` is the gate.
### Deferred (out of scope for G.4)
- **Cross-sentence coreference** (`Aaron has 5. He gives 2 to
Bob.`) — needs per-discourse state; pinned as refusal probe.
- **Ellipsis** (`Aaron has 5 apples, Carson 3`) — needs verb
reconstruction; pinned out-of-scope.
- **Three-way+ conjunctions** (`A and B and C`) — combinatorial
explosion + ambiguity; deferred to a future axis.
- **Collective readings** (`A and B saved $40 together`) —
explicitly refused; collective semantics needs a different
binding-graph node type.
- **Currency / unit prefix** (`$40`) — refused at the value slot
(the `$` is not a `_VALUE` character). Deferred to **G.3
numeric-literals axis**, which is the natural place for currency
/ percentage / decimal literal handling. Documented impact on the
GSM8K probe gate (case 0026 stays refused).
- **Same-unit conjoined object summation** — would require either
parser-side sum (analogous to conj-embedded) or solver-side
state-merge; deferred until a sum-shaped CandidateInitial proves
necessary outside this axis.
- **Solver / binding-graph changes.** If a multi-clause case parses
but does not solve, that's a downstream gap and gets its own ADR.
### GSM8K-probe gate (chosen)
G.4 gates on:
> **Multi-clause statement-clause refusals in the candidate-graph
> probe (`evals/gsm8k_math/train_sample/v1/report.json`) strictly
> decrease.**
Counter (in `test_gsm8k_candidate_graph_multi_clause_refusals_decreased`)
matches refused cases citing a statement-clause refusal whose
embedded sentence text contains a multi-clause anchor pattern
(`each <init-verb>`, `with N <unit> in each`, or `has N <unit> and N
<unit>`).
| Probe report | Baseline (origin/main 481e0c3) | After G.4 | Δ |
|---|---|---|---|
| Multi-clause statement-clause refusals (`report.json`) | 2 | 1 | 1 |
| `wrong` (`report.json`) | 0 | 0 | 0 |
| `admission_rate` (`report.json`) | 0/50 | 0/50 | 0 |
| Legacy `train_sample_coverage_report.json` | byte-identical | byte-identical | 0 |
Baseline-2 cases: `gsm8k-train-sample-v1-0026` (`Aaron and his
brother Carson each saved up $40 ...` — refused on `$40` value
slot; deferred to G.3) and `-0042` (`Ella has 4 bags with 20 apples
in each bag and six bags with 25 apples in each bag.` — now parses,
refusal moves to question layer). `admission_rate` does not rise
because downstream layers (question-form admission for derived
initial states) are out of G.4 scope.
### Legacy probe report (refreshed, byte-identical)
`evals/gsm8k_math/train_sample/v1/train_sample_coverage_report.json`
runs through `generate.math_parser.parse_problem` (legacy
first-match-wins), which G.4 does not touch. Refreshed and pinned
via `test_gsm8k_legacy_probe_safety_rail_intact`.
---
## Invariants
- **`g4_wrong_count_is_zero`** — every G.4 axis case passes or
refuses; no case admits a wrong shape. Pinned by
`test_runner_wrong_count_is_zero`.
- **`g4_closed_set_refusals_hold`** — all 8 refusal probes admit
zero multi-clause candidates. Pinned by
`test_refusal_cases_emit_no_admitted_multi_clause` (parametric).
- **`g4_distributive_each_only`** — `each ... together` and `each
... altogether` refuse. Pinned by
`test_refuses_each_with_together`.
- **`g4_cross_sentence_refuses`** — multi-clause extractors do not
fire across sentence boundaries. Pinned by
`test_cross_sentence_pronoun_refuses_multi_clause`.
- **`g4_report_deterministic`** — `report.json` is byte-equal across
back-to-back runs.
- **`gsm8k_safety_rail_intact`** — `admitted_wrong == 0` on both
GSM8K probe reports.
- **`gsm8k_multi_clause_refusal_strictly_decreased`** — chosen G.4
gate.
---
## Acceptance evidence
- `evals/math_capability_axes/G4_multi_clause/v1/runner.py` exits 0
with `wrong == 0` on all 32 curated cases.
- `tests/test_adr_0131_G4_multi_clause.py` (26 tests): per-shape
emission, refusal-set, distributive-only policy, cross-sentence
refusal, runner byte-equality, GSM8K-probe gate.
- Candidate-graph probe `report.json`: multi-clause statement
refusal count 2 → 1 (case 0042 moves from statement to question
refusal).
- Legacy probe `train_sample_coverage_report.json` refreshed and
byte-identical.
- B3 lane + ADR-0126 candidate-graph tests + ADR-0131.G probe
tests all pass (95/95 across the regression sweep).
---
## Consequences
- The candidate-graph topology can now see four multi-clause
initial-state shapes the per-statement parser previously refused.
Downstream question-form admission for derived initial states
(case 0042) becomes a natural next unblock.
- The same Cartesian-product / "branches that disagree → refuse"
decision rule handles the new multi-candidate emissions; no
graph-side edits, no admissibility weakening.
- Highest-risk axis lands without breaching `wrong == 0`:
multi-candidate emission stays tightly scoped, refuses on every
documented adversarial probe, and the derived-value emissions
refuse on every shape-mismatch (mixed unit, scope-mismatch,
collision-prone same-unit).
- Future axes inherit the same axis-lane harness layout under
`evals/math_capability_axes/`.
---
## Out of scope
- **Solver changes.** If a multi-clause case parses but does not
solve, the gap is downstream; file a follow-up ADR (no solver
stubs, no admissibility relaxation).
- **Currency / numeric-literal handling.** Case 0026 (`$40`) stays
refused; the G.3 numeric-literals axis is the natural place.
- **Three-way / ellipsis / cross-sentence shapes.** Deferred per
the closed-set discipline.
- **Probe runner contract.** ADR-0131.G pinned `run_lane` as the
legacy probe's contract; G.4 does not change that. The
candidate-graph probe (`report.json`) is the measurement surface
that moves.