core/docs/decisions/ADR-0131.G.4-multi-clause.md
Shay de26d7f792 feat(ADR-0131.G.4): multi-clause composition (conj subjects + conj objects + embedded quantifiers + conj embedded) — admission 0/50 (Δ0), multi-clause refusals 2→1
Highest-risk axis of the ADR-0131.G capability iteration: within-
sentence multi-clause composition. Four extractors land in the
candidate-emitting parser; no graph-side or solver changes.

Parser extension (generate/math_candidate_parser.py)
- _conj_subject_each_candidates: '<A> and [his/her/their <kin>] <B>
  each <verb> <N> <unit>' → 2 CandidateInitial (one per actor).
- _conj_object_candidates: '<E> has <N1> <unit1> and <N2> <unit2>' →
  2 CandidateInitial for the same entity; same-unit conjuncts refuse
  (would silently collide under solver overwrite-on-collision).
- _embedded_quantifier_candidates: '<E> has <N> <container> with <M>
  <unit> in each [<container>]' → 1 derived CandidateInitial
  (value=N*M).
- _embedded_quantifier_candidates (conj branch): '... <N1> <C> with
  <M1> <U> in each ... and <N2> <C> with <M2> <U> in each ...' → 1
  SUM CandidateInitial (value=N1*M1+N2*M2); mixed-unit refuses.
- CandidateInitial anchor whitelist widened to include
  saved/earned/got/received/bought/made/paid (and inflections) —
  narrow widening needed for the conjoined-subject-each shape.

Closed-set discipline
- Distributive 'each' only — 'each ... together/altogether' refuses.
- Two-way conjunction only — 3-way refuses by non-match.
- Cross-sentence coreference stays refused (within-sentence axis).
- Ambiguous 'each' scope refuses (container2 must agree).

Curated axis lane (32 cases)
- evals/math_capability_axes/G4_multi_clause/v1/cases.jsonl:
  conj_subject_each ×6, conj_object ×6, embedded_quantifier ×6,
  conj_embedded ×6, refusal ×8.
- evals/math_capability_axes/G4_multi_clause/v1/runner.py +
  report.json: deterministic; wrong==0 gate; byte-equal across runs.

Tests (26 new)
- tests/test_adr_0131_G4_multi_clause.py: per-shape emission,
  refusal probes (parametric), distributive-only policy,
  cross-sentence refusal, runner byte-equality, GSM8K-probe gate.

GSM8K-probe gate (chosen: multi-clause refusals ↓)
- evals/gsm8k_math/train_sample/v1/report.json (candidate-graph
  probe): multi-clause statement-refusal count 2 → 1. Case 0042
  ('Ella has 4 bags with 20 apples in each bag and six bags with 25
  apples in each bag.') moves from statement-clause refusal to
  question-layer refusal. Case 0026 ('Aaron and his brother Carson
  each saved up $40') stays refused on the '$' value slot
  (deferred to G.3 numeric-literals axis).
- evals/gsm8k_math/train_sample/v1/train_sample_coverage_report.json
  (legacy probe): refreshed, byte-identical (legacy parser
  untouched).

B3 + candidate-graph + GSM8K probe lanes all pass (95/95
regression). wrong==0 preserved everywhere — load-bearing for the
highest-risk axis.
2026-05-23 14:43:16 -07:00

12 KiB
Raw Blame History

ADR-0131.G.4 — Capability axis: multi-clause composition (conjoined subjects, conjoined objects, embedded quantifiers)

Status: Proposed Date: 2026-05-23 Author: CORE agents + reviewers Parent: ADR-0131.G Depends on: ADR-0126 (candidate graph), ADR-0127 (substance qualifier policy), ADR-0131.3, ADR-0132..ADR-0135


Context

GSM8K paragraphs frequently introduce starting state via within-sentence composition the per-statement candidate parser refuses today:

Baseline refusal sentence Capability missing
Aaron and his brother Carson each saved up $40 ... conjoined subject + distributive each
Francine has five full boxes of crayons and 5 loose crayons conjoined object NPs sharing a verb
Ella has 4 bags with 20 apples in each bag and six bags with 25 apples in each bag embedded quantifier + conjunction

This is the highest-risk axis of ADR-0131.G's four near-term capability extensions. Multi-clause emission means the round-trip filter does more work; multi-candidate ambiguity makes confabulation risk higher. Refusal-first stays paramount: admission gains must be small and load-bearing.


Decision

Land four within-sentence multi-clause extractors in generate/math_candidate_parser.py. All emit CandidateInitial records (initial state, not operations — the shapes here introduce starting holdings, they do not mutate state):

Extractor Shape (closed set) Emission
_conj_subject_each_candidates <A> and [his/her/their <kin>] <B> each <verb> <N> <unit> two CandidateInitial (one per actor), same (N, unit)
_conj_object_candidates <E> has <N1> <unit1> and <N2> <unit2> two CandidateInitial for the same entity; same-unit conjuncts refuse
_embedded_quantifier_candidates <E> has <N> <container> with <M> <unit> in each [<container>] one derived CandidateInitial with value = N*M, unit = <unit>
_embedded_quantifier_candidates (conj branch) <E> has <N1> <C> with <M1> <U> in each ... and <N2> <C> with <M2> <U> in each ... one SUM CandidateInitial with value = N1*M1 + N2*M2, unit = <U>; mixed-unit conjuncts refuse

Wired into the existing extract_initial_candidates public entry point — the binding graph (math_candidate_graph._filtered_statement_choices) already consumes through that path; no graph-side edit required (the read-only audit concluded composed candidates are reachable through existing edges).

Closed-set discipline

  • Distributive each only. Surface markers together, in total, altogether immediately abort emission (explicit contradiction with distributive reading). Pinned by test_refuses_each_with_together / test_collective_without_each_refuses.
  • Two-way conjunction only. Three-way A and B and C each ... is out of closed-set shape and refuses by non-match.
  • Same-unit conjoined object refuses. Two same-unit conjuncts on the same entity (Sam has 5 dimes and 3 dimes) would silently collide under the solver's state[(entity, unit)] overwrite semantics (math_solver.py:206); refusing keeps wrong == 0.
  • Ambiguous each scope refuses. Ella has 4 bags with 20 apples in each boxboxbags ⇒ refuse.
  • Mixed-unit conjoined embedded refuses. Apples + pears cannot be summed.
  • No cross-sentence state. Multi-sentence inputs are processed per-sentence; pronouns / coreference across sentences stay refused (out of within-sentence axis scope).

CandidateInitial anchor widening

CandidateInitial.__post_init__ whitelists a narrow set of initial-state-introducing verbs needed for the conjoined-subject-each shape (saved, earned, got, received, bought, made, paid, plus their inflected variants). The widening is keyed on lowercase tokens; the _token_in check in math_candidate_graph._initial_admissible confirms the anchor word appears in source. Verb-class widening for the general parser remains G.1's scope; G.4 widens only what conjoined-subject-each needs.

Derived-value provenance

Embedded-quantifier and conjoined-embedded emissions carry a derived value (N*M and N1*M1 + N2*M2 respectively) that does not appear as a single source token. The round-trip filter's "value grounds in source" check (_value_grounds) is satisfied by anchoring matched_value_token on the per-container M (or first per-container M1 for the sum). This is a deliberate, documented widening of the source-grounding spirit: the components of the derived value all appear in the source, and the parser commits to the canonical arithmetic composition. Refusing on component-mismatch (mixed units, wrong container scope) and refusing on indefinite quantifiers in any value slot together keep the derivation honest. The alternative — emitting two flat candidates for conjoined-embedded — was rejected: under the solver's overwrite-on-collision semantics it would silently drop one conjunct's contribution, breaching wrong == 0.

No graph-side edits

math_candidate_graph.py is unchanged. Multi-candidate emission flows through the existing per-sentence choice space + Cartesian product; conjoined-subject-each and conjoined-object emissions land in _filtered_statement_choices like any other initial candidate. The solver's state[(entity, unit)] model naturally accommodates distinct entities (each-shape) and distinct units (object-shape); collision-prone shapes refuse at the parser. This decision is the "read-only audit only edit if composed candidate is unreachable" posture from the brief.

Curated coverage cases (G.4 axis lane)

evals/math_capability_axes/G4_multi_clause/v1/cases.jsonl32 cases:

Category Cases Notes
conj_subject_each 6 incl. kin-appositive, word-form value
conj_object 6 distinct-unit conjuncts only
embedded_quantifier 6 with + without explicit container2
conj_embedded 6 same-unit only
refusal 8 together / altogether / 3-way / cross-entity / scope-mismatch / mixed-unit / cross-sentence / same-unit collision

Runner emits a deterministic report.json; wrong == 0 is the gate.

Deferred (out of scope for G.4)

  • Cross-sentence coreference (Aaron has 5. He gives 2 to Bob.) — needs per-discourse state; pinned as refusal probe.
  • Ellipsis (Aaron has 5 apples, Carson 3) — needs verb reconstruction; pinned out-of-scope.
  • Three-way+ conjunctions (A and B and C) — combinatorial explosion + ambiguity; deferred to a future axis.
  • Collective readings (A and B saved $40 together) — explicitly refused; collective semantics needs a different binding-graph node type.
  • Currency / unit prefix ($40) — refused at the value slot (the $ is not a _VALUE character). Deferred to G.3 numeric-literals axis, which is the natural place for currency / percentage / decimal literal handling. Documented impact on the GSM8K probe gate (case 0026 stays refused).
  • Same-unit conjoined object summation — would require either parser-side sum (analogous to conj-embedded) or solver-side state-merge; deferred until a sum-shaped CandidateInitial proves necessary outside this axis.
  • Solver / binding-graph changes. If a multi-clause case parses but does not solve, that's a downstream gap and gets its own ADR.

GSM8K-probe gate (chosen)

G.4 gates on:

Multi-clause statement-clause refusals in the candidate-graph probe (evals/gsm8k_math/train_sample/v1/report.json) strictly decrease.

Counter (in test_gsm8k_candidate_graph_multi_clause_refusals_decreased) matches refused cases citing a statement-clause refusal whose embedded sentence text contains a multi-clause anchor pattern (each <init-verb>, with N <unit> in each, or has N <unit> and N <unit>).

Probe report Baseline (origin/main 481e0c3) After G.4 Δ
Multi-clause statement-clause refusals (report.json) 2 1 1
wrong (report.json) 0 0 0
admission_rate (report.json) 0/50 0/50 0
Legacy train_sample_coverage_report.json byte-identical byte-identical 0

Baseline-2 cases: gsm8k-train-sample-v1-0026 (Aaron and his brother Carson each saved up $40 ... — refused on $40 value slot; deferred to G.3) and -0042 (Ella has 4 bags with 20 apples in each bag and six bags with 25 apples in each bag. — now parses, refusal moves to question layer). admission_rate does not rise because downstream layers (question-form admission for derived initial states) are out of G.4 scope.

Legacy probe report (refreshed, byte-identical)

evals/gsm8k_math/train_sample/v1/train_sample_coverage_report.json runs through generate.math_parser.parse_problem (legacy first-match-wins), which G.4 does not touch. Refreshed and pinned via test_gsm8k_legacy_probe_safety_rail_intact.


Invariants

  • g4_wrong_count_is_zero — every G.4 axis case passes or refuses; no case admits a wrong shape. Pinned by test_runner_wrong_count_is_zero.
  • g4_closed_set_refusals_hold — all 8 refusal probes admit zero multi-clause candidates. Pinned by test_refusal_cases_emit_no_admitted_multi_clause (parametric).
  • g4_distributive_each_onlyeach ... together and each ... altogether refuse. Pinned by test_refuses_each_with_together.
  • g4_cross_sentence_refuses — multi-clause extractors do not fire across sentence boundaries. Pinned by test_cross_sentence_pronoun_refuses_multi_clause.
  • g4_report_deterministicreport.json is byte-equal across back-to-back runs.
  • gsm8k_safety_rail_intactadmitted_wrong == 0 on both GSM8K probe reports.
  • gsm8k_multi_clause_refusal_strictly_decreased — chosen G.4 gate.

Acceptance evidence

  • evals/math_capability_axes/G4_multi_clause/v1/runner.py exits 0 with wrong == 0 on all 32 curated cases.
  • tests/test_adr_0131_G4_multi_clause.py (26 tests): per-shape emission, refusal-set, distributive-only policy, cross-sentence refusal, runner byte-equality, GSM8K-probe gate.
  • Candidate-graph probe report.json: multi-clause statement refusal count 2 → 1 (case 0042 moves from statement to question refusal).
  • Legacy probe train_sample_coverage_report.json refreshed and byte-identical.
  • B3 lane + ADR-0126 candidate-graph tests + ADR-0131.G probe tests all pass (95/95 across the regression sweep).

Consequences

  • The candidate-graph topology can now see four multi-clause initial-state shapes the per-statement parser previously refused. Downstream question-form admission for derived initial states (case 0042) becomes a natural next unblock.
  • The same Cartesian-product / "branches that disagree → refuse" decision rule handles the new multi-candidate emissions; no graph-side edits, no admissibility weakening.
  • Highest-risk axis lands without breaching wrong == 0: multi-candidate emission stays tightly scoped, refuses on every documented adversarial probe, and the derived-value emissions refuse on every shape-mismatch (mixed unit, scope-mismatch, collision-prone same-unit).
  • Future axes inherit the same axis-lane harness layout under evals/math_capability_axes/.

Out of scope

  • Solver changes. If a multi-clause case parses but does not solve, the gap is downstream; file a follow-up ADR (no solver stubs, no admissibility relaxation).
  • Currency / numeric-literal handling. Case 0026 ($40) stays refused; the G.3 numeric-literals axis is the natural place.
  • Three-way / ellipsis / cross-sentence shapes. Deferred per the closed-set discipline.
  • Probe runner contract. ADR-0131.G pinned run_lane as the legacy probe's contract; G.4 does not change that. The candidate-graph probe (report.json) is the measurement surface that moves.