feat(ADR-0131.G.5): aggregate answer composition — combined/together cues wired, axis lane 20/20, wrong==0 (#197 )

Closes the vocabulary gap: `combined` and `together` added to `_Q_TOTAL_RE`
and `_Q_ENTITY_RE` tail alternations. Both map to `entity=None` semantics;
the solver's existing sum path is unchanged.

Ships:
- Parser one-line regex extension (`generate/math_candidate_parser.py`)
- 20-case curated axis lane (`G5_aggregate/v1/`) — 5 shapes × 4 cues
- Runner + byte-equal report (20/20 pass, wrong=0)
- 25 tests covering cue vocab, 2/3-entity sums, degenerate aggregate,
  refusals, byte-equality, B3 regression guard, GSM8K safety rail
- ADR-0131.G.5

No admission movement on GSM8K probe (statement-parse bottleneck unchanged).

2026-05-23 19:42:55 -07:00

3.4 KiB

Raw Blame History

ADR-0131.G.5 — Aggregate Answer Composition

Status: Accepted
Parent: ADR-0131.G — GSM8K Coverage Probe
Date: 2026-05-23

Context

The aggregate-answer path — questions like "How many apples do they have altogether?" — was functionally complete before this ADR. The parser (_Q_TOTAL_RE in generate/math_candidate_parser.py) already emitted Unknown(entity=None, unit=<unit>) for aggregate cues, and the solver (generate/math_solver.py) already summed all terminal state entries matching the questioned unit when entity is None.

What was missing:

Vocabulary gap (now closed): "combined" and "together" were absent from _Q_TOTAL_RE's tail alternation, causing questions using those cues to be refused even when the solver would have produced the correct sum.
No pinned lane: no curated axis cases proved the 2-entity, 3-entity, and degenerate aggregate paths end-to-end through parse_and_solve.

Decision

Closed aggregate-cue vocabulary

Exactly four cues are admitted:

Cue	Example tail
`in total`	"How many apples do they have in total?"
`altogether`	"How many apples do they have altogether?"
`combined`	"How many apples do they have combined?"
`together`	"How many apples do they have together?"

All four map to entity=None semantics — the solver sums all state entries whose unit matches the questioned unit, across all entities.

Solver path (pre-existing)

The entity is None branch in _resolve_unknown was not changed. It sums v for (_, unit), v in state.items() if unit == unknown.unit. This ADR extends the cue vocabulary and pins the lane, not the solver.

Axis lane

20 curated cases at evals/math_capability_axes/G5_aggregate/v1/cases.jsonl:

Shape	Count	Purpose
2-entity sum, no operations	4	one case per cue
3-entity sum, no operations	4	one case per cue
2-entity sum with add/subtract op	4	mixed cues
Single-entity degenerate	4	regression guard
Refusal: outside closed cue	4	wrong==0 probe

Refusal cases use question forms outside the closed _Q_TOTAL_RE pattern (e.g., "How many apples does everyone have?", "What is the total number of coins?") to verify the parser correctly refuses paraphrases not in the closed cue set.

Gate

wrong == 0 on every axis case. GSM8K admitted_wrong == 0 preserved (no admission movement expected — all 50 sample cases still refuse at statement parsing; question-layer work cannot lift that).

Deferred

Implicit aggregation without a cue word: "How many apples do Sam and Tom have?" requires coreference resolution (named-entity → pronoun-equivalent grouping). Out of scope for the closed-cue model.
Rate-based aggregation: "How many dollars did they earn in total?" where the unit derives from a rate operation. Requires rate-verb support in the statement parser.
GSM8K admission lift: all 50 sample cases fail at statement parsing (rate verbs, compound sentences, implicit entities). Question-layer cue extensions cannot move that number.

Evidence

Axis runner: evals/math_capability_axes/G5_aggregate/v1/runner.py
Report: evals/math_capability_axes/G5_aggregate/v1/report.json
Tests: tests/test_adr_0131_G5_aggregate.py
B3 lane unchanged.
GSM8K admitted_wrong == 0 preserved.

3.4 KiB Raw Blame History