0770648257
* feat(adr-0189): comparative reading — anchor-verb widening + multi-word units
The candidate-graph comparative extractor (ADR-0131.G.2) read only has/have +
single-word units, so real-GSM8K comparatives ('Brooke does three times as many
jumping jacks as Sidney') didn't parse — a dark statement in 17 places blocking
15 of the 47 refused train_sample cases, despite the ADR-0123 solver already
supporting compare_additive/compare_multiplicative.
Widens the anchor-verb set (reusing legacy vetted lemmas + does/collected/
gained/studied…), EXCLUDING polarity-inverting verbs (lose/spend/give/sell/win)
to preserve wrong=0; admits 1-2 word units via the existing multi-word
_unit_grounds branch. Feeds the existing solver unchanged.
wrong=0 proven: G2_comparatives 29/29, G3 20/0, G4 32/32, train_sample 3/47/0
byte-identical; polarity-inverting verbs proven refused (failing-under-violation).
Chain composes correctly in isolation (146 -> 438). Flips 0 cases ALONE — every
comparative case needs a composing partner (aggregation / multi-word-noun
injection); this ships the component, not yet a flip.
- generate/math_candidate_parser.py: _comparison_anchor_verb widening + 1-2 word
unit slots in the two multiplicative comparative regexes.
- tests/test_adr_0131_G2a_*: 5 tests incl. polarity-inversion wrong=0 guards.
- docs/decisions/ADR-0189: gap, change, wrong=0 evidence, honest scope.
* feat(adr-0189a): first metric move 3/47/0 -> 4/46/0 (case 0024, comprehension-composed)
Case 0024 now SOLVES (answer 438) by composing three general comprehension
capabilities feeding the unchanged ADR-0123 solver:
1. day-of-week count enumeration: Sidney = 20+36+40+50 = 146
(_day_enumeration_candidates; derived sum grounds via first count token,
mirroring _embedded_quantifier; closed to the 7 day names)
2. comparative reading (ADR-0189): Brooke = 3 x Sidney
3. activity question 'How many <unit> did <Entity> <verb>?' (_Q_DID_RE)
Plus do/does/did added to the CandidateInitial anchor whitelist (production-
possession), admitted only via the closed day-enumeration shape.
wrong=0 PROVEN across every lane: all 8 capability axes wrong=0 (G2_comparatives
29/29, G3 20/0, G4 32/32, G5/S1/S3/S4 all pass), train_sample 4/46/0 wrong=0,
verify_lane_shas exit 0 (no pinned lane changed), generate_claims --check OK.
872 tests pass; new tests are failing-under-violation incl. wrong=0 guards
(non-day comma list not summed; polarity-inverting comparative verbs refused).
Re-baselined report.json + train_sample_coverage_report.json (latter also clears
pre-existing reason drift) + CLAIMS.md to the new 4/46/0 metric. Decode-not-guess:
0024 solved by READING its structure, not storing an answer. Remaining pre-existing
failures (G3 committed-report, telemetry) unrelated, fail on pristine main.
- generate/math_candidate_parser.py: day-enum extractor + _Q_DID_RE + does-anchor.
- tests/test_adr_0189a_day_enum_activity.py: 5 tests (incl. end-to-end 0024=438).
- docs/decisions/ADR-0189a + report.json/coverage/CLAIMS re-baseline.
77 lines
4 KiB
Markdown
77 lines
4 KiB
Markdown
# ADR-0189 — Comparative reading: anchor-verb widening + multi-word units
|
||
|
||
**Status:** Proposed (implemented in this PR). Extends
|
||
[ADR-0131.G.2](./ADR-0131-G-gsm8k-grammar-coverage.md) (candidate-graph
|
||
comparative extractors). Serving-path capability; landed wrong=0-proven.
|
||
|
||
> **One line.** The candidate-graph comparative extractor read only `has`/`have`
|
||
> + single-word units, so real-GSM8K comparatives (`Brooke **does** three times
|
||
> as many **jumping jacks** as Sidney`) didn't parse. This widens the anchor-verb
|
||
> set (excluding polarity-inverting verbs) and admits 1–2-word units. It feeds the
|
||
> existing ADR-0123 `compare_multiplicative`/`compare_additive` solver; wrong=0 is
|
||
> preserved (G2_comparatives 29/29; train_sample 3/47/0 byte-identical).
|
||
|
||
---
|
||
|
||
## 1. The gap (microscope finding, 2026-05-29)
|
||
|
||
The candidate-graph parser emits **no comparative operation** for the most common
|
||
real-GSM8K comparative surfaces, even though the **solver already supports them**
|
||
(ADR-0123 `compare_additive`/`compare_multiplicative`, fully wired in
|
||
`math_solver`). Measured: comparatives are a **dark statement in 17 places,
|
||
blocking 15 of the 47 refused `train_sample` cases**.
|
||
|
||
Two extractor limits caused the misses (`_comparison_anchor_verb()` was
|
||
`(?:has|have)` and units were `\w+`):
|
||
|
||
- **Verb coverage.** `does`/`collected`/`gained`/`studied`/`makes`/… were not
|
||
recognized. (ADR-0131.G.2 deferred this explicitly: "past-tense + lemma-widening
|
||
are deferred … to keep the precedence story narrow" — a scope limit, not a
|
||
wrong=0 constraint.)
|
||
- **Multi-word units.** `jumping jacks` (case 0024) could not match.
|
||
|
||
## 2. The change
|
||
|
||
- `generate/math_candidate_parser._comparison_anchor_verb()` — widen to the
|
||
already-vetted legacy `math_parser._COMPARE_VERB` lemmas plus the
|
||
production/activity verbs seen in real GSM8K comparatives. **Deliberately
|
||
EXCLUDES** polarity-inverting verbs (`lose/lost`, `win/won`, `spend/spent`,
|
||
`use/used`, `give/gave`, `sell/sold`) — admitting them could read a comparison
|
||
backwards and breach wrong=0.
|
||
- The two multiplicative regexes admit an optional second unit word
|
||
(`\w+(?:\s+\w+)?`), grounded via the existing multi-word branch of
|
||
`_unit_grounds`.
|
||
|
||
## 3. wrong=0 evidence
|
||
|
||
- **G2_comparatives lane: 29/29, wrong=0** (the dedicated comparative lane).
|
||
- G3_numerics 20 correct / 0 wrong; G4_multi_clause 32/32; train_sample
|
||
**3/47/0 byte-identical** (no flip, no wrong).
|
||
- Polarity-inverting verbs proven refused (`test_…_polarity_inverting_verb_not_admitted`,
|
||
`test_spend_verb_not_admitted`) — failing-under-violation.
|
||
- The round-trip filter is unchanged: the comparator anchor (`twice`/`N times`/
|
||
`more`/`fewer`) and the reference actor still must ground.
|
||
|
||
## 4. Honest scope — a necessary component, not a standalone solve
|
||
|
||
This widening flips **zero** `train_sample` cases by itself, because every
|
||
comparative-blocked case also needs a composing partner the reader still lacks
|
||
(e.g. case 0024 needs Sidney's `20+36+40+50=146` day-of-week aggregation **and**
|
||
multi-word counted-noun injection before `Brooke = 3 × 146 = 438` can solve). The
|
||
comparative + question chain is proven to compose correctly in isolation
|
||
(`Sidney has 146 apples. Brooke has three times as many apples as Sidney.` →
|
||
**438**). This ADR ships the comparative component; the first metric flip
|
||
(`3/47/0 → 4/46/0`) lands when its companion capability (aggregation /
|
||
multi-word-noun injection) lands for a shared target case.
|
||
|
||
## 5. Why this obeys the standing principles
|
||
|
||
- **Decode, don't guess.** The engine could already *reason* about comparison
|
||
(ADR-0123 solver); this teaches it to *read* the comparison — closing a
|
||
comprehension gap, not adding a stored answer.
|
||
- **General, not overfit.** A widened closed verb set + multi-word units is a
|
||
fundamental construction across 15 cases, not a per-case surface regex.
|
||
- **wrong=0 > coverage.** Polarity-inverting verbs excluded; G2 lane 29/29;
|
||
serving 3/47/0 byte-identical; round-trip filter untouched.
|
||
- **No contradiction** with in-use ADRs: extends ADR-0131.G.2's extractor and
|
||
feeds the ADR-0123 solver unchanged.
|