feat(adr-0194): labeled-container subject entity shape — 'Jar A has N' parses, wrong=0-proven (substrate) (#499)
GSM8K labels containers/regions with a trailing single-letter or short-numeric
label ('Jar A has 28 marbles', 'Section G has 10 cars', 'District 2 has 19
voters'); the initial-possession entity slot captured only 'Jar' and the label
broke the match. Adds a separate sibling pattern _INITIAL_HAS_LABELED_RE
(mirroring ADR-0136.S.4 localisation) that REQUIRES the label, so the global
_ENTITY is unchanged and bare subjects yield no duplicate candidate.
- Composes with ADR-0193 aggregate question: 'Jar A has 28 marbles. Jar B has
12 marbles. How many marbles are there in total?' -> 40.0.
- 0 real-corpus metric flip (honest substrate): the one real multi-container
aggregate additionally needs comparative + multiplicative + lowercase-ref.
- wrong=0 HOLDS full corpus (7,473 q); train_sample byte-identical 4/46/0;
synthetic-registry capability-axis gate + G5 lane green; smoke 67 passed.
- Label bounded by the possession verb: multi-word nouns ('Jar Apple') do NOT
match. wrong=0 held downstream by completeness + round-trip + disagreement.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
39bf549e94
commit
84db129629
3 changed files with 234 additions and 0 deletions
84
docs/decisions/ADR-0194-labeled-container-subject.md
Normal file
84
docs/decisions/ADR-0194-labeled-container-subject.md
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
# ADR-0194 — Labeled-container subject entity shape
|
||||
|
||||
**Status:** Proposed (implemented in this PR).
|
||||
**Extends:** [ADR-0136.S.4](./ADR-0136.S.4-novel-initial-form.md) (sibling-pattern
|
||||
localisation), [ADR-0123a](./ADR-0123a-inference-shape-synonym.md) (entity slot).
|
||||
**Composes with:** [ADR-0193](./ADR-0193-aggregate-existential-question-frame.md)
|
||||
(aggregate question frame).
|
||||
**Substrate: 0 real-corpus metric flip by design; the value is the entity-shape
|
||||
generalisation + proven composition with the aggregate question.**
|
||||
|
||||
> **One line.** GSM8K labels containers/regions with a trailing single-letter
|
||||
> or short-numeric label ("Jar A has 28 marbles", "Section G has 10 cars",
|
||||
> "District 2 has 19 voters"). The initial-possession entity slot
|
||||
> (`_ENTITY = (?:[A-Z]\w+|[Tt]he\s+\w+)`) captures only "Jar" and then expects
|
||||
> the possession verb, so the label breaks the match and the statement parses
|
||||
> to nothing. This adds a separate sibling pattern that REQUIRES a label.
|
||||
|
||||
---
|
||||
|
||||
## 1. The gap
|
||||
|
||||
Both reader paths reject the labeled subject:
|
||||
- the candidate parser's `_INITIAL_HAS_RE` (`extract_initial_candidates` → 0
|
||||
candidates);
|
||||
- the recognizer's discrete_count anchor (proper-noun single-token subject →
|
||||
0 anchors).
|
||||
|
||||
"Jamie has 28 marbles" parses (1 candidate); "Jar A has 28 marbles" does not —
|
||||
purely because of the trailing label.
|
||||
|
||||
## 2. Decision
|
||||
|
||||
Add `_INITIAL_HAS_LABELED_RE` in `generate/math_candidate_parser.py`, consumed
|
||||
by a dedicated `_init_has_labeled_candidates` helper wired into
|
||||
`extract_initial_candidates`:
|
||||
|
||||
```
|
||||
^<Noun> <label> (has|have|had|started) <value> [adj] [unit] [of/in/for/with …]
|
||||
label = a single uppercase letter OR 1-2 digits
|
||||
```
|
||||
|
||||
- **Sibling-pattern localisation** (mirrors ADR-0136.S.4's
|
||||
`_INITIAL_HAS_INDEF_RE`): the global `_ENTITY` is unchanged for every other
|
||||
path (operations, comparisons, questions).
|
||||
- **The label is required**, so a bare subject ("Jamie has 28 marbles") never
|
||||
reaches this pattern and yields no duplicate candidate.
|
||||
- **The label is bounded by the following possession verb**, so a multi-word
|
||||
noun does NOT match: "Jar Apple has 5 marbles" → no candidate ("Apple" is not
|
||||
a single-letter label), "Box Set has 12 items" → no candidate.
|
||||
- Same value/unit tail and money normalisation as `_INITIAL_HAS_RE`.
|
||||
|
||||
### Why this is safe (the firewall is the precondition)
|
||||
|
||||
The label widening only makes a statement *parse* into an initial possession.
|
||||
wrong=0 is held downstream by the completeness guard (ADR-0191) + the
|
||||
round-trip filter + branch disagreement — a mis-parse leaves source quantities
|
||||
uncovered and refuses. Full-corpus verification: **wrong=0 HOLDS** (7,473 q);
|
||||
`train_sample` byte-identical **4/46/0**; the synthetic-registry capability-axis
|
||||
`wrong=0` gate and the G5 aggregate lane both stay green.
|
||||
|
||||
## 3. Evidence
|
||||
|
||||
- **Composes with ADR-0193:** "Jar A has 28 marbles. Jar B has 12 marbles. How
|
||||
many marbles are there in total?" → **40.0**; three-container variant → 10.0.
|
||||
- **0 real-corpus flip** (honest framing): of the 3 real container-subject
|
||||
problems under an aggregate question, the only multi-container aggregate
|
||||
("Jar A has 28 marbles. Jar B has 12 more marbles than jar A. Jar C has twice
|
||||
as many as jar B. … altogether?") additionally requires **comparative-additive
|
||||
+ comparative-multiplicative + lowercase-reference** resolution. This is the
|
||||
composition-wall lesson again: an entity-shape widening is necessary, not
|
||||
sufficient.
|
||||
- **Tests:** `tests/test_labeled_container_subject.py` (labeled containers
|
||||
parse; bare subjects yield no duplicate; multi-word nouns don't match a
|
||||
label; composes with the aggregate question).
|
||||
|
||||
## 4. Consequences & follow-ups
|
||||
|
||||
- The labeled-container entity shape is banked; it becomes load-bearing the
|
||||
moment **comparative reading** (the actual aggregate blocker — ADR-0131.G.2 /
|
||||
`core-comparatives`) can compose `compare_additive`/`compare_multiplicative`
|
||||
ops into an aggregate answer, and the **lowercase-reference** ("jar A" inside
|
||||
a later clause) resolves to the labeled entity.
|
||||
- Compound enumeration and the recognizer-path labeled subject remain closed;
|
||||
open them only if a serving need is proven under the firewall.
|
||||
|
|
@ -256,6 +256,26 @@ _INITIAL_HAS_INDEF_RE: Final[re.Pattern[str]] = re.compile(
|
|||
r"\s*\.?$"
|
||||
)
|
||||
|
||||
# ADR-0194 — labeled-container subject: "Jar A has 28 marbles.",
|
||||
# "Section G has 10 cars.", "District 2 has 19 voters.". GSM8K labels
|
||||
# containers/regions with a trailing single-letter or short-numeric label
|
||||
# that the bare _ENTITY slot cannot absorb. Sibling to _INITIAL_HAS_RE that
|
||||
# REQUIRES the label, so it never duplicates the bare-subject candidate;
|
||||
# _ENTITY stays unchanged for every other path. The label is a single
|
||||
# uppercase letter OR 1-2 digits, bounded by the following possession verb
|
||||
# (so a multi-word noun like "Jar Apple" does NOT match — "Apple" is not a
|
||||
# single-letter label). Same value/unit tail as _INITIAL_HAS_RE. wrong=0
|
||||
# is held downstream (completeness + round-trip + disagreement).
|
||||
_INITIAL_HAS_LABELED_RE: Final[re.Pattern[str]] = re.compile(
|
||||
r"^(?P<entity>[A-Z]\w+\s+(?:[A-Z]|\d{1,2}))\s+"
|
||||
r"(?P<anchor>has|have|had|started)(?:\s+(?:up|with))?\s+"
|
||||
rf"(?P<value>{_VALUE})"
|
||||
r"(?:\s+(?:full|loose|empty|whole|broken|new|old|small|large|fresh|raw|flat))?"
|
||||
r"(?:\s+(?P<unit>\w+))?"
|
||||
r"(?:\s+(?:of|in|for|with)\s+.+)?"
|
||||
r"\s*\.?$"
|
||||
)
|
||||
|
||||
# ADR-0136.S.4 — Shape B: prepositional-prefix existential.
|
||||
# "In a building, there are a hundred ladies on the first-floor studying."
|
||||
# Sibling to _INITIAL_THERE_ARE_RE; prefix is "In a <place>" (not bare
|
||||
|
|
@ -557,6 +577,9 @@ def extract_initial_candidates(sentence: str) -> list[CandidateInitial]:
|
|||
# ADR-0136.S.4 — Shape A: "A <noun> has N <unit>" indefinite-article subject.
|
||||
out.extend(_init_has_indef_candidates(sentence))
|
||||
|
||||
# ADR-0194 — labeled-container subject: "Jar A has 28 marbles."
|
||||
out.extend(_init_has_labeled_candidates(sentence))
|
||||
|
||||
m2 = _INITIAL_THERE_ARE_RE.match(s)
|
||||
if m2 is not None:
|
||||
value_raw = m2.group("value")
|
||||
|
|
@ -1946,6 +1969,52 @@ def _init_has_indef_candidates(sentence: str) -> list[CandidateInitial]:
|
|||
return []
|
||||
|
||||
|
||||
def _init_has_labeled_candidates(sentence: str) -> list[CandidateInitial]:
|
||||
"""ADR-0194 — labeled-container subject: 'Jar A has 28 marbles.'
|
||||
|
||||
Sibling to the _INITIAL_HAS_RE block in extract_initial_candidates.
|
||||
Entity is '<Noun> <label>' (label = single uppercase letter or 1-2
|
||||
digits), preserved by _normalize_entity. REQUIRES a label, so a
|
||||
bare subject ('Jamie has 28 marbles') never reaches here and yields
|
||||
no duplicate candidate. Same value/unit resolution and money
|
||||
normalization as the definite-subject path.
|
||||
"""
|
||||
s = sentence.strip().rstrip(".")
|
||||
m = _INITIAL_HAS_LABELED_RE.match(s)
|
||||
if m is None:
|
||||
return []
|
||||
value_raw = m.group("value")
|
||||
rv = _resolve_value(value_raw)
|
||||
if rv is None:
|
||||
return []
|
||||
entity = _normalize_entity(m.group("entity"))
|
||||
unit_raw = m.group("unit")
|
||||
if rv.unit_override is not None:
|
||||
resolved_unit: str = rv.unit_override
|
||||
elif unit_raw is not None:
|
||||
resolved_unit = _canonicalize_unit(unit_raw)
|
||||
else:
|
||||
return []
|
||||
value, final_unit = _money_unit_normalization(rv.value, resolved_unit)
|
||||
assert final_unit is not None
|
||||
try:
|
||||
return [
|
||||
CandidateInitial(
|
||||
initial=InitialPossession(
|
||||
entity=entity,
|
||||
quantity=Quantity(value=value, unit=final_unit),
|
||||
),
|
||||
source_span=sentence,
|
||||
matched_anchor=m.group("anchor"),
|
||||
matched_value_token=value_raw,
|
||||
matched_unit_token=unit_raw if unit_raw is not None else final_unit,
|
||||
matched_entity_token=m.group("entity"),
|
||||
)
|
||||
]
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
|
||||
def _init_there_are_prefix_candidates(sentence: str) -> list[CandidateInitial]:
|
||||
"""Shape B — prepositional-prefix existential: 'In a building, there are 100 ladies.'
|
||||
|
||||
|
|
|
|||
81
tests/test_labeled_container_subject.py
Normal file
81
tests/test_labeled_container_subject.py
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
"""ADR-0194 — labeled-container subject entity shape.
|
||||
|
||||
GSM8K routinely labels containers/regions with a trailing single-letter or
|
||||
short-numeric label: "Jar A has 28 marbles.", "Section G has 15 rows.",
|
||||
"District 2 has 19 voters.". The initial-possession parser's entity slot
|
||||
(``_ENTITY = (?:[A-Z]\\w+|[Tt]he\\s+\\w+)``) captures only "Jar" and then
|
||||
expects the possession verb, so the label breaks the match and the statement
|
||||
produces no candidate.
|
||||
|
||||
This adds a SEPARATE sibling pattern ``_INITIAL_HAS_LABELED_RE`` (mirroring
|
||||
ADR-0136.S.4's ``_INITIAL_HAS_INDEF_RE`` localisation) that REQUIRES a label,
|
||||
so it never duplicates the bare-subject main pattern. The global ``_ENTITY``
|
||||
is unchanged. wrong=0 is held downstream by the completeness guard
|
||||
(ADR-0191) + round-trip + branch disagreement — the label widening only makes
|
||||
a statement *parse*; a mis-parse leaves quantities uncovered and refuses.
|
||||
|
||||
SUBSTRATE: 0 real-corpus metric flip (the one real multi-container aggregate,
|
||||
"Jar A has 28 marbles. Jar B has 12 more than jar A. Jar C has twice as many
|
||||
as jar B. ...altogether?", additionally needs comparative + multiplicative
|
||||
reading). Its value is the entity-shape generalisation + that it composes
|
||||
with the ADR-0193 aggregate question (proven below).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from generate.math_candidate_parser import extract_initial_candidates
|
||||
from generate.math_candidate_graph import parse_and_solve
|
||||
|
||||
|
||||
# --- Labeled-container subjects now parse as initial possessions -----------
|
||||
@pytest.mark.parametrize("sentence,entity,value,unit", [
|
||||
("Jar A has 28 marbles.", "Jar A", 28.0, "marbles"),
|
||||
("Box B has 15 marbles.", "Box B", 15.0, "marbles"),
|
||||
("Section G has 10 cars.", "Section G", 10.0, "cars"),
|
||||
("District 2 has 19 voters.", "District 2", 19.0, "voters"),
|
||||
("Tank 1 has 40 liters.", "Tank 1", 40.0, "liters"),
|
||||
])
|
||||
def test_labeled_container_parses(sentence, entity, value, unit) -> None:
|
||||
cands = extract_initial_candidates(sentence)
|
||||
assert len(cands) == 1, f"expected one candidate for {sentence!r}"
|
||||
c = cands[0]
|
||||
assert c.initial.entity == entity
|
||||
assert c.initial.quantity.value == value
|
||||
assert c.initial.quantity.unit == unit
|
||||
|
||||
|
||||
# --- No duplicate candidates for bare (unlabeled) subjects -----------------
|
||||
def test_bare_subject_single_candidate_unchanged() -> None:
|
||||
"""The labeled pattern requires a label, so 'Jamie has 28 marbles'
|
||||
still yields exactly one candidate (from the main pattern only)."""
|
||||
cands = extract_initial_candidates("Jamie has 28 marbles.")
|
||||
assert len(cands) == 1
|
||||
assert cands[0].initial.entity == "Jamie"
|
||||
|
||||
|
||||
# --- The label must not swallow a following content word -------------------
|
||||
@pytest.mark.parametrize("sentence", [
|
||||
"Jar Apple has 5 marbles.", # 'Apple' is not a single-letter label
|
||||
"Box Set has 12 items.", # 'Set' is not a label
|
||||
])
|
||||
def test_multiword_noun_not_a_label(sentence: str) -> None:
|
||||
cands = extract_initial_candidates(sentence)
|
||||
assert cands == [], f"{sentence!r} must not parse as a labeled container"
|
||||
|
||||
|
||||
# --- Composes with the ADR-0193 aggregate question -------------------------
|
||||
def test_composes_with_aggregate_question() -> None:
|
||||
res = parse_and_solve(
|
||||
"Jar A has 28 marbles. Jar B has 12 marbles. "
|
||||
"How many marbles are there in total?"
|
||||
)
|
||||
assert res.answer == 40.0, res.refusal_reason
|
||||
|
||||
|
||||
def test_composes_three_containers() -> None:
|
||||
res = parse_and_solve(
|
||||
"Jar A has 5 marbles. Jar B has 3 marbles. Jar C has 2 marbles. "
|
||||
"How many marbles are there altogether?"
|
||||
)
|
||||
assert res.answer == 10.0, res.refusal_reason
|
||||
Loading…
Reference in a new issue