feat(adr-0163-f2): EX-6 hyphen-bonded unit extraction — pseudo-accumulation confusers refuse (wrong 7->5) (#473)
The confuser probe's two pseudo-accumulation misfires (0005 ->796, 0007 ->996)
both traced to the same extraction blind spot: a number bonded to its unit by a
hyphen (`25-foot sections`, `20-inch pieces`) was invisible to the base
`number + space + word` pattern, so the self-verification completeness clause
never saw the divisor and the bare `buys ... gives` accumulation read as
"complete". This is the highest-leverage lever the 2026-05-29 session named
("the gate must see the fractions/25-foot it currently misses").
EX-6 adds a tight, ADR-0165-safe lexeme pass: a digit run, a single hyphen, an
alphabetic unit word. The alphabetic-only unit group keeps numeric ranges (`3-5`)
out; taking only the first hyphen segment keeps the postmodifier tail
(`25-year-old`) from inflating the unit — so it stays clear of the deferred EX-3
multi-word-unit traps. Over-extraction here is strictly refuse-preferring: making
the divisor visible drives 0005/0007 to refuse via the polarity-None
`cuts`/`splits` clause, never to a wrong answer.
Evidence (deterministic, the microscope):
- confuser probe: wrong 7 -> 5; pseudo-accumulation 0 wrong / 4 refused;
genuine positives still 7 solved; pair-tells unchanged (4).
- train_sample (capability): 3/47/0 byte-identical.
- practice accumulation: 3/47/0 (wrong=0) byte-identical.
- smoke 67 passed; lane-SHA freeze 8/8 (serving frozen).
Tests:
- TestEX6HyphenatedUnitNumbers pins the new lexeme (value+unit, decimal, no
double-count, word-compound unaffected, numeric range not read as a unit).
- TestProbeBaseline tightened wrong 7->5; new test_pseudo_accumulation_does_not_misfire
is the failing-under-violation obligation (fails loudly if the pass regresses).
- TestSlashFractionLeakHazard pins the deferred `1/4`->`4` denominator leak: not
fixed here because suppressing the leaked operand *removes* a quantity and can
unblock the completeness clause (not unambiguously refuse-preferring), so it
needs its own train_sample + probe validation.
This commit is contained in:
parent
62d9db7a7c
commit
475d44d245
3 changed files with 121 additions and 6 deletions
|
|
@ -30,6 +30,13 @@ this module, so none of this can move the serving ``3/47/0``):
|
|||
units are blanked (empty, like a sentence-final number): the value stays
|
||||
grounded, the unit is honestly unknown. Closed lexeme set (``_NON_UNIT_WORDS``),
|
||||
not a grammar template (ADR-0165).
|
||||
* **EX-6 — hyphen-bonded number-units** (ADR-0163-F2). A number bonded to its unit
|
||||
by a hyphen (``25-foot``, ``20-inch``, ``2.5-mile``) was invisible to the base
|
||||
``number + space + word`` pattern, so the completeness check never saw the
|
||||
divisor — the blind spot behind the pseudo-accumulation confusers (0005/0007).
|
||||
Tight lexeme (digit run, hyphen, alphabetic unit word); the alphabetic-only unit
|
||||
group keeps numeric ranges (``3-5``) out and only the first hyphen segment is
|
||||
taken, so it stays clear of the deferred EX-3 multi-word-unit traps below.
|
||||
|
||||
EX-3 (multi-word units) is deliberately **not** integrated. Two distinct traps
|
||||
defeat the tightest lookahead-anchored rule the brief admits:
|
||||
|
|
@ -94,6 +101,19 @@ _FINAL_NUMBER_RE: Final[re.Pattern[str]] = re.compile(
|
|||
r"(?<![\w.])(\d+(?:\.\d+)?)(?=\s*(?:[.?!]|$))"
|
||||
)
|
||||
|
||||
# EX-6: a number bonded to its unit by a hyphen (``25-foot``, ``20-inch``,
|
||||
# ``2.5-mile``). The base ``_QTY_RE`` requires whitespace before the unit word,
|
||||
# so a hyphen-bonded unit was invisible — the blind spot behind the
|
||||
# pseudo-accumulation confusers (``25-foot`` / ``20-inch`` divisors the
|
||||
# completeness check never saw). Tight lexeme: a digit run, a single hyphen, an
|
||||
# alphabetic unit word. The trailing ``[a-zA-Z]+`` (not ``\d``) keeps numeric
|
||||
# ranges (``3-5``) out, and taking only the first hyphen segment keeps the
|
||||
# postmodifier tail (``25-year-old`` -> unit ``year``) from inflating the unit —
|
||||
# so this stays clear of the deferred EX-3 multi-word-unit traps.
|
||||
_HYPHEN_QTY_RE: Final[re.Pattern[str]] = re.compile(
|
||||
r"(?<![\w.])(\d+(?:\.\d+)?)-([a-zA-Z]+)"
|
||||
)
|
||||
|
||||
|
||||
# Function words that are never units. When the token immediately after a number
|
||||
# is one of these (``$0.75 each``, ``$40 to go``, ``3/4 of``), the single-word unit
|
||||
|
|
@ -162,9 +182,10 @@ def extract_quantities(problem_text: str) -> tuple[Quantity, ...]:
|
|||
passes never double-count a number:
|
||||
|
||||
1. EX-4 same-unit list (claims every number in the list);
|
||||
2. digit + single unit word (skips numbers a list already claimed);
|
||||
3. EX-1 word-number + unit word (alphabetic, disjoint from digit spans);
|
||||
4. EX-5 sentence-final bare number (skips any already-claimed digit).
|
||||
2. EX-6 hyphen-bonded number-unit (``25-foot``; claims the digit span);
|
||||
3. digit + single unit word (skips numbers a list/hyphen pass already claimed);
|
||||
4. EX-1 word-number + unit word (alphabetic, disjoint from digit spans);
|
||||
5. EX-5 sentence-final bare number (skips any already-claimed digit).
|
||||
"""
|
||||
found: list[tuple[int, Quantity]] = []
|
||||
claimed: list[tuple[int, int]] = []
|
||||
|
|
@ -179,6 +200,16 @@ def extract_quantities(problem_text: str) -> tuple[Quantity, ...]:
|
|||
found.append((pos, quantity))
|
||||
claimed.append((pos, pos + len(num.group(0))))
|
||||
|
||||
# 1b. EX-6 — hyphen-bonded number-unit (``25-foot``). Claims the digit span so
|
||||
# the bare/final passes never re-surface the number with a blank unit.
|
||||
for match in _HYPHEN_QTY_RE.finditer(problem_text):
|
||||
if _claimed(match.start(1), claimed):
|
||||
continue
|
||||
quantity = _quantity(match.group(1), match.group(2))
|
||||
if quantity is not None:
|
||||
found.append((match.start(1), quantity))
|
||||
claimed.append(match.span(1))
|
||||
|
||||
# 2. digit + single unit word — the original base pattern.
|
||||
for match in _QTY_RE.finditer(problem_text):
|
||||
if _claimed(match.start(1), claimed):
|
||||
|
|
|
|||
|
|
@ -17,9 +17,14 @@ from evals.gsm8k_math.confusers.v1.runner import (
|
|||
summarize,
|
||||
)
|
||||
|
||||
# Honest measured baseline (2026-05-29). The probe revealed the sealed composers
|
||||
# wrongly answer these confuser categories; this pins them so they cannot grow.
|
||||
_BASELINE_WRONG = 7
|
||||
# Honest measured baseline. The probe revealed the sealed composers wrongly answer
|
||||
# these confuser categories; this pins them so they cannot grow.
|
||||
# 2026-05-29 (initial): wrong=7, pair_tells=4.
|
||||
# 2026-05-29 (EX-6 hyphen-bonded units, ADR-0163-F2): the two pseudo-accumulation
|
||||
# misfires (0005->796, 0007->996) now refuse — the 25-foot/20-inch divisor is
|
||||
# finally visible to the completeness/polarity gate. wrong 7->5. pair_tells
|
||||
# unchanged (the pseudo-accumulation cases carry no positive twin).
|
||||
_BASELINE_WRONG = 5
|
||||
_BASELINE_PAIR_TELLS = 4
|
||||
|
||||
|
||||
|
|
@ -68,5 +73,19 @@ class TestProbeBaseline:
|
|||
def test_pair_tells_do_not_regress(self) -> None:
|
||||
assert len(pair_inconsistencies()) <= _BASELINE_PAIR_TELLS
|
||||
|
||||
def test_pseudo_accumulation_does_not_misfire(self) -> None:
|
||||
# ADR-0163-F2 EX-6: the hyphen-bonded-unit extraction made the
|
||||
# 25-foot/20-inch divisor visible, so the pseudo-accumulation confusers
|
||||
# (0005, 0007) refuse instead of committing 796/996. This is the specific
|
||||
# obligation the fix proves — it fails loudly if the hyphen pass regresses
|
||||
# (the gate would again miss the divisor and the category would misfire).
|
||||
results = run_probe()
|
||||
pseudo = [r for r in results if r.category == "pseudo-accumulation"]
|
||||
assert pseudo, "pseudo-accumulation cases missing from corpus"
|
||||
assert all(r.verdict != "wrong" for r in pseudo), (
|
||||
"pseudo-accumulation confuser misfired — the hyphen-bonded-unit "
|
||||
"divisor is no longer reaching the completeness/polarity gate."
|
||||
)
|
||||
|
||||
def test_deterministic(self) -> None:
|
||||
assert summarize(run_probe()) == summarize(run_probe())
|
||||
|
|
|
|||
|
|
@ -94,6 +94,71 @@ class TestNoRegression:
|
|||
assert _triples("It costs 0.75 dollars.") == [(0.75, "dollars", "0.75")]
|
||||
|
||||
|
||||
class TestEX6HyphenatedUnitNumbers:
|
||||
"""ADR-0163-F2 — hyphen-bonded number-unit tokens (``25-foot``, ``20-inch``).
|
||||
|
||||
The base ``_QTY_RE`` requires ``number + whitespace + unit word``, so a number
|
||||
bonded to its unit by a hyphen (``25-foot sections``, ``20-inch pieces``) was
|
||||
invisible — the very blind spot behind the pseudo-accumulation confusers
|
||||
(0005 ``→796``, 0007 ``→996``): the completeness clause could not see the
|
||||
``25``/``20`` divisor, so the accumulation reading was wrongly "complete".
|
||||
|
||||
This is a tight, ADR-0165-safe lexeme pattern (digit run, a hyphen, an
|
||||
alphabetic unit word) — strictly distinct from the deferred EX-3 multi-word
|
||||
*space-separated* unit problem. Over-extraction here only costs refusals
|
||||
(the gate is refuse-preferring), never a wrong answer.
|
||||
"""
|
||||
|
||||
def test_hyphen_bonded_unit_extracts_value_and_unit(self) -> None:
|
||||
assert _triples("She cuts it into 20-inch pieces.") == [(20.0, "inch", "20")]
|
||||
|
||||
def test_hyphen_bonded_unit_mid_sentence(self) -> None:
|
||||
assert _triples("She splits it into 25-foot sections.") == [(25.0, "foot", "25")]
|
||||
|
||||
def test_decimal_hyphen_bonded_unit(self) -> None:
|
||||
assert _triples("He ran a 2.5-mile loop.") == [(2.5, "mile", "2.5")]
|
||||
|
||||
def test_hyphen_unit_not_double_counted_as_final_number(self) -> None:
|
||||
# the hyphen pass claims the digit span; the bare-final pass must not
|
||||
# also surface "25" with an empty unit.
|
||||
assert len(extract_quantities("It was 25-foot.")) == 1
|
||||
|
||||
def test_word_compound_still_takes_word_number_path(self) -> None:
|
||||
# digit-hyphen is a different lexeme from the EX-1 word-word compound;
|
||||
# "twenty-four" must still resolve via the word-number path, unaffected.
|
||||
assert _triples("The team scored twenty-four points.") == [
|
||||
(24.0, "points", "twenty-four"),
|
||||
]
|
||||
|
||||
def test_numeric_range_is_not_read_as_unit(self) -> None:
|
||||
# "3-5" is digit-hyphen-DIGIT; the unit group requires letters, so the
|
||||
# hyphen pass must not fire and invent a unit from the second number.
|
||||
assert all(q.unit != "5" for q in extract_quantities("Pick 3-5 apples."))
|
||||
|
||||
|
||||
class TestSlashFractionLeakHazard:
|
||||
"""Honest pin (deferred hazard): a slash-fraction leaks its denominator.
|
||||
|
||||
``"gives 1/4 to a friend"`` extracts the bare ``4`` (the base ``_QTY_RE``
|
||||
sees ``4`` + space + the function-word ``to``, blanks the unit, and emits a
|
||||
standalone quantity ``4``). This is a grounded-but-wrong operand — it is the
|
||||
second half of the pseudo-accumulation misfire (``1000 - 4 = 996``).
|
||||
|
||||
It is **not** fixed in this PR on purpose. Suppressing the leaked ``4`` *removes*
|
||||
a quantity, which can *unblock* the completeness clause (a derivation that was
|
||||
"incomplete" because it ignored the spurious ``4`` could become "complete" and
|
||||
commit) — i.e. the fix is not unambiguously refuse-preferring and needs its own
|
||||
train_sample + probe validation. The hyphen-unit pass (TestEX6) already drives
|
||||
confusers 0005/0007 to *refuse* via the polarity-None ``cuts``/``splits`` clause,
|
||||
so this leak is currently dormant behind that refusal. This test pins the leak
|
||||
so a future fraction-operand PR addresses it deliberately, not by accident.
|
||||
"""
|
||||
|
||||
def test_slash_fraction_denominator_currently_leaks(self) -> None:
|
||||
# documents current (leaky) behavior — flip this when fractions are modeled.
|
||||
assert (4.0, "", "4") in _triples("She gives 1/4 to a friend.")
|
||||
|
||||
|
||||
class TestEX3StillDeferred:
|
||||
"""Honest pin: EX-3 (multi-word units) remains deferred after the Track C redo.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue