feat(ADR-0170/W2): DCS-S1 acquisition verbs — first CandidateOperation emission (#377)
Second implementation PR of the ADR-0170 wave. Extends the DCS injector to emit ``CandidateOperation(kind='add')`` for acquisition verbs alongside the existing ``CandidateInitial`` emission for possession verbs. Proves the W1 type-widening with real emission of both union members. ## What changes ### `generate/recognizer_match.py` - New `_ACQUISITION_VERBS` frozenset (12 verbs: collect/get/receive/buy inflections). Each member is a subset of `ADD_VERBS` so the downstream CandidateOperation post-init whitelist accepts the matched_verb token. - Extractor now accepts either possession OR acquisition verbs and records `anchor_kind` (`"possession"` | `"acquisition"`) plus `verb_token` in the parsed anchor schema. ### `generate/recognizer_anchor_inject.py` - `inject_discrete_count_statement` dispatches on `anchor_kind`: - `"possession"` → `CandidateInitial` (existing behavior unchanged) - `"acquisition"` → `CandidateOperation(add)` (new) - New helper `_build_operation_from_discrete_count_acquisition` constructs the operation. Operand uses `_resolve_count_value`; matched_verb uses `_locate_token` for round-trip ground check. - Return type uses `InjectorEmission` from W1. ### Tests - `tests/test_adr_0170_w2_dcs_acquisition_verbs.py` (new) — 22 tests: - Verb-set membership pins - Acquisition ⊂ ADD_VERBS sanity check - Possession + Acquisition disjoint - Extractor records anchor_kind correctly - Injector emits CandidateOperation for acquisition verbs - Possession path still emits CandidateInitial unchanged - Deliberate exclusions (gained / donated / saved) still refuse - Case 0050 hazard pinned (does/contemplates not in either set) - Determinism + roundtrip_admissible passes - Updated `tests/test_adr_0163_d2_discrete_count_injection.py` to reflect new anchor schema fields (anchor_kind, verb_token). - Updated `tests/test_adr_0170_w1_injector_type_widening.py` — the DCS injector now legitimately returns `tuple[InjectorEmission, ...]` (not narrower). ## Deliberate exclusions These verbs are NOT in `_ACQUISITION_VERBS` and the extractor refuses them — preserving wrong=0: - `gained / gains / gain` — delta-of-attribute (weight, age), not acquisition. Admitting as add-operation would risk wrong>0 on questions that ask total state. - `donated / donates / donate` — SUBTRACT semantics (actor gives away). - `saved / saves / save` — ambiguous (time vs money vs effort). Widening this set is operator-reviewable per `feedback-wrong-zero- hazard-case-0050` discipline. ## ADR-0131.G.1 branch-disagreement discipline preserved The regex parser already emits `CandidateOperation(add)` for acquisition verbs via `ADD_VERBS` for single-word units. The new DCS injector path emits the same kind of operation for multi-word units (where the regex parser fails). Collapsed-tie when both paths emit identical operations on overlapping shapes; no disagreement. ## Test plan - tests/test_adr_0170_w2_dcs_acquisition_verbs.py: 22 passed (new) - tests/test_adr_0163_d2_discrete_count_injection.py: ~30 passed (existing tests updated for new schema fields) - tests/test_adr_0170_w1_injector_type_widening.py: 6 passed - tests/test_recognizer_skip_wrong_zero.py + brief_11b + brief_11 + candidate_graph_wiring + candidate_domain_partition: passed - evals/gsm8k_math/train_sample/v1: counts=correct=3 refused=47 wrong=0 unchanged (case 0023 still has S2/S3 downstream blockers; W2's value is infrastructure, not direct lift) ## Hard invariants - `wrong == 0` preserved (case 0050 hazard pin + deliberate verb exclusions + roundtrip_admissible gate) - ADR-0166: no new eval lanes - No teaching-store / pack mutation - ADR-0131.G.1 branch-disagreement discipline preserved (acquisition → operation, not initial) - Five-layer wrong=0 safety net (ADR-0163.D.2) intact and extended ## W3 NOT in this PR — honest skip Initial plan was to bundle W2 + W3 (A1 currency_amount injector). Inspection of the 4 actual `currency_amount` GSM8K refusals showed none match A1's narrow form (`<ProperNoun> earns|charges $<amount>`): | Case | Statement | Reason narrow form doesn't fit | |---|---|---| | 0019 | "this requires 3 vet appointments, which cost $400 each" | anaphoric subject + multi-quantity | | 0026 | "Aaron and his brother Carson each saved up $40" | multi-subject + "each" | | 0028 | "It cost $100,000 to open initially" | pronoun subject | | 0043 | "Her mother gave her an additional $4, and her father twice as much" | multi-clause + comparative + transfer | Shipping W3 as-designed would have re-introduced the dead-code pattern #373 just cleaned up. Skipped honestly; ADR-0172 Tier 1's decomposer (the next wave) will surface category-shape mismatches like this programmatically.
This commit is contained in:
parent
eeeec80127
commit
b190f3b6c5
5 changed files with 502 additions and 34 deletions
|
|
@ -46,7 +46,12 @@ from typing import Mapping, Union
|
||||||
|
|
||||||
from evals.refusal_taxonomy.shape_categories import ShapeCategory
|
from evals.refusal_taxonomy.shape_categories import ShapeCategory
|
||||||
from generate.math_candidate_parser import CandidateInitial, CandidateOperation
|
from generate.math_candidate_parser import CandidateInitial, CandidateOperation
|
||||||
from generate.math_problem_graph import InitialPossession, MathGraphError, Quantity
|
from generate.math_problem_graph import (
|
||||||
|
InitialPossession,
|
||||||
|
MathGraphError,
|
||||||
|
Operation,
|
||||||
|
Quantity,
|
||||||
|
)
|
||||||
from generate.recognizer_match import RecognizerMatch
|
from generate.recognizer_match import RecognizerMatch
|
||||||
|
|
||||||
# ADR-0170 — the widened injector emission type. Per-category injectors
|
# ADR-0170 — the widened injector emission type. Per-category injectors
|
||||||
|
|
@ -92,26 +97,49 @@ def inject_from_match(
|
||||||
def inject_discrete_count_statement(
|
def inject_discrete_count_statement(
|
||||||
match: RecognizerMatch,
|
match: RecognizerMatch,
|
||||||
sentence: str,
|
sentence: str,
|
||||||
) -> tuple[CandidateInitial, ...]:
|
) -> tuple[InjectorEmission, ...]:
|
||||||
"""Build CandidateInitial(s) from ``discrete_count`` parsed anchors.
|
"""Build CandidateInitial OR CandidateOperation from ``discrete_count``
|
||||||
|
parsed anchors, dispatched on the matcher's ``anchor_kind``.
|
||||||
|
|
||||||
v1 narrowness: the matcher emits at most one anchor (further anchors
|
Per ADR-0170 W2 — the matcher records ``anchor_kind`` as either
|
||||||
refuse extraction). When the anchor is absent (detection-only
|
``"possession"`` (verbs ``has/have/had``) or ``"acquisition"``
|
||||||
fallback), the injector returns ``()`` and the candidate-graph
|
(verbs in ``_ACQUISITION_VERBS``).
|
||||||
continues with the round-2 skip-only behavior.
|
|
||||||
|
- ``possession`` → ``CandidateInitial`` (existing behavior; the
|
||||||
|
sentence asserts an initial state)
|
||||||
|
- ``acquisition`` → ``CandidateOperation(kind='add')`` (new in W2;
|
||||||
|
the sentence asserts an add-operation, preserving
|
||||||
|
ADR-0131.G.1's branch-disagreement discipline — the regex
|
||||||
|
parser's ADD_VERBS path emits the same kind of operation for
|
||||||
|
single-word units, so the injector path complements it on
|
||||||
|
multi-word units without conflicting)
|
||||||
|
|
||||||
|
v1 narrowness: at most one anchor per match; absent or
|
||||||
|
unconstructable anchors return ``()``.
|
||||||
"""
|
"""
|
||||||
if not match.parsed_anchors:
|
if not match.parsed_anchors:
|
||||||
return ()
|
return ()
|
||||||
|
|
||||||
out: list[CandidateInitial] = []
|
out: list[InjectorEmission] = []
|
||||||
for anchor in match.parsed_anchors:
|
for anchor in match.parsed_anchors:
|
||||||
cand = _build_initial_from_discrete_count(anchor, sentence)
|
anchor_kind = anchor.get("anchor_kind", "possession")
|
||||||
|
if anchor_kind == "possession":
|
||||||
|
cand: InjectorEmission | None = _build_initial_from_discrete_count(
|
||||||
|
anchor, sentence
|
||||||
|
)
|
||||||
|
elif anchor_kind == "acquisition":
|
||||||
|
cand = _build_operation_from_discrete_count_acquisition(
|
||||||
|
anchor, sentence
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
# Unknown anchor_kind — under-admit. Future widenings (e.g.
|
||||||
|
# "depletion" verbs as CandidateOperation(subtract)) extend
|
||||||
|
# this branch.
|
||||||
|
return ()
|
||||||
if cand is None:
|
if cand is None:
|
||||||
# Under-admit on any failure to construct. The other
|
# Under-admit on any failure to construct. Partial
|
||||||
# already-built candidates for this sentence remain
|
# admission would mean the downstream Cartesian product
|
||||||
# admissible only if they all pass; partial admission would
|
# enumerates a graph missing state.
|
||||||
# mean the downstream Cartesian product enumerates a graph
|
|
||||||
# missing state — under-admit instead.
|
|
||||||
return ()
|
return ()
|
||||||
out.append(cand)
|
out.append(cand)
|
||||||
return tuple(out)
|
return tuple(out)
|
||||||
|
|
@ -194,6 +222,104 @@ def _build_initial_from_discrete_count(
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _build_operation_from_discrete_count_acquisition(
|
||||||
|
anchor: Mapping[str, object],
|
||||||
|
sentence: str,
|
||||||
|
) -> CandidateOperation | None:
|
||||||
|
"""Construct one CandidateOperation(kind='add') from a discrete_count
|
||||||
|
anchor whose ``anchor_kind == "acquisition"``.
|
||||||
|
|
||||||
|
Per ADR-0170 W2 — acquisition verbs (``collected``, ``received``,
|
||||||
|
``bought``, ``got``) are routed to operations, not initials, in
|
||||||
|
accordance with ADR-0131.G.1's branch-disagreement discipline. The
|
||||||
|
solver's defaults-from-zero rule resolves single-statement
|
||||||
|
acquisitions correctly (``0 + N = N``).
|
||||||
|
|
||||||
|
Refuses (returns ``None``) when any field cannot be coerced, when
|
||||||
|
the literal verb token cannot be located in the surface, or when
|
||||||
|
the constructed ``CandidateOperation`` would violate its post-init
|
||||||
|
invariants. The result is admissibility-checked upstream by
|
||||||
|
``roundtrip_admissible``.
|
||||||
|
|
||||||
|
Anchor schema (same as possession, with ``anchor_kind`` discriminator):
|
||||||
|
{
|
||||||
|
"kind": "discrete_count",
|
||||||
|
"anchor_kind": "acquisition",
|
||||||
|
"subject_role": <str>,
|
||||||
|
"count_token": <str>,
|
||||||
|
"count_kind": <"integer"|"word">,
|
||||||
|
"counted_noun": <str>,
|
||||||
|
"verb_token": <str>, # e.g. "collected"
|
||||||
|
}
|
||||||
|
"""
|
||||||
|
subject_role = anchor.get("subject_role")
|
||||||
|
count_token = anchor.get("count_token")
|
||||||
|
count_kind = anchor.get("count_kind")
|
||||||
|
counted_noun = anchor.get("counted_noun")
|
||||||
|
verb_token = anchor.get("verb_token")
|
||||||
|
|
||||||
|
if (
|
||||||
|
not isinstance(subject_role, str) or not subject_role
|
||||||
|
or not isinstance(count_token, str) or not count_token
|
||||||
|
or not isinstance(count_kind, str)
|
||||||
|
or not isinstance(counted_noun, str) or not counted_noun
|
||||||
|
or not isinstance(verb_token, str) or not verb_token
|
||||||
|
):
|
||||||
|
return None
|
||||||
|
|
||||||
|
value = _resolve_count_value(count_token, count_kind)
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
# Locate the literal verb surface in the sentence so the
|
||||||
|
# round-trip ground check in ``roundtrip_admissible`` succeeds.
|
||||||
|
# The matcher already confirmed ``verb_token`` is in
|
||||||
|
# ``_ACQUISITION_VERBS`` (which is itself a subset of
|
||||||
|
# ``ADD_VERBS``), so the downstream CandidateOperation post-init
|
||||||
|
# whitelist accepts the matched_verb token.
|
||||||
|
located_verb = _locate_token(sentence, verb_token)
|
||||||
|
if located_verb is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
operand = Quantity(value=value, unit=counted_noun)
|
||||||
|
op = Operation(
|
||||||
|
actor=subject_role,
|
||||||
|
kind="add",
|
||||||
|
operand=operand,
|
||||||
|
)
|
||||||
|
except MathGraphError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
return CandidateOperation(
|
||||||
|
op=op,
|
||||||
|
source_span=sentence,
|
||||||
|
matched_verb=located_verb,
|
||||||
|
matched_value_token=count_token,
|
||||||
|
matched_unit_token=counted_noun,
|
||||||
|
matched_actor_token=subject_role,
|
||||||
|
)
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _locate_token(sentence: str, target_lc: str) -> str | None:
|
||||||
|
"""Return the literal-surface form of ``target_lc`` (lowercased) in
|
||||||
|
``sentence`` whitespace-tokenized, or ``None`` if absent.
|
||||||
|
|
||||||
|
Used by the acquisition-verb path to extract the matched verb
|
||||||
|
surface for ``CandidateOperation.matched_verb``. Falls back to
|
||||||
|
``None`` only when the matcher's recorded ``verb_token`` somehow
|
||||||
|
diverges from the sentence surface — the under-admit path.
|
||||||
|
"""
|
||||||
|
for raw in sentence.split():
|
||||||
|
tok = raw.strip(".,;:!?\"'()[]{}").lower()
|
||||||
|
if tok == target_lc:
|
||||||
|
return tok
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def _resolve_count_value(count_token: str, count_kind: str) -> int | None:
|
def _resolve_count_value(count_token: str, count_kind: str) -> int | None:
|
||||||
"""Map ``count_token`` to a numeric value.
|
"""Map ``count_token`` to a numeric value.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -457,6 +457,32 @@ _POSSESSION_VERBS: Final[frozenset[str]] = frozenset({
|
||||||
"has", "have", "had",
|
"has", "have", "had",
|
||||||
})
|
})
|
||||||
|
|
||||||
|
# ADR-0170 W2 — acquisition verbs: surface verbs that grammatically place
|
||||||
|
# the actor as the *gainer* of the operand quantity, NOT as having the
|
||||||
|
# operand as an initial state. Per ADR-0131.G.1 these verbs route to
|
||||||
|
# CandidateOperation(add), not CandidateInitial — emitting them as
|
||||||
|
# initials would create branch disagreement with the regex parser's
|
||||||
|
# ADD_VERBS path.
|
||||||
|
#
|
||||||
|
# Each member is also a member of generate.math_roundtrip.ADD_VERBS so
|
||||||
|
# the downstream CandidateOperation post-init whitelist accepts the
|
||||||
|
# matched_verb token.
|
||||||
|
#
|
||||||
|
# DELIBERATELY EXCLUDED:
|
||||||
|
# - "gained / gains / gain": delta-of-attribute (weight, age) — admitting
|
||||||
|
# as add-operation risks wrong>0 on questions that ask total state
|
||||||
|
# - "donated / donates / donate": SUBTRACT verb (actor gives away)
|
||||||
|
# - "saved / saves / save": ambiguous (saved time vs saved up money)
|
||||||
|
#
|
||||||
|
# Widening this set is operator-reviewable per the wrong=0 hazard
|
||||||
|
# documented in feedback-wrong-zero-hazard-case-0050.
|
||||||
|
_ACQUISITION_VERBS: Final[frozenset[str]] = frozenset({
|
||||||
|
"collected", "collects", "collect",
|
||||||
|
"received", "receives", "receive",
|
||||||
|
"bought", "buys", "buy",
|
||||||
|
"got", "gets", "get",
|
||||||
|
})
|
||||||
|
|
||||||
# Pronoun subjects refused at extraction (ambiguous referent). The
|
# Pronoun subjects refused at extraction (ambiguous referent). The
|
||||||
# extractor requires a concrete proper-noun subject the source span can
|
# extractor requires a concrete proper-noun subject the source span can
|
||||||
# ground.
|
# ground.
|
||||||
|
|
@ -566,7 +592,11 @@ def _try_extract_discrete_count_anchor(
|
||||||
return None
|
return None
|
||||||
|
|
||||||
verb = m.group("verb").lower()
|
verb = m.group("verb").lower()
|
||||||
if verb not in _POSSESSION_VERBS:
|
if verb in _POSSESSION_VERBS:
|
||||||
|
anchor_kind = "possession"
|
||||||
|
elif verb in _ACQUISITION_VERBS:
|
||||||
|
anchor_kind = "acquisition"
|
||||||
|
else:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
count_token = m.group("count")
|
count_token = m.group("count")
|
||||||
|
|
@ -608,6 +638,11 @@ def _try_extract_discrete_count_anchor(
|
||||||
"count_token": count_token,
|
"count_token": count_token,
|
||||||
"count_kind": count_kind,
|
"count_kind": count_kind,
|
||||||
"counted_noun": canon,
|
"counted_noun": canon,
|
||||||
|
# ADR-0170 W2 — anchor_kind discriminates the downstream
|
||||||
|
# injector path: "possession" → CandidateInitial (existing);
|
||||||
|
# "acquisition" → CandidateOperation(add) (new).
|
||||||
|
"anchor_kind": anchor_kind,
|
||||||
|
"verb_token": verb,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -73,12 +73,16 @@ def _ratified_registry():
|
||||||
class TestExtractionCorrectness:
|
class TestExtractionCorrectness:
|
||||||
def test_basic_integer_count(self) -> None:
|
def test_basic_integer_count(self) -> None:
|
||||||
a = _try_extract("Sam has 5 apples.")
|
a = _try_extract("Sam has 5 apples.")
|
||||||
|
# Post-W2 (ADR-0170): anchor carries anchor_kind discriminator
|
||||||
|
# and verb_token for the injector's dispatch + admissibility.
|
||||||
assert a == {
|
assert a == {
|
||||||
"kind": "discrete_count",
|
"kind": "discrete_count",
|
||||||
"subject_role": "Sam",
|
"subject_role": "Sam",
|
||||||
"count_token": "5",
|
"count_token": "5",
|
||||||
"count_kind": "integer",
|
"count_kind": "integer",
|
||||||
"counted_noun": "apples",
|
"counted_noun": "apples",
|
||||||
|
"anchor_kind": "possession",
|
||||||
|
"verb_token": "has",
|
||||||
}
|
}
|
||||||
|
|
||||||
def test_past_tense_had(self) -> None:
|
def test_past_tense_had(self) -> None:
|
||||||
|
|
@ -162,11 +166,40 @@ class TestExtractionRefusal:
|
||||||
# 'widgets' is not in the spec's observed_counted_nouns.
|
# 'widgets' is not in the spec's observed_counted_nouns.
|
||||||
assert _try_extract("Sam has 5 widgets.") is None
|
assert _try_extract("Sam has 5 widgets.") is None
|
||||||
|
|
||||||
def test_non_possession_verb_refused(self) -> None:
|
def test_non_possession_non_acquisition_verb_refused(self) -> None:
|
||||||
# 'wants', 'collected', 'bought' — operation verbs, not state.
|
# Post-W2 (ADR-0170): possession verbs (has/have/had) AND
|
||||||
|
# acquisition verbs (collected/received/bought/got) extract
|
||||||
|
# successfully — the latter dispatched to CandidateOperation(add)
|
||||||
|
# in the injector. Verbs outside both sets still refuse.
|
||||||
assert _try_extract("Michael wants 10 pounds.") is None
|
assert _try_extract("Michael wants 10 pounds.") is None
|
||||||
assert _try_extract("Nicole collected 400 paperclips.") is None
|
# 'gained' is deliberately EXCLUDED from _ACQUISITION_VERBS
|
||||||
assert _try_extract("Sam bought 5 apples.") is None
|
# (delta-of-attribute hazard); must still refuse.
|
||||||
|
assert _try_extract("Orlando gained 5 pounds.") is None
|
||||||
|
# 'donated' is a SUBTRACT verb (actor gives away); deferred
|
||||||
|
# until a separate W2.1 PR adds depletion-verb handling.
|
||||||
|
assert _try_extract("Alice donated 3 books.") is None
|
||||||
|
|
||||||
|
def test_acquisition_verbs_extract_with_anchor_kind(self) -> None:
|
||||||
|
# Post-W2 (ADR-0170): acquisition verbs extract with
|
||||||
|
# anchor_kind='acquisition'. The injector then emits
|
||||||
|
# CandidateOperation(add) rather than CandidateInitial.
|
||||||
|
result = _try_extract("Nicole collected 400 paperclips.")
|
||||||
|
assert result is not None
|
||||||
|
assert result["anchor_kind"] == "acquisition"
|
||||||
|
assert result["verb_token"] == "collected"
|
||||||
|
|
||||||
|
result_buy = _try_extract("Sam bought 5 apples.")
|
||||||
|
assert result_buy is not None
|
||||||
|
assert result_buy["anchor_kind"] == "acquisition"
|
||||||
|
assert result_buy["verb_token"] == "bought"
|
||||||
|
|
||||||
|
def test_possession_verbs_extract_with_possession_kind(self) -> None:
|
||||||
|
# Pre-W2 behavior preserved: possession verbs extract with
|
||||||
|
# anchor_kind='possession'; injector emits CandidateInitial.
|
||||||
|
result = _try_extract("Sam has 5 apples.")
|
||||||
|
assert result is not None
|
||||||
|
assert result["anchor_kind"] == "possession"
|
||||||
|
assert result["verb_token"] == "has"
|
||||||
|
|
||||||
def test_owns_outside_v1_whitelist(self) -> None:
|
def test_owns_outside_v1_whitelist(self) -> None:
|
||||||
# v1 restricts to has/have/had to align with CandidateInitial's
|
# v1 restricts to has/have/had to align with CandidateInitial's
|
||||||
|
|
|
||||||
|
|
@ -72,24 +72,19 @@ def test_inject_from_match_return_type_is_widened():
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
def test_discrete_count_injector_still_emits_only_candidate_initial():
|
def test_discrete_count_injector_return_type_post_w2():
|
||||||
"""W1 is type-level only. The existing
|
"""W1 was type-level only; W2 (ADR-0170 implementation) extends the
|
||||||
``inject_discrete_count_statement`` returns ``CandidateInitial`` —
|
DCS injector to emit ``CandidateOperation(add)`` for acquisition
|
||||||
not ``CandidateOperation`` — at runtime. This is the byte-identical
|
verbs alongside ``CandidateInitial`` for possession verbs.
|
||||||
behavior guarantee for the W1 PR.
|
|
||||||
|
|
||||||
Mechanically: pre-W1 the function returned
|
The function's return type is now the widened
|
||||||
``tuple[CandidateInitial, ...]``. Post-W1 it still does. Subsequent
|
``tuple[InjectorEmission, ...]``. The pin verifies the contract is
|
||||||
PRs (W2 DCS-S1 acquisition, W3 currency, W4 multiplicative) widen
|
consistent with the dispatcher's widened type — runtime emission
|
||||||
the per-injector emission shapes; W1 ships only the dispatcher
|
is verified separately by the W2 acquisition-verb tests."""
|
||||||
contract."""
|
|
||||||
import inspect
|
import inspect
|
||||||
sig = inspect.signature(inject_discrete_count_statement)
|
sig = inspect.signature(inject_discrete_count_statement)
|
||||||
# With ``from __future__ import annotations`` the return annotation
|
# Post-W2 the DCS injector itself emits the widened union type.
|
||||||
# is stored as a string. The W1 pin is that the existing DCS
|
assert sig.return_annotation == "tuple[InjectorEmission, ...]"
|
||||||
# injector's *narrower* return type is unchanged — only the
|
|
||||||
# dispatcher (``inject_from_match``) widens.
|
|
||||||
assert sig.return_annotation == "tuple[CandidateInitial, ...]"
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
|
||||||
279
tests/test_adr_0170_w2_dcs_acquisition_verbs.py
Normal file
279
tests/test_adr_0170_w2_dcs_acquisition_verbs.py
Normal file
|
|
@ -0,0 +1,279 @@
|
||||||
|
"""ADR-0170 W2 — DCS-S1 acquisition verbs: first ``CandidateOperation``
|
||||||
|
emission from the recognizer-injector path.
|
||||||
|
|
||||||
|
W2 proves the W1 contract widening with concrete real-emission code:
|
||||||
|
the DCS injector now dispatches on the matcher's recorded
|
||||||
|
``anchor_kind`` and emits ``CandidateOperation(add)`` for acquisition
|
||||||
|
verbs (collected / received / bought / got) instead of
|
||||||
|
``CandidateInitial``.
|
||||||
|
|
||||||
|
This preserves ADR-0131.G.1's branch-disagreement discipline:
|
||||||
|
acquisition verbs route to operations, not initials, so the regex
|
||||||
|
parser's ADD_VERBS path and the injector's CandidateOperation path
|
||||||
|
emit compatible kinds for the same sentence (collapsed-tie OK).
|
||||||
|
|
||||||
|
Hard invariants:
|
||||||
|
|
||||||
|
- ``wrong == 0`` — verify against case 0050 hazard
|
||||||
|
- The acquisition path emits only ``CandidateOperation(add)``,
|
||||||
|
matching ADR-0131.G.1
|
||||||
|
- Verbs deliberately excluded (gained, donated, saved) still refuse
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from evals.refusal_taxonomy.shape_categories import ShapeCategory
|
||||||
|
from generate.math_candidate_parser import CandidateInitial, CandidateOperation
|
||||||
|
from generate.math_problem_graph import Operation, Quantity
|
||||||
|
from generate.recognizer_anchor_inject import (
|
||||||
|
_build_operation_from_discrete_count_acquisition,
|
||||||
|
inject_discrete_count_statement,
|
||||||
|
inject_from_match,
|
||||||
|
)
|
||||||
|
from generate.recognizer_match import (
|
||||||
|
_ACQUISITION_VERBS,
|
||||||
|
_POSSESSION_VERBS,
|
||||||
|
_try_extract_discrete_count_anchor,
|
||||||
|
_padded_lower,
|
||||||
|
)
|
||||||
|
from generate.math_candidate_graph import _load_ratified_registry_or_empty
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Helpers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def _dcs_spec():
|
||||||
|
reg = _load_ratified_registry_or_empty()
|
||||||
|
for r in reg:
|
||||||
|
if r.shape_category.value == "discrete_count_statement":
|
||||||
|
return r.canonical_pattern
|
||||||
|
raise RuntimeError("no ratified discrete_count_statement spec on main")
|
||||||
|
|
||||||
|
|
||||||
|
def _extract(stmt: str):
|
||||||
|
return _try_extract_discrete_count_anchor(stmt, _padded_lower(stmt), _dcs_spec())
|
||||||
|
|
||||||
|
|
||||||
|
def _make_match(parsed_anchors):
|
||||||
|
from generate.recognizer_registry import RatifiedRecognizer
|
||||||
|
from generate.recognizer_match import RecognizerMatch
|
||||||
|
|
||||||
|
rec = RatifiedRecognizer(
|
||||||
|
proposal_id="test-w2",
|
||||||
|
shape_category=ShapeCategory.DISCRETE_COUNT_STATEMENT,
|
||||||
|
canonical_pattern=dict(_dcs_spec()),
|
||||||
|
spec_digest="test-digest",
|
||||||
|
review_date="2026-05-27",
|
||||||
|
ratified_at_revision="test",
|
||||||
|
)
|
||||||
|
return RecognizerMatch(
|
||||||
|
recognizer=rec,
|
||||||
|
category=ShapeCategory.DISCRETE_COUNT_STATEMENT,
|
||||||
|
outcome="admissible",
|
||||||
|
graph_intent="count",
|
||||||
|
parsed_anchors=parsed_anchors,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Verb-set membership pins
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_acquisition_verbs_set_contains_expected_verbs():
|
||||||
|
expected = {"collected", "collects", "collect",
|
||||||
|
"received", "receives", "receive",
|
||||||
|
"bought", "buys", "buy",
|
||||||
|
"got", "gets", "get"}
|
||||||
|
assert _ACQUISITION_VERBS == frozenset(expected)
|
||||||
|
|
||||||
|
|
||||||
|
def test_possession_verbs_set_unchanged():
|
||||||
|
# Pre-W2 set preserved.
|
||||||
|
assert _POSSESSION_VERBS == frozenset({"has", "have", "had"})
|
||||||
|
|
||||||
|
|
||||||
|
def test_acquisition_and_possession_sets_disjoint():
|
||||||
|
assert _ACQUISITION_VERBS.isdisjoint(_POSSESSION_VERBS)
|
||||||
|
|
||||||
|
|
||||||
|
def test_acquisition_verbs_subset_of_add_verbs():
|
||||||
|
# Every acquisition verb must be in ADD_VERBS so the downstream
|
||||||
|
# CandidateOperation post-init whitelist accepts the matched_verb
|
||||||
|
# token.
|
||||||
|
from generate.math_roundtrip import ADD_VERBS
|
||||||
|
assert _ACQUISITION_VERBS.issubset(ADD_VERBS)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Extractor — anchor_kind discrimination
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"verb,canonical",
|
||||||
|
[
|
||||||
|
("collected", "collected"),
|
||||||
|
("received", "received"),
|
||||||
|
("bought", "bought"),
|
||||||
|
("got", "got"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_extractor_records_acquisition_anchor_kind(verb: str, canonical: str):
|
||||||
|
anchor = _extract(f"Nicole {verb} 400 paperclips.")
|
||||||
|
assert anchor is not None
|
||||||
|
assert anchor["anchor_kind"] == "acquisition"
|
||||||
|
assert anchor["verb_token"] == canonical
|
||||||
|
|
||||||
|
|
||||||
|
def test_extractor_records_possession_anchor_kind():
|
||||||
|
anchor = _extract("Nicole has 400 paperclips.")
|
||||||
|
assert anchor is not None
|
||||||
|
assert anchor["anchor_kind"] == "possession"
|
||||||
|
assert anchor["verb_token"] == "has"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Injector — emits CandidateOperation for acquisition
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_acquisition_anchor_produces_candidate_operation_add():
|
||||||
|
anchor = _extract("Nicole collected 400 paperclips.")
|
||||||
|
assert anchor is not None
|
||||||
|
match = _make_match((anchor,))
|
||||||
|
out = inject_discrete_count_statement(match, "Nicole collected 400 paperclips.")
|
||||||
|
assert len(out) == 1
|
||||||
|
cand = out[0]
|
||||||
|
assert isinstance(cand, CandidateOperation), (
|
||||||
|
f"acquisition anchor must emit CandidateOperation, got {type(cand).__name__}"
|
||||||
|
)
|
||||||
|
assert cand.op.kind == "add"
|
||||||
|
assert cand.op.actor == "Nicole"
|
||||||
|
assert cand.op.operand.value == 400
|
||||||
|
assert cand.op.operand.unit == "paperclips"
|
||||||
|
assert cand.matched_verb == "collected"
|
||||||
|
|
||||||
|
|
||||||
|
def test_possession_anchor_still_produces_candidate_initial():
|
||||||
|
"""Pre-W2 behavior preserved: possession anchors still emit
|
||||||
|
CandidateInitial, not CandidateOperation."""
|
||||||
|
anchor = _extract("Nicole has 400 paperclips.")
|
||||||
|
assert anchor is not None
|
||||||
|
match = _make_match((anchor,))
|
||||||
|
out = inject_discrete_count_statement(match, "Nicole has 400 paperclips.")
|
||||||
|
assert len(out) == 1
|
||||||
|
cand = out[0]
|
||||||
|
assert isinstance(cand, CandidateInitial)
|
||||||
|
assert cand.matched_anchor == "has"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("verb", ["collected", "received", "bought", "got"])
|
||||||
|
def test_all_acquisition_verbs_emit_candidate_operation(verb: str):
|
||||||
|
anchor = _extract(f"Sam {verb} 5 apples.")
|
||||||
|
assert anchor is not None
|
||||||
|
match = _make_match((anchor,))
|
||||||
|
out = inject_discrete_count_statement(match, f"Sam {verb} 5 apples.")
|
||||||
|
assert len(out) == 1
|
||||||
|
cand = out[0]
|
||||||
|
assert isinstance(cand, CandidateOperation)
|
||||||
|
assert cand.op.kind == "add"
|
||||||
|
assert cand.matched_verb == verb
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Deliberate exclusions — verbs that must still refuse
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_gained_still_refused_delta_of_attribute_hazard():
|
||||||
|
"""``gained`` is delta-of-attribute (weight, age), not acquisition;
|
||||||
|
admitting it as add-operation would risk wrong>0 on questions
|
||||||
|
that ask total state. Deliberately excluded from
|
||||||
|
_ACQUISITION_VERBS."""
|
||||||
|
assert _extract("Orlando gained 5 pounds.") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_donated_still_refused_subtract_verb():
|
||||||
|
"""``donated`` is a SUBTRACT verb (actor gives away). Future W2.1
|
||||||
|
PR may add depletion-verb handling; for now, refuses."""
|
||||||
|
assert _extract("Alice donated 3 books.") is None
|
||||||
|
|
||||||
|
|
||||||
|
def test_saved_still_refused_ambiguous():
|
||||||
|
"""``saved`` is ambiguous between time/money/effort. Deliberately
|
||||||
|
excluded from _ACQUISITION_VERBS until disambiguation lands."""
|
||||||
|
assert _extract("Bob saved 50 apples.") is None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Case 0050 hazard pin — wrong=0 safety net
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_case_0050_hazard_unaffected_by_w2():
|
||||||
|
"""Case gsm8k-train-sample-v1-0050 must remain refused at
|
||||||
|
sentence_index=0. The acquisition-verb extension does not affect
|
||||||
|
the case 0050 sentence ``Mark does a gig every other day for 2
|
||||||
|
weeks.`` — ``does`` is not in _POSSESSION_VERBS or
|
||||||
|
_ACQUISITION_VERBS, so the DCS extractor refuses."""
|
||||||
|
from generate.math_candidate_graph import parse_and_solve
|
||||||
|
case_text = (
|
||||||
|
"Mark does a gig every other day for 2 weeks. "
|
||||||
|
"For each gig, he plays 3 songs. "
|
||||||
|
"2 of the songs are 5 minutes long and the last song is twice that long. "
|
||||||
|
"How many minutes did he play?"
|
||||||
|
)
|
||||||
|
result = parse_and_solve(case_text)
|
||||||
|
assert not result.is_admitted, (
|
||||||
|
f"case 0050 admitted post-W2: answer={result.answer!r} "
|
||||||
|
f"graph={result.selected_graph!r}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Determinism + wrong=0 invariant
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_w2_emission_deterministic_across_reruns():
|
||||||
|
"""Same anchor → byte-identical CandidateOperation. The new
|
||||||
|
acquisition path inherits the determinism contract."""
|
||||||
|
anchor = _extract("Nicole collected 400 paperclips.")
|
||||||
|
match = _make_match((anchor,))
|
||||||
|
out1 = inject_discrete_count_statement(match, "Nicole collected 400 paperclips.")
|
||||||
|
out2 = inject_discrete_count_statement(match, "Nicole collected 400 paperclips.")
|
||||||
|
assert out1 == out2
|
||||||
|
|
||||||
|
|
||||||
|
def test_w2_admission_path_passes_roundtrip_admissible():
|
||||||
|
"""The injected CandidateOperation must pass
|
||||||
|
``roundtrip_admissible`` — the existing wrong=0 safety net for
|
||||||
|
operations. This is the layer-3 check in ADR-0163.D.2's
|
||||||
|
five-layer net, now extended to the acquisition path."""
|
||||||
|
from generate.math_roundtrip import roundtrip_admissible
|
||||||
|
anchor = _extract("Nicole collected 400 paperclips.")
|
||||||
|
match = _make_match((anchor,))
|
||||||
|
out = inject_discrete_count_statement(match, "Nicole collected 400 paperclips.")
|
||||||
|
assert len(out) == 1
|
||||||
|
assert roundtrip_admissible(out[0])
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Dispatcher pin
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_inject_from_match_dispatches_to_acquisition_path():
|
||||||
|
"""The W1 dispatcher routes through to the W2 acquisition path
|
||||||
|
via the type-widened return contract."""
|
||||||
|
anchor = _extract("Sam bought 5 apples.")
|
||||||
|
match = _make_match((anchor,))
|
||||||
|
out = inject_from_match(match, "Sam bought 5 apples.")
|
||||||
|
assert len(out) == 1
|
||||||
|
assert isinstance(out[0], CandidateOperation)
|
||||||
|
assert out[0].op.kind == "add"
|
||||||
Loading…
Reference in a new issue