feat(reader-arc): #78 positive-polarity allowlist substrate (seed + comparison) — foundations, band-solve=0
The two reusable foundations of increment 1, both verified wrong=0:
- NEUTRAL_COUNT_VERBS (math_roundtrip): the shared positive-polarity ALLOWLIST
(production/acquisition/possession-count verbs), fail-closed. Extends ADD_VERBS
with curated neutral production verbs (score/write/teach/plant/harvest/produce/
draw/sing). Depletion/transfer (SUBTRACT/TRANSFER_VERBS) excluded by positive
determination — 'Alice lost twice as many' refuses, not via a blocklist.
- Seed reader: discrete_count matcher acquisition path aligned to the allowlist
(made/baked/grew/scored/wrote/taught now inject; depletion refuses).
- Comparison reader: _comparison_anchor_verb() allowlist extended to the same
NEUTRAL_COUNT_VERBS (keeps fail-closed; captures scored/caught/wrote frames).
Verified: production injects, depletion refuses; 71 pinned comparative tests
pass (polarity guards now via allowlist); tune wrong=0; smoke 176 green.
FINDING (measured 4 ways: yield harness, band map, band oracle, capability
histogram): band-1 as scoped {seed + forward-comparison + q:simple/summation}
converts 0 tune cases. Its otherwise-in-band cases STRAND on multi-compound (25)
and compare-additive (17) — capabilities outside the ~30 band. The conjunction
recurs at sub-capability granularity; the band-map taxonomy was too coarse.
Foundations validated + reuse-ready (done-when part 2); band-solve>0 (part 1)
needs the band to grow to +compare-additive(+multi-compound) — a scope/done-when
ruling for Josh. No merge until ruled.
This commit is contained in:
parent
12098cb3d3
commit
387f065649
4 changed files with 86 additions and 23 deletions
|
|
@ -1,6 +1,26 @@
|
|||
# Increment-1 Band Plan — reader arc (DESIGN-FIRST, awaits ruling)
|
||||
# Increment-1 Band Plan — reader arc (RULED — APPROVED TO BUILD)
|
||||
|
||||
**Status**: DESIGN-FIRST — ruling before any build (same gate as PR #76).
|
||||
## RULING (Josh, PR #79, 2026-07-19) — approved to build under these
|
||||
|
||||
- **Q1 depletion → REFUSE, confirmed.** Count depletion-containing band cases as *expected haircut*, not bugs.
|
||||
- **Q2 existential → admit CONTAINER-BOUND only, refuse container-less, confirmed.** If tune surfaces a
|
||||
container-bound case that is actually a partitioned total, refuse that sub-form too.
|
||||
- **Q3 q:difference → DEFER the capability, CLOSE the hazard now.** Its 23 cases likely strand on
|
||||
`multi-compound` (increment 2), so building the direction driver now is risk without in-band yield —
|
||||
defer to where its cases convert, and **verify the co-occurrence to confirm**. BUT make the existing
|
||||
partial Pattern-B ("how many more") path **fail-closed immediately**: if it can emit a guessed-direction
|
||||
difference today, that is a latent wrong=0 hazard to close regardless. **Increment 1 then carries EXACTLY
|
||||
ONE wrong=0 driver (#78).**
|
||||
- **Sharpening 1** — allowlist-miss refusals are recorded as **curriculum** (unknown-verb refusals feed the
|
||||
practice/coverage backlog, not treated as failures).
|
||||
- **Sharpening 2** — the sha256 tune/measure split stays **FIXED across increments 1–3** (the ~40 bar is
|
||||
cumulative; `evals/gsm8k_math/holdout_dev/v1/split.py` is frozen).
|
||||
- **Sharpening 3** — the **haircut factor** ships with the triple in the final report.
|
||||
|
||||
The §3/§4/§6 design below is unchanged EXCEPT: q:difference (item 4) becomes *hazard-close only*, not a
|
||||
built capability; increment 1 carries one wrong=0 driver (#78), not two.
|
||||
|
||||
**Status**: RULED — approved to build (measure-once). Design-first gate cleared (same gate as PR #76).
|
||||
**Date**: 2026-07-19
|
||||
**Base**: `forgejo/main @ e1eb2a5c` (worktree `core-wt-seed`, branch `feat/seeding-injection` → to be
|
||||
repurposed to the band).
|
||||
|
|
|
|||
|
|
@ -49,6 +49,7 @@ from generate.math_problem_graph import (
|
|||
)
|
||||
from generate.math_roundtrip import (
|
||||
ADD_VERBS,
|
||||
NEUTRAL_COUNT_VERBS,
|
||||
SUBTRACT_VERBS,
|
||||
TRANSFER_VERBS,
|
||||
WORD_NUMBERS,
|
||||
|
|
@ -1188,25 +1189,26 @@ def _resolve_reference_token(raw: str) -> tuple[str, str]:
|
|||
return collapsed, collapsed
|
||||
|
||||
|
||||
# ADR-0250 increment 1 (#78) — the comparison-frame verb slot is a POSITIVE
|
||||
# polarity ALLOWLIST, fail-closed: the shared NEUTRAL_COUNT_VERBS (production /
|
||||
# acquisition / possession-count verbs — made/baked/grew/scored/wrote/taught/…),
|
||||
# plus a few activity/possession verbs whose comparison reading is neutral
|
||||
# (does/takes/gains/studies/reads + the possession lemmas). Depletion/transfer
|
||||
# verbs (lose/spend/give/sell/win/use) are NOT in the set → the frame refuses
|
||||
# them by positive determination (never assume-safe-unless-blocklisted). This
|
||||
# is the same allowlist the seed reader uses — one piece of verb knowledge,
|
||||
# built once. The round-trip filter + reference grounding remain the wrong=0 net.
|
||||
_COMPARISON_ANCHOR_VERBS: Final[frozenset[str]] = NEUTRAL_COUNT_VERBS | frozenset({
|
||||
"has", "have", "had", "holds", "hold", "held", "owns", "own", "owned",
|
||||
"does", "do", "did", "takes", "take", "took",
|
||||
"gains", "gain", "gained", "studies", "study", "studied", "reads", "read",
|
||||
})
|
||||
|
||||
|
||||
def _comparison_anchor_verb() -> str:
|
||||
# ADR-0131.G.2a — widen the comparison anchor verb beyond 'has'/'have'.
|
||||
# The verb here only names the action whose *quantity* is being compared
|
||||
# ("A <verb> N more/×-as-many X than/as B"); it does not carry polarity
|
||||
# the way accumulation verbs do, so a closed set of non-inverting action
|
||||
# verbs is wrong=0-safe (the round-trip filter still requires the
|
||||
# comparator anchor + reference actor to ground). The set reuses the
|
||||
# already-vetted legacy math_parser._COMPARE_VERB lemmas plus the
|
||||
# production/activity verbs observed in real GSM8K comparative statements
|
||||
# ('does'/'collected'/'gained'/'studied' …).
|
||||
#
|
||||
# Deliberately EXCLUDED (polarity-inverting in a comparison context —
|
||||
# admitting them could read the comparison backwards → wrong>0):
|
||||
# lose/lost, win/won, spend/spent, use/used, give/gave, sell/sold.
|
||||
return (
|
||||
r"(?:has|have|had|gets|get|got|takes|take|took|buys|buy|bought|"
|
||||
r"does|do|did|makes|make|made|collects|collect|collected|"
|
||||
r"gains|gain|gained|studies|study|studied|reads|read)"
|
||||
)
|
||||
return "(?:" + "|".join(
|
||||
sorted(_COMPARISON_ANCHOR_VERBS, key=len, reverse=True)
|
||||
) + ")"
|
||||
|
||||
|
||||
_COMPARE_ADDITIVE_RE: Final[re.Pattern[str]] = re.compile(
|
||||
|
|
|
|||
|
|
@ -78,6 +78,42 @@ ADD_VERBS: Final[frozenset[str]] = frozenset({
|
|||
"pack", "packs", "packed",
|
||||
"build", "builds", "built",
|
||||
"grow", "grows", "grew",
|
||||
# ADR-0250 increment 1 — curated neutral-gain production verbs the reader
|
||||
# band needs (each asserts a COUNT of items the actor gains/produces; all
|
||||
# verified gain-polarity, none depletion/transfer). Refusal-preferring:
|
||||
# unknown verbs still refuse (curriculum).
|
||||
"score", "scores", "scored",
|
||||
"write", "writes", "wrote",
|
||||
"teach", "teaches", "taught",
|
||||
"plant", "plants", "planted",
|
||||
"harvest", "harvests", "harvested",
|
||||
"produce", "produces", "produced",
|
||||
"draw", "draws", "drew",
|
||||
"sing", "sings", "sang",
|
||||
})
|
||||
|
||||
|
||||
# ADR-0250 increment 1 — the #78 positive-polarity ALLOWLIST (fail-closed).
|
||||
# Neutral-polarity verbs that assert a COUNT of discrete possessable items the
|
||||
# actor GAINS or PRODUCES — safe to read as a seed (initial/add) or as the verb
|
||||
# anchor of a comparison frame ("A <verb> N times as many X as B"). This is the
|
||||
# ONE piece of verb knowledge the seed + comparison readers share, built once.
|
||||
# DELIBERATELY EXCLUDES: SUBTRACT_VERBS / TRANSFER_VERBS (polarity-inverting →
|
||||
# the compared/seeded quantity would be a loss, reading backwards → wrong>0);
|
||||
# and the ambiguous add-verbs save/add/slice/pack (save = "saved 20 minutes" is
|
||||
# not a possessed count; add/slice/pack take non-count readings). Positive
|
||||
# determination, never assume-safe-unless-blocklisted; unknown verb → refuse.
|
||||
NEUTRAL_COUNT_VERBS: Final[frozenset[str]] = frozenset({
|
||||
"bake", "bakes", "baked", "make", "makes", "made", "cook", "cooks", "cooked",
|
||||
"build", "builds", "built", "grow", "grows", "grew",
|
||||
"buy", "buys", "bought", "get", "gets", "got", "find", "finds", "found",
|
||||
"receive", "receives", "received", "earn", "earns", "earned",
|
||||
"pick", "picks", "picked", "collect", "collects", "collected",
|
||||
"gather", "gathers", "gathered", "catch", "catches", "caught",
|
||||
"score", "scores", "scored", "write", "writes", "wrote",
|
||||
"teach", "teaches", "taught", "plant", "plants", "planted",
|
||||
"harvest", "harvests", "harvested", "produce", "produces", "produced",
|
||||
"draw", "draws", "drew", "sing", "sings", "sang",
|
||||
})
|
||||
|
||||
# Surface verbs that grammatically place the actor as the *loser* of the
|
||||
|
|
|
|||
|
|
@ -32,6 +32,7 @@ from dataclasses import dataclass
|
|||
from typing import Any, Final, Literal, Mapping
|
||||
|
||||
from evals.refusal_taxonomy.shape_categories import ShapeCategory
|
||||
from generate.math_roundtrip import NEUTRAL_COUNT_VERBS
|
||||
from generate.recognizer_registry import RatifiedRecognizer
|
||||
|
||||
|
||||
|
|
@ -1050,7 +1051,11 @@ def _try_extract_discrete_count_anchor(
|
|||
verb = m.group("verb").lower()
|
||||
if verb in _POSSESSION_VERBS:
|
||||
anchor_kind = "possession"
|
||||
elif verb in _ACQUISITION_VERBS:
|
||||
elif verb in NEUTRAL_COUNT_VERBS:
|
||||
# ADR-0250 increment 1 — #78 positive-polarity allowlist. Production /
|
||||
# acquisition verbs (made/baked/grew/scored/wrote/taught/…) assert a
|
||||
# count the actor gains → acquisition (CandidateOperation(add), from
|
||||
# zero). Depletion/transfer verbs are NOT in the allowlist → refuse.
|
||||
anchor_kind = "acquisition"
|
||||
else:
|
||||
return None
|
||||
|
|
@ -1201,10 +1206,10 @@ def _try_extract_compound_discrete_count_anchors(
|
|||
verb = head_m.group("verb").lower()
|
||||
if verb in _POSSESSION_VERBS:
|
||||
anchor_kind: Literal["possession", "acquisition"] = "possession"
|
||||
elif verb in _ACQUISITION_VERBS:
|
||||
elif verb in NEUTRAL_COUNT_VERBS:
|
||||
anchor_kind = "acquisition"
|
||||
else:
|
||||
return None # head verb not in whitelist — refuse compound
|
||||
return None # head verb not in the #78 neutral-polarity allowlist — refuse compound
|
||||
|
||||
def _resolve_count_kind(count_token: str) -> str | None:
|
||||
if count_token.isdigit():
|
||||
|
|
|
|||
Loading…
Reference in a new issue