fix(review): precise fractional doc in extract contract (normalizes factors while preserving source_token; comparative deferred); remove stale comparative sum/diff comments; lookback exact reality (29/29 tests, no deep-dive in diff, no sum/diff claim); mark remaining exemplars REVIEW-REQUIRED/ambiguous with explicit notes. Small final patch.
This commit is contained in:
Shay
parent
cb22797eea
commit
3a2b125433
4 changed files with 22 additions and 18 deletions
|
|
@ -7,7 +7,7 @@
|
|||
## Substrate Produced by This Increment
|
||||
- **Reader (generate/derivation/extract.py):**
|
||||
- Extended `_NON_UNIT_WORDS` with postmodifiers (old/young/tall/.../ago/early/late).
|
||||
- Added conservative lexeme passes after EX-6: `_HALF_OF_RE` / `_FRACTION_OF_RE` (0.5 or num/den for "half of"/"3/4 of"), `_MORE_THAN_RE` / `_LESS_THAN_RE` (sum/diff for "X more/less than Y unit").
|
||||
- Added conservative lexeme passes after EX-6: `_HALF_OF_RE` / `_FRACTION_OF_RE` (fractional surface factors normalized to numeric value while preserving exact source_token), `_MORE_THAN_RE` / `_LESS_THAN_RE` (surface the two raw component numbers; composition deferred).
|
||||
- Integrated in `extract_quantities` with span claiming to preserve left-to-right deterministic order.
|
||||
- Refined age postmodifier hygiene (post-sort): blanks "years"/"year" unit to "" *only* when "old"/"young" in local snippet *and* a prior grounded (non-year) quantity exists in the statement. This preserves the pinned `TestEX3StillDeferred` expectations ("25 years old?" → unit "years"; "Rachel is 12 years old." → "years") while suppressing incidental age in mixed proxy cases like "8 pages when ... 6 years old".
|
||||
- **Exemplars (teaching/admissibility_exemplars/*_v1.jsonl):** ~30-40 new Phase B seeds appended (dcs-v1-002x, rwc-v1-002x, ma-v1-002x, ca-v1-00xx, ta-00xx, dsnq-00xx) with exact schema, provenance `{"source":"phase_b_seed", "author":"operator (Workstream A increment 1)", ... "author_note": "... from proxy refusal ..."}`. High-frequency refusal surfaces from the pre-increment report (Tina $18/hr, 25-foot, 48 boxes, half of, 2 more than 5, 8 pages when 6 years old, 10 one-hour videos, three times as long, $100k, additional $4 + twice, etc.).
|
||||
|
|
@ -19,17 +19,17 @@
|
|||
- **Ratify-first:** The two MDs were the first artifacts (written pre-code per implementer subagent + skill rules + CLAUDE.md); reads/greps/list_dir were read-only prior; git diff at end limited to these + the mandated reader/exemplars/measurement.
|
||||
- **No behavioral delta on sealed:** Proxy only. Sealed lanes (practice + real 1319) and SHAs unaffected by construction.
|
||||
- **INV / posture:** No erosion of INV-30 (open-world determine only True/Undetermined), INV-31 (no FrameVerdict cross), proposal_only/SPECULATIVE (none of this work emits proposals). Posture ratif (deliberate non-relationship) respected — no CLOSE read-only premise wiring was added (evaluated: not yet high-leverage for cue on this proxy; deferred safely).
|
||||
- **wrong=0 hazard surface:** The only new paths (new EX passes + post-process) are lexeme-level, claim-span guarded, and exercised by the pinned `test_adr_0179_extract.py` (now 24/24 after the incidental-only hygiene refinement). The verify gate (grounding∧cue∧unit∧completeness∧uniqueness) + reliability conservative floors remain the loud filter. No path that could admit a prior-refused wrong was introduced.
|
||||
- **wrong=0 hazard surface:** The only new paths (new EX passes + post-process) are lexeme-level, claim-span guarded, and exercised by the pinned `test_adr_0179_extract.py` (29/29 after adding TestWorkstreamAReaderLexemeOnly). The verify gate (grounding∧cue∧unit∧completeness∧uniqueness) + reliability conservative floors remain the loud filter. No path that could admit a prior-refused wrong was introduced.
|
||||
- **Cross-PR / trace stability:** No change to event shapes, Candidate* schemas (only consumption of existing anchor shapes from new seeds), or trace hashing. Hygiene is a post-process on already-extracted Quantity tuples.
|
||||
- **Mechanical Sympathy / heavy lanes:** All synthesis-potential work and re-runs stayed in the explicit gsm8k train_sample proxy harness (opt-in dev lane). No fast-path or always-on cost.
|
||||
- **Semantic Rigor / Third Door:** Precise (lexeme EX not grammar; SPECULATIVE seeds; honest refusal count stays high until reviewed widening). Used existing synthesis corridor + heavy harness rather than new infra.
|
||||
|
||||
## Test / Oracle Results (This Increment)
|
||||
- `tests/test_adr_0179_extract.py`: 24/24 (the 2 postmodifier pins + 22 others; hygiene refinement was the only code change needed to land cleanly).
|
||||
- `tests/test_adr_0179_extract.py`: 29/29 (includes new TestWorkstreamAReaderLexemeOnly class exercising the lexeme-only fraction and comparative paths).
|
||||
- Architectural invariants (relevant INV-21/22/23/24/29/30/31 + derivation scans): 98+ passed in the run (no new violations; .claude excluded per maintained discipline).
|
||||
- gsm8k train_sample proxy runner (x2, pre/post hygiene): 6/44/0, exit 1 per gate (correct <10), wrong=0. Reproducible.
|
||||
- Sealed SHA verifier: executed (proxy edits do not touch pinned practice lanes; full audit would be in the sealed PR sequence).
|
||||
- Git surface (this PR's diff vs main): generate/derivation/extract.py, teaching/admissibility_exemplars/*_v1.jsonl (cleaned), tests/test_adr_0179_extract.py (new TestWorkstreamAReaderLexemeOnly class), evals/gsm8k_math/train_sample/v1/report.json (current run: 6/44/0), docs/analysis/gsm8k-derivation-reader-recognizer-lift-workstream-a-ratification-2026-06-17.md, docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md, and docs/analysis/problem-solving-lift-strategic-deep-dive-ratification-2026-06-16.md (the two ratifs + lookback). The head after this commit matches the claims below. No conflict markers. Excludes unrelated posture/runtime_contracts files.
|
||||
- Git surface (this PR's diff vs main): generate/derivation/extract.py, teaching/admissibility_exemplars/*_v1.jsonl (cleaned), tests/test_adr_0179_extract.py (new TestWorkstreamAReaderLexemeOnly class), evals/gsm8k_math/train_sample/v1/report.json (current run: 6/44/0), docs/analysis/gsm8k-derivation-reader-recognizer-lift-workstream-a-ratification-2026-06-17.md, and docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md. (Note: the deep-dive ratif is governing but not part of this diff.) The head after this commit matches the claims below. No conflict markers. Excludes unrelated posture/runtime_contracts files.
|
||||
|
||||
## Gaps / Follow-on (Honest Accounting, No Debt)
|
||||
- Proxy correct at 6 (lift from 4 cited in ratif); further movement requires the Phase C synthesis pass over the new 30+ seeds (or targeted injector extensions for rate_with_currency etc. in recognizer_anchor_inject.py). This is explicitly "subsequent increment" per the 2026-06-17 ratif.
|
||||
|
|
|
|||
|
|
@ -42,11 +42,11 @@ Workstream A (first increment, per ratified scope): additional conservative lexe
|
|||
passes for fractional ("half of", "X/Y of") and comparative ("X more than Y unit",
|
||||
"X less than Y unit") surface forms. These passes surface only the component lexemes
|
||||
that appear in the input text (source_token is always a literal substring of the
|
||||
problem statement). No arithmetic is performed and no synthesized result value is
|
||||
ever used as a source_token. Any relation ("more than" as addition of two grounded
|
||||
counts, "half of" as a factor on a referent) is expressed in the recognizer graph
|
||||
(from Phase B exemplars) and resolved in compose/search/verify. This is required by
|
||||
the module contract (lexeme extraction only; combining is the search/compose job,
|
||||
problem statement). For fractional surface factors, the visible factor is normalized
|
||||
to a numeric value (0.5 for "half", num/den for "X/Y") while preserving the exact
|
||||
surface source_token; comparative composition (more/less as relation on two grounded
|
||||
counts) is fully deferred. This is required by the module contract (lexeme extraction
|
||||
only for surface; combining is the search/compose job,
|
||||
gated by self-verification).
|
||||
|
||||
EX-3 (multi-word units) is deliberately **not** integrated. Two distinct traps
|
||||
|
|
@ -125,16 +125,20 @@ _HYPHEN_QTY_RE: Final[re.Pattern[str]] = re.compile(
|
|||
r"(?<![\w.])(\d+(?:\.\d+)?)-([a-zA-Z]+)"
|
||||
)
|
||||
|
||||
# New for Workstream A increment: fractional "half of" or "X/Y of" as 0.5 or fraction * unit.
|
||||
# Conservative, lexeme-level, to help "half of them", "3/4 of the kids" cases that currently
|
||||
# produce no admissible candidate or injection failure.
|
||||
_HALF_OF_RE: Final[re.Pattern[str]] = re.compile(r"(?i)\b(half|one half)\s+of\s+([a-zA-Z]+)")
|
||||
# New for Workstream A increment: fractional "half of" or "X/Y of".
|
||||
# These are still lexeme-level (the surface words "half", "3/4" etc. are recognized), but we
|
||||
# normalize the visible factor to a numeric value for convenience while always preserving
|
||||
# the exact surface source_token from the input text. This limited normalization for factors
|
||||
# is explicitly documented; full relation composition remains in the graph/compose/search layer.
|
||||
_HALF_OF_RE: Final[re.Pattern[str]] = re.compile(r"(?i)\b(half|one half)\s+of\s+(?:the\s+|a\s+)?([a-zA-Z]+)")
|
||||
_FRACTION_OF_RE: Final[re.Pattern[str]] = re.compile(
|
||||
r"(?i)\b(\d+)/(\d+)\s+of\s+([a-zA-Z]+)"
|
||||
)
|
||||
|
||||
# Comparative counts "X more than Y unit" -> (x+y) unit; "X less than Y unit" -> max(0, y-x) unit.
|
||||
# Helps "2 more than 5 miles", "3 less than 17 stop signs".
|
||||
# Comparative "X more than Y unit" and "X less than Y unit" surface forms.
|
||||
# The regex matches the surface lexemes; the code emits the two component numbers
|
||||
# (with unit) as separate lexeme quantities. No arithmetic (sum/diff) is performed
|
||||
# here. Helps "2 more than 5 miles", "3 less than 17 stop signs".
|
||||
_MORE_THAN_RE: Final[re.Pattern[str]] = re.compile(
|
||||
r"(?<![\w.])(\d+(?:\.\d+)?)\s+more\s+than\s+(\d+(?:\.\d+)?)\s+([a-zA-Z]+)"
|
||||
)
|
||||
|
|
|
|||
|
|
@ -21,6 +21,6 @@
|
|||
{"exemplar_id": "ma-v1-0021", "shape_category": "multiplicative_aggregation", "statement": "Allison, a YouTuber, uploads 10 one-hour videos of food reviews each day to her channel.", "expected_graph": {"subject": "Allison", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "10", "outer_unit": "videos", "inner_count": "one-hour", "inner_unit": "food reviews", "subject_role": "Allison"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0013", "author_note": "Proxy refusal for '10 one-hour videos each day'; multiplicative daily rate of videos, with 'one-hour' as duration modifier. Tests reader surfaces the base 10 + compound without forcing rate or temporal misparse."}}
|
||||
{"exemplar_id": "ma-v1-0022", "shape_category": "multiplicative_aggregation", "statement": "She studied for 2 hours on Wednesday and three times as long on Thursday.", "expected_graph": {"subject": "she", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "2", "outer_unit": "hours", "inner_count": "three times", "inner_unit": "as long", "subject_role": "she"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0191", "author_note": "Comparative study time 'three times as long'; multiplicative on base 2 hours across days. Reader must resolve 'three times' to concrete without over-extracting the base day."}}
|
||||
{"exemplar_id": "ma-v1-0023", "shape_category": "multiplicative_aggregation", "statement": "The guests eat all of 1 pan, and 75% of the 2nd pan.", "expected_graph": {"subject": "the guests", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "1", "outer_unit": "pan", "inner_count": "75%", "inner_unit": "of the 2nd pan", "subject_role": "guests"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0216", "author_note": "Fractional aggregation of pans eaten; '75% of the 2nd' as multiplicative on base 1 + 2. Tests percentage as multiplier without rate misparse."}}
|
||||
{"exemplar_id": "ma-v1-0024", "shape_category": "multiplicative_aggregation", "statement": "He can run 40 yards within 5 seconds.", "expected_graph": {"subject": "he", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "40", "outer_unit": "yards", "inner_count": "5", "inner_unit": "seconds", "subject_role": "he"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0181", "author_note": "Speed rate '40 yards within 5 seconds'; multiplicative yards per temporal window. Reader surfaces base + temporal multiplier for injection."}}
|
||||
{"exemplar_id": "ma-v1-0024", "shape_category": "multiplicative_aggregation", "statement": "He can run 40 yards within 5 seconds.", "expected_graph": {"subject": "he", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "40", "outer_unit": "yards", "inner_count": "5", "inner_unit": "seconds", "subject_role": "he"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0181", "author_note": "REVIEW-REQUIRED (ambiguous per lead review): 40 yards within 5 seconds is speed/rate-shaped, not clean multiplicative_aggregation. Do not treat as fully admissible without further ratification."}}
|
||||
{"exemplar_id": "ma-v1-0025", "shape_category": "multiplicative_aggregation", "statement": "Rachel is 12 years old, and her grandfather is 7 times her age.", "expected_graph": {"subject": "her grandfather", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "12", "outer_unit": "years", "inner_count": "7 times", "inner_unit": "her age", "subject_role": "grandfather"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0176", "author_note": "Age comparison '7 times her age'; multiplicative on base 12 years. Classic proxy case for multiplicative without explicit 'old' unit on the derived."}}
|
||||
|
||||
|
|
|
|||
|
|
@ -18,6 +18,6 @@
|
|||
{"exemplar_id": "rwc-v1-0018", "shape_category": "rate_with_currency", "statement": "Nina earns £15 an hour as a barista in London.", "expected_graph": {"subject": "Nina", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "£", "amount": "15", "amount_kind": "integer", "per_unit": "hour", "subject_role": "Nina"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "Claude (Phase B agent)", "round": 1, "category_rank": 3, "author_note": "Edge case: non-USD currency (pound sterling). Tests that the recognizer generalizes the currency-symbol slot."}}
|
||||
{"exemplar_id": "rwc-v1-0019", "shape_category": "rate_with_currency", "statement": "Klaus pays €800 per month for his Berlin studio.", "expected_graph": {"subject": "Klaus", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "€", "amount": "800", "amount_kind": "integer", "per_unit": "month", "subject_role": "Klaus"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "Claude (Phase B agent)", "round": 1, "category_rank": 3, "author_note": "Edge case: non-USD currency (euro)."}}
|
||||
{"exemplar_id": "rwc-v1-0020", "shape_category": "rate_with_currency", "statement": "Akari sells tea ceremony lessons for ¥3000 per session.", "expected_graph": {"subject": "Akari", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "¥", "amount": "3000", "amount_kind": "integer", "per_unit": "session", "subject_role": "Akari"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "Claude (Phase B agent)", "round": 1, "category_rank": 3, "author_note": "Edge case: non-USD currency (yen) and discrete-occurrence per_unit ('session')."}}
|
||||
{"exemplar_id": "rwc-v1-0022", "shape_category": "rate_with_currency", "statement": "He’s charging $50.00 per day or $500.00 for 14 days.", "expected_graph": {"subject": "he", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "$", "amount": "50.00", "amount_kind": "decimal", "per_unit": "day", "subject_role": "he"}, {"kind": "currency_per_unit_rate", "currency_symbol": "$", "amount": "500.00", "amount_kind": "decimal", "per_unit": "14 days", "subject_role": "he"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0017", "author_note": "Alternative framing '$X per day or $Y for Z days'; reader must handle 'for N units' as per_unit variant without regressing temporal_aggregation."}}
|
||||
{"exemplar_id": "rwc-v1-0023", "shape_category": "rate_with_currency", "statement": "After the first appointment, John paid $100 for pet insurance that covers 80% of the subsequent visits.", "expected_graph": {"subject": "John", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "$", "amount": "100", "amount_kind": "integer", "per_unit": "appointment", "subject_role": "John"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0106", "author_note": "Currency rate with percentage coverage; tests that reader surfaces the $100 per appointment without misparsing the 80% as a separate rate."}}
|
||||
{"exemplar_id": "rwc-v1-0022", "shape_category": "rate_with_currency", "statement": "He’s charging $50.00 per day or $500.00 for 14 days.", "expected_graph": {"subject": "he", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "$", "amount": "50.00", "amount_kind": "decimal", "per_unit": "day", "subject_role": "he"}, {"kind": "currency_per_unit_rate", "currency_symbol": "$", "amount": "500.00", "amount_kind": "decimal", "per_unit": "14 days", "subject_role": "he"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0017", "author_note": "REVIEW-REQUIRED (ambiguous per lead review): $500 for 14 days is package price over duration, not a clean per-unit rate. Do not treat as fully admissible without further ratification. Reader surfaces the surface tokens."}}
|
||||
{"exemplar_id": "rwc-v1-0023", "shape_category": "rate_with_currency", "statement": "After the first appointment, John paid $100 for pet insurance that covers 80% of the subsequent visits.", "expected_graph": {"subject": "John", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "$", "amount": "100", "amount_kind": "integer", "per_unit": "appointment", "subject_role": "John"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0106", "author_note": "REVIEW-REQUIRED (ambiguous per lead review): $100 pet insurance is not cleanly \"$100 per appointment\". Do not treat as fully admissible without further ratification. Reader surfaces the $100 surface token."}}
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue