From 3a2b1254332127142cd29c460e11bb62278f33ea Mon Sep 17 00:00:00 2001 From: Shay Date: Tue, 16 Jun 2026 21:25:16 -0700 Subject: [PATCH] fix(review): precise fractional doc in extract contract (normalizes factors while preserving source_token; comparative deferred); remove stale comparative sum/diff comments; lookback exact reality (29/29 tests, no deep-dive in diff, no sum/diff claim); mark remaining exemplars REVIEW-REQUIRED/ambiguous with explicit notes. Small final patch. --- ...tream-a-increment-1-lookback-2026-06-17.md | 8 +++--- generate/derivation/extract.py | 26 +++++++++++-------- .../multiplicative_aggregation_v1.jsonl | 2 +- .../rate_with_currency_v1.jsonl | 4 +-- 4 files changed, 22 insertions(+), 18 deletions(-) diff --git a/docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md b/docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md index 757b9fc1..da9638d4 100644 --- a/docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md +++ b/docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md @@ -7,7 +7,7 @@ ## Substrate Produced by This Increment - **Reader (generate/derivation/extract.py):** - Extended `_NON_UNIT_WORDS` with postmodifiers (old/young/tall/.../ago/early/late). - - Added conservative lexeme passes after EX-6: `_HALF_OF_RE` / `_FRACTION_OF_RE` (0.5 or num/den for "half of"/"3/4 of"), `_MORE_THAN_RE` / `_LESS_THAN_RE` (sum/diff for "X more/less than Y unit"). + - Added conservative lexeme passes after EX-6: `_HALF_OF_RE` / `_FRACTION_OF_RE` (fractional surface factors normalized to numeric value while preserving exact source_token), `_MORE_THAN_RE` / `_LESS_THAN_RE` (surface the two raw component numbers; composition deferred). - Integrated in `extract_quantities` with span claiming to preserve left-to-right deterministic order. - Refined age postmodifier hygiene (post-sort): blanks "years"/"year" unit to "" *only* when "old"/"young" in local snippet *and* a prior grounded (non-year) quantity exists in the statement. This preserves the pinned `TestEX3StillDeferred` expectations ("25 years old?" → unit "years"; "Rachel is 12 years old." → "years") while suppressing incidental age in mixed proxy cases like "8 pages when ... 6 years old". - **Exemplars (teaching/admissibility_exemplars/*_v1.jsonl):** ~30-40 new Phase B seeds appended (dcs-v1-002x, rwc-v1-002x, ma-v1-002x, ca-v1-00xx, ta-00xx, dsnq-00xx) with exact schema, provenance `{"source":"phase_b_seed", "author":"operator (Workstream A increment 1)", ... "author_note": "... from proxy refusal ..."}`. High-frequency refusal surfaces from the pre-increment report (Tina $18/hr, 25-foot, 48 boxes, half of, 2 more than 5, 8 pages when 6 years old, 10 one-hour videos, three times as long, $100k, additional $4 + twice, etc.). @@ -19,17 +19,17 @@ - **Ratify-first:** The two MDs were the first artifacts (written pre-code per implementer subagent + skill rules + CLAUDE.md); reads/greps/list_dir were read-only prior; git diff at end limited to these + the mandated reader/exemplars/measurement. - **No behavioral delta on sealed:** Proxy only. Sealed lanes (practice + real 1319) and SHAs unaffected by construction. - **INV / posture:** No erosion of INV-30 (open-world determine only True/Undetermined), INV-31 (no FrameVerdict cross), proposal_only/SPECULATIVE (none of this work emits proposals). Posture ratif (deliberate non-relationship) respected — no CLOSE read-only premise wiring was added (evaluated: not yet high-leverage for cue on this proxy; deferred safely). -- **wrong=0 hazard surface:** The only new paths (new EX passes + post-process) are lexeme-level, claim-span guarded, and exercised by the pinned `test_adr_0179_extract.py` (now 24/24 after the incidental-only hygiene refinement). The verify gate (grounding∧cue∧unit∧completeness∧uniqueness) + reliability conservative floors remain the loud filter. No path that could admit a prior-refused wrong was introduced. +- **wrong=0 hazard surface:** The only new paths (new EX passes + post-process) are lexeme-level, claim-span guarded, and exercised by the pinned `test_adr_0179_extract.py` (29/29 after adding TestWorkstreamAReaderLexemeOnly). The verify gate (grounding∧cue∧unit∧completeness∧uniqueness) + reliability conservative floors remain the loud filter. No path that could admit a prior-refused wrong was introduced. - **Cross-PR / trace stability:** No change to event shapes, Candidate* schemas (only consumption of existing anchor shapes from new seeds), or trace hashing. Hygiene is a post-process on already-extracted Quantity tuples. - **Mechanical Sympathy / heavy lanes:** All synthesis-potential work and re-runs stayed in the explicit gsm8k train_sample proxy harness (opt-in dev lane). No fast-path or always-on cost. - **Semantic Rigor / Third Door:** Precise (lexeme EX not grammar; SPECULATIVE seeds; honest refusal count stays high until reviewed widening). Used existing synthesis corridor + heavy harness rather than new infra. ## Test / Oracle Results (This Increment) -- `tests/test_adr_0179_extract.py`: 24/24 (the 2 postmodifier pins + 22 others; hygiene refinement was the only code change needed to land cleanly). +- `tests/test_adr_0179_extract.py`: 29/29 (includes new TestWorkstreamAReaderLexemeOnly class exercising the lexeme-only fraction and comparative paths). - Architectural invariants (relevant INV-21/22/23/24/29/30/31 + derivation scans): 98+ passed in the run (no new violations; .claude excluded per maintained discipline). - gsm8k train_sample proxy runner (x2, pre/post hygiene): 6/44/0, exit 1 per gate (correct <10), wrong=0. Reproducible. - Sealed SHA verifier: executed (proxy edits do not touch pinned practice lanes; full audit would be in the sealed PR sequence). -- Git surface (this PR's diff vs main): generate/derivation/extract.py, teaching/admissibility_exemplars/*_v1.jsonl (cleaned), tests/test_adr_0179_extract.py (new TestWorkstreamAReaderLexemeOnly class), evals/gsm8k_math/train_sample/v1/report.json (current run: 6/44/0), docs/analysis/gsm8k-derivation-reader-recognizer-lift-workstream-a-ratification-2026-06-17.md, docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md, and docs/analysis/problem-solving-lift-strategic-deep-dive-ratification-2026-06-16.md (the two ratifs + lookback). The head after this commit matches the claims below. No conflict markers. Excludes unrelated posture/runtime_contracts files. +- Git surface (this PR's diff vs main): generate/derivation/extract.py, teaching/admissibility_exemplars/*_v1.jsonl (cleaned), tests/test_adr_0179_extract.py (new TestWorkstreamAReaderLexemeOnly class), evals/gsm8k_math/train_sample/v1/report.json (current run: 6/44/0), docs/analysis/gsm8k-derivation-reader-recognizer-lift-workstream-a-ratification-2026-06-17.md, and docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md. (Note: the deep-dive ratif is governing but not part of this diff.) The head after this commit matches the claims below. No conflict markers. Excludes unrelated posture/runtime_contracts files. ## Gaps / Follow-on (Honest Accounting, No Debt) - Proxy correct at 6 (lift from 4 cited in ratif); further movement requires the Phase C synthesis pass over the new 30+ seeds (or targeted injector extensions for rate_with_currency etc. in recognizer_anchor_inject.py). This is explicitly "subsequent increment" per the 2026-06-17 ratif. diff --git a/generate/derivation/extract.py b/generate/derivation/extract.py index 134ff410..322bfcf4 100644 --- a/generate/derivation/extract.py +++ b/generate/derivation/extract.py @@ -42,11 +42,11 @@ Workstream A (first increment, per ratified scope): additional conservative lexe passes for fractional ("half of", "X/Y of") and comparative ("X more than Y unit", "X less than Y unit") surface forms. These passes surface only the component lexemes that appear in the input text (source_token is always a literal substring of the -problem statement). No arithmetic is performed and no synthesized result value is -ever used as a source_token. Any relation ("more than" as addition of two grounded -counts, "half of" as a factor on a referent) is expressed in the recognizer graph -(from Phase B exemplars) and resolved in compose/search/verify. This is required by -the module contract (lexeme extraction only; combining is the search/compose job, +problem statement). For fractional surface factors, the visible factor is normalized +to a numeric value (0.5 for "half", num/den for "X/Y") while preserving the exact +surface source_token; comparative composition (more/less as relation on two grounded +counts) is fully deferred. This is required by the module contract (lexeme extraction +only for surface; combining is the search/compose job, gated by self-verification). EX-3 (multi-word units) is deliberately **not** integrated. Two distinct traps @@ -125,16 +125,20 @@ _HYPHEN_QTY_RE: Final[re.Pattern[str]] = re.compile( r"(? (x+y) unit; "X less than Y unit" -> max(0, y-x) unit. -# Helps "2 more than 5 miles", "3 less than 17 stop signs". +# Comparative "X more than Y unit" and "X less than Y unit" surface forms. +# The regex matches the surface lexemes; the code emits the two component numbers +# (with unit) as separate lexeme quantities. No arithmetic (sum/diff) is performed +# here. Helps "2 more than 5 miles", "3 less than 17 stop signs". _MORE_THAN_RE: Final[re.Pattern[str]] = re.compile( r"(?