fix(exemplars): remove three REVIEW-REQUIRED ambiguous-but-machine-admissible seeds (rwc-v1-0022, rwc-v1-0023, ma-v1-0024) from ingested JSONL to eliminate synthesis-poisoning risk per final review. Update lookback wording for actual seed count and rationale. Small final patch.
This commit is contained in:
Shay
parent
3a2b125433
commit
0846a51373
3 changed files with 2 additions and 5 deletions
|
|
@ -10,7 +10,7 @@
|
|||
- Added conservative lexeme passes after EX-6: `_HALF_OF_RE` / `_FRACTION_OF_RE` (fractional surface factors normalized to numeric value while preserving exact source_token), `_MORE_THAN_RE` / `_LESS_THAN_RE` (surface the two raw component numbers; composition deferred).
|
||||
- Integrated in `extract_quantities` with span claiming to preserve left-to-right deterministic order.
|
||||
- Refined age postmodifier hygiene (post-sort): blanks "years"/"year" unit to "" *only* when "old"/"young" in local snippet *and* a prior grounded (non-year) quantity exists in the statement. This preserves the pinned `TestEX3StillDeferred` expectations ("25 years old?" → unit "years"; "Rachel is 12 years old." → "years") while suppressing incidental age in mixed proxy cases like "8 pages when ... 6 years old".
|
||||
- **Exemplars (teaching/admissibility_exemplars/*_v1.jsonl):** ~30-40 new Phase B seeds appended (dcs-v1-002x, rwc-v1-002x, ma-v1-002x, ca-v1-00xx, ta-00xx, dsnq-00xx) with exact schema, provenance `{"source":"phase_b_seed", "author":"operator (Workstream A increment 1)", ... "author_note": "... from proxy refusal ..."}`. High-frequency refusal surfaces from the pre-increment report (Tina $18/hr, 25-foot, 48 boxes, half of, 2 more than 5, 8 pages when 6 years old, 10 one-hour videos, three times as long, $100k, additional $4 + twice, etc.).
|
||||
- **Exemplars (teaching/admissibility_exemplars/*_v1.jsonl):** New Phase B seeds appended (dcs-v1-002x, rwc-v1-002x, ma-v1-002x, ca-v1-00xx, ta-00xx, dsnq-00xx) with exact schema, provenance `{"source":"phase_b_seed", "author":"operator (Workstream A increment 1)", ... "author_note": "... from proxy refusal ..."}`. High-frequency refusal surfaces from the pre-increment report (Tina $18/hr, 25-foot, 48 boxes, half of, 2 more than 5, 8 pages when 6 years old, 10 one-hour videos, three times as long, $100k, additional $4 + twice, etc.). Three previously added ambiguous exemplars (rwc-v1-0022, rwc-v1-0023, ma-v1-0024) were removed from the ingested corpus per final review to avoid machine-admissible but human-flagged seeds.
|
||||
- **Rebaseline:** `evals/gsm8k_math/train_sample/v1/runner.py` executed (uv + PYTHONPATH); report.json updated. Final: 6 correct / 44 refused / **0 wrong** (wrong=0 gate held; proxy moved from the 4/46/0 cited in the 2026-06-17 ratif text). Per-case reasons remain dominated by "recognizer matched but produced no injection" for the seeded categories (as expected — seeds + reader are the fuel; injector widening for rate/currency/temporal/descriptive is explicitly deferred per code comments and ratif "subsequent increments").
|
||||
- **No other logic:** report.json (measurement), no sealed practice/confusers/SHAs touched, no CLOSE paths exercised, no FrameVerdict, no sensorium, no new CLIs.
|
||||
|
||||
|
|
@ -32,7 +32,7 @@
|
|||
- Git surface (this PR's diff vs main): generate/derivation/extract.py, teaching/admissibility_exemplars/*_v1.jsonl (cleaned), tests/test_adr_0179_extract.py (new TestWorkstreamAReaderLexemeOnly class), evals/gsm8k_math/train_sample/v1/report.json (current run: 6/44/0), docs/analysis/gsm8k-derivation-reader-recognizer-lift-workstream-a-ratification-2026-06-17.md, and docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md. (Note: the deep-dive ratif is governing but not part of this diff.) The head after this commit matches the claims below. No conflict markers. Excludes unrelated posture/runtime_contracts files.
|
||||
|
||||
## Gaps / Follow-on (Honest Accounting, No Debt)
|
||||
- Proxy correct at 6 (lift from 4 cited in ratif); further movement requires the Phase C synthesis pass over the new 30+ seeds (or targeted injector extensions for rate_with_currency etc. in recognizer_anchor_inject.py). This is explicitly "subsequent increment" per the 2026-06-17 ratif.
|
||||
- Proxy correct at 6 (lift from 4 cited in ratif); further movement requires the Phase C synthesis pass over the new seeds (or targeted injector extensions for rate_with_currency etc. in recognizer_anchor_inject.py). This is explicitly "subsequent increment" per the 2026-06-17 ratif.
|
||||
- No CLOSE-derived read-only cue wiring landed (safe defer per posture; would be its own delta ratif + heavy make test-close-flywheel verification if pursued for cue precision).
|
||||
- Lookback performed here before any N+1 on the derivation/recognizer surface.
|
||||
- Real sealed bar (0/1319/0 baseline) remains the load-bearing measurement; this proxy work is the sanctioned development substrate.
|
||||
|
|
|
|||
|
|
@ -21,6 +21,5 @@
|
|||
{"exemplar_id": "ma-v1-0021", "shape_category": "multiplicative_aggregation", "statement": "Allison, a YouTuber, uploads 10 one-hour videos of food reviews each day to her channel.", "expected_graph": {"subject": "Allison", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "10", "outer_unit": "videos", "inner_count": "one-hour", "inner_unit": "food reviews", "subject_role": "Allison"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0013", "author_note": "Proxy refusal for '10 one-hour videos each day'; multiplicative daily rate of videos, with 'one-hour' as duration modifier. Tests reader surfaces the base 10 + compound without forcing rate or temporal misparse."}}
|
||||
{"exemplar_id": "ma-v1-0022", "shape_category": "multiplicative_aggregation", "statement": "She studied for 2 hours on Wednesday and three times as long on Thursday.", "expected_graph": {"subject": "she", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "2", "outer_unit": "hours", "inner_count": "three times", "inner_unit": "as long", "subject_role": "she"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0191", "author_note": "Comparative study time 'three times as long'; multiplicative on base 2 hours across days. Reader must resolve 'three times' to concrete without over-extracting the base day."}}
|
||||
{"exemplar_id": "ma-v1-0023", "shape_category": "multiplicative_aggregation", "statement": "The guests eat all of 1 pan, and 75% of the 2nd pan.", "expected_graph": {"subject": "the guests", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "1", "outer_unit": "pan", "inner_count": "75%", "inner_unit": "of the 2nd pan", "subject_role": "guests"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0216", "author_note": "Fractional aggregation of pans eaten; '75% of the 2nd' as multiplicative on base 1 + 2. Tests percentage as multiplier without rate misparse."}}
|
||||
{"exemplar_id": "ma-v1-0024", "shape_category": "multiplicative_aggregation", "statement": "He can run 40 yards within 5 seconds.", "expected_graph": {"subject": "he", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "40", "outer_unit": "yards", "inner_count": "5", "inner_unit": "seconds", "subject_role": "he"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0181", "author_note": "REVIEW-REQUIRED (ambiguous per lead review): 40 yards within 5 seconds is speed/rate-shaped, not clean multiplicative_aggregation. Do not treat as fully admissible without further ratification."}}
|
||||
{"exemplar_id": "ma-v1-0025", "shape_category": "multiplicative_aggregation", "statement": "Rachel is 12 years old, and her grandfather is 7 times her age.", "expected_graph": {"subject": "her grandfather", "quantity_anchors": [{"kind": "multiplicative_aggregate", "outer_count": "12", "outer_unit": "years", "inner_count": "7 times", "inner_unit": "her age", "subject_role": "grandfather"}], "graph_intent": "aggregate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0176", "author_note": "Age comparison '7 times her age'; multiplicative on base 12 years. Classic proxy case for multiplicative without explicit 'old' unit on the derived."}}
|
||||
|
||||
|
|
|
|||
|
|
@ -18,6 +18,4 @@
|
|||
{"exemplar_id": "rwc-v1-0018", "shape_category": "rate_with_currency", "statement": "Nina earns £15 an hour as a barista in London.", "expected_graph": {"subject": "Nina", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "£", "amount": "15", "amount_kind": "integer", "per_unit": "hour", "subject_role": "Nina"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "Claude (Phase B agent)", "round": 1, "category_rank": 3, "author_note": "Edge case: non-USD currency (pound sterling). Tests that the recognizer generalizes the currency-symbol slot."}}
|
||||
{"exemplar_id": "rwc-v1-0019", "shape_category": "rate_with_currency", "statement": "Klaus pays €800 per month for his Berlin studio.", "expected_graph": {"subject": "Klaus", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "€", "amount": "800", "amount_kind": "integer", "per_unit": "month", "subject_role": "Klaus"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "Claude (Phase B agent)", "round": 1, "category_rank": 3, "author_note": "Edge case: non-USD currency (euro)."}}
|
||||
{"exemplar_id": "rwc-v1-0020", "shape_category": "rate_with_currency", "statement": "Akari sells tea ceremony lessons for ¥3000 per session.", "expected_graph": {"subject": "Akari", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "¥", "amount": "3000", "amount_kind": "integer", "per_unit": "session", "subject_role": "Akari"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "Claude (Phase B agent)", "round": 1, "category_rank": 3, "author_note": "Edge case: non-USD currency (yen) and discrete-occurrence per_unit ('session')."}}
|
||||
{"exemplar_id": "rwc-v1-0022", "shape_category": "rate_with_currency", "statement": "He’s charging $50.00 per day or $500.00 for 14 days.", "expected_graph": {"subject": "he", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "$", "amount": "50.00", "amount_kind": "decimal", "per_unit": "day", "subject_role": "he"}, {"kind": "currency_per_unit_rate", "currency_symbol": "$", "amount": "500.00", "amount_kind": "decimal", "per_unit": "14 days", "subject_role": "he"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0017", "author_note": "REVIEW-REQUIRED (ambiguous per lead review): $500 for 14 days is package price over duration, not a clean per-unit rate. Do not treat as fully admissible without further ratification. Reader surfaces the surface tokens."}}
|
||||
{"exemplar_id": "rwc-v1-0023", "shape_category": "rate_with_currency", "statement": "After the first appointment, John paid $100 for pet insurance that covers 80% of the subsequent visits.", "expected_graph": {"subject": "John", "quantity_anchors": [{"kind": "currency_per_unit_rate", "currency_symbol": "$", "amount": "100", "amount_kind": "integer", "per_unit": "appointment", "subject_role": "John"}], "graph_intent": "rate", "outcome": "admissible"}, "provenance": {"source": "phase_b_seed", "author": "operator (Workstream A increment 1)", "round": 2, "category_rank": 1, "train_case_id": "gsm8k-train-sample-v1-0106", "author_note": "REVIEW-REQUIRED (ambiguous per lead review): $100 pet insurance is not cleanly \"$100 per appointment\". Do not treat as fully admissible without further ratification. Reader surfaces the $100 surface token."}}
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue