core/generate/derivation
Shay 80240ea9b8
feat(derivation): Workstream A inc 1 — lexeme reader + cleaned exemplars + proxy rebaseline (#796)
* feat(derivation): Workstream A inc 1 — lexeme-only reader components for fractions/comparatives + cleaned Phase B exemplars + proxy rebaseline (ratified, post-review fixes)

- generate/derivation/extract.py: lexeme-only passes (surface tokens only; no synthesized values or non-surface source_tokens for "half of", "X/Y of", "X more/less than"). Composition deferred per module contract (ADR-0165). Hygiene and EX-6 preserved. Docstring updated.
- teaching/admissibility_exemplars/*_v1.jsonl: grown + cleaned (removed category-polluted seeds, deduplicated ma IDs, fixed comparative exemplar to surface tokens per lexeme rule).
- evals/gsm8k_math/train_sample/v1/report.json: rebaselined after fixes (runner executed; wrong=0 held).
- docs/analysis/*-2026-06-1[6-7].md + lookback: ratif + honest post-review lookback (local paths removed, scope claims match actual clean diff, extractor/exemplar fixes documented).

Governing: the two 2026-06-16/17 ratifications (included; local session path replaced with stable ref). Scope per ratif: reader (lexeme) + exemplar growth (clean) + rebaseline + tests + lookback. No sealed, no CLOSE (deferred), 0 wrong.

Added tests in test_adr_0179_extract.py for new lexeme paths + grounding.

Branch prepared clean from origin/main; diff limited to scoped files only (no posture ratif, no runtime_contracts drift).

See updated lookback and PR body for full evidence (6/44/0 or current post-fix,  tests green, etc.).

(Addresses review: Semantic Rigor restored, branch hygiene, exemplar quality, tests, docs.)

* fix(review): resolve all verifiability blockers

- generate/derivation/extract.py: conflict markers removed (was from stash); lexeme-only implementation confirmed clean.
- tests/test_adr_0179_extract.py: added TestWorkstreamAReaderLexemeOnly class with direct tests for half-of, fraction-of, more/less components, source_token grounding, no-synthesis invariants.
- evals/gsm8k_math/train_sample/v1/report.json: fresh rebaseline via runner with fixed code (6/44/0, wrong=0).
- docs/analysis/gsm8k-workstream-a-increment-1-lookback-2026-06-17.md: updated to exactly match the actual diff and actual report numbers on this head.
- No conflict markers remain in extract, tests, lookback, ratifs, or exemplars (verified by grep).
- Branch head now clean and matches all claims in PR body/lookback.

This is the verifiable head for the lead engineer review.

* fix(test): make lexeme-only half test case-insensitive on source_token to match actual extraction (preserves input casing like other EX tests)

* fix(review): commit current report.json (6/44/0 from runner) and updated lookback so they appear in branch diff; lookback now exactly matches actual files in git diff --name-only and actual test count (29) and report numbers.

* fix(review): precise fractional doc in extract contract (normalizes factors while preserving source_token; comparative deferred); remove stale comparative sum/diff comments; lookback exact reality (29/29 tests, no deep-dive in diff, no sum/diff claim); mark remaining exemplars REVIEW-REQUIRED/ambiguous with explicit notes. Small final patch.

* fix(exemplars): remove three REVIEW-REQUIRED ambiguous-but-machine-admissible seeds (rwc-v1-0022, rwc-v1-0023, ma-v1-0024) from ingested JSONL to eliminate synthesis-poisoning risk per final review. Update lookback wording for actual seed count and rationale. Small final patch.

* docs(lookback): align exemplar category list with actual diff (only dcs/rwc/ma touched in this increment; no ca/ta/dsnq changes)
2026-06-16 21:57:05 -07:00
..
state test(derivation): add ADR-0184 semantic replay equivalence harness 2026-06-10 14:35:22 -07:00
__init__.py feat(derivation): define semantic-ledger candidate source boundary 2026-06-10 13:17:53 -07:00
accumulate.py feat(derivation): define semantic-ledger candidate source boundary 2026-06-10 13:17:53 -07:00
clauses.py
comparatives.py
compose.py feat(adr-0178-gb3a): clause-scoped referent guard — refuse cross-clause aggregation (#456) 2026-05-29 09:15:52 -07:00
extract.py feat(derivation): Workstream A inc 1 — lexeme reader + cleaned exemplars + proxy rebaseline (#796) 2026-06-16 21:57:05 -07:00
goal_residual.py feat(r4): flip cv-0005 to serving — train_sample 6/44/0 -> 7/43/0 (ADR-0207 §5 step 2) 2026-06-03 22:20:12 -07:00
model.py
multistep.py feat(adr-0177-cp2a): cue-precision ledger training + measurement (+ unit hygiene) (#461) 2026-05-29 10:21:58 -07:00
pool.py feat(derivation): define semantic-ledger candidate source boundary 2026-06-10 13:17:53 -07:00
product_bridge.py feat(adr-0195): GSM8K product promotion bridge — serving 4/46/0 → 6/44/0, wrong=0 (#500) 2026-05-30 17:33:56 -07:00
r1_reconstruction.py feat(derivation): define semantic-ledger candidate source boundary 2026-06-10 13:17:53 -07:00
search.py feat(adr-0182): cross-composer disagreement pooling — distractor 0014 + disguised-polarity refuse (confuser wrong 5->2) (#476) 2026-05-29 13:22:19 -07:00
target.py feat(adr-0182): prior-state question guard — temporal-scope confuser refuses (confuser wrong 2->1, pair-tells ->0) (#480) 2026-05-29 14:22:28 -07:00
verify.py feat(derivation): math-serving reach seam — select_self_verified is policy-aware (ADR-0206 §5) 2026-06-06 15:46:14 -07:00