Commit graph

2 commits

Author SHA1 Message Date
Shay
7451e7cd74
feat(adr-0177-cp2a): cue-precision ledger training + measurement (+ unit hygiene) (#461)
CP-2a populates the CP-1 ledger from gold-labelled candidate readings and reports
per-pattern reliability — the measurement the cue-precision thesis rests on. Plus
the function-word unit filter, whose value this measurement makes concrete (clean
unit_shape labelling).

What landed (all sealed; serving 3/47/0 byte-identical):
- generate/cue_precision/trainer.py — train_from_cases(cases, enumerators): folds
  gold-labelled candidate chains into the ledger via record_case. Decoupled (the
  candidate enumerators are injected, so the package still imports nothing from
  search). candidates_for dedupes a reading shared by two enumerators.
- generate/derivation/multistep.py — extracted the enumeration half of search_chain
  into public candidate_chains(problem_text); search_chain now delegates (verified
  byte-identical: ms3 tests + practice counts unchanged). CP-2 needs the readings
  the search weighs, not just the one it resolves.
- generate/derivation/extract.py — function-word unit filter (_NON_UNIT_WORDS):
  blanks spurious function-word units ($0.75 each -> "", 3/4 of -> "") that
  corrupt same-unit detection and unit_shape. Closed lexeme set, ADR-0165-safe.
- evals/gsm8k_math/practice/v1/cue_precision_report.py — trains over 200 sealed
  cases (50 train_sample + 150 ADR-0163-F additive) with the real enumerators and
  prints the per-pattern reliability table.
- tests/test_adr_0177_cp2a_training.py — trainer obligations (credit/dedupe/
  determinism/empty) via synthetic enumerators; real-measurement well-formedness;
  search_chain parity.

Load-bearing finding (recorded in ADR-0177): over 200 cases EVERY (cue,op,unit_shape)
pattern floors at ~0.0 reliability (best: for-multiply-cross_unit 0.0116 at 2/34).
The blunt product/sum-of-all readings are almost always wrong vs gold, so the
conservative floor correctly trusts nothing. => CP-2b (trust reliable cues) is
blocked on candidate GENERATION, not the ledger: candidate readings must get less
crude (clause/referent structure, ADR-0178 GB-3b) before any cue earns reliability.
Cue-precision and compositional structure are coupled; structure comes first.

Verification: 107 targeted tests green (CP-2a/CP-1/extract/ms3/GB-1/2/3/MS-1/2) +
architectural invariants; serving CLAIMS.md sha unchanged; practice 4/1/45 and
0/1/149 unchanged. Inert: trains/reports only, consulted by no search/gate.
2026-05-29 10:21:58 -07:00
Shay
1f559344ca
feat(adr-0177-cp1): cue-precision reliability ledger substrate (inert) (#458)
CP-1 of ADR-0177: the per-(cue, op, unit_shape) reliability ledger + credit
assignment mechanism. Mirrors the ADR-0175 per-class ledger discipline
(core/reliability_gate/ledger.py): counts-only integers, reliability via the
pinned conservative_floor, refusals never counted as commitments.

- generate/cue_precision/ledger.py
  - CuePattern: (cue, op, unit_shape) key; op in VALID_OPS, unit_shape closed-set.
  - pattern_for_step / patterns_in_chain: per-step extraction. unit_shape compares
    the operand unit to the model's running (primary/start) unit; a dimensionless
    comparative scalar scales within the dimension -> same_unit.
  - PatternTally: counts-only (correct/wrong, no refused axis); reliability =
    conservative_floor(correct, committed); 0.0 while cold/below N_MIN.
  - CuePrecisionLedger: immutable pattern->tally map (canonical sorted tuple);
    record_chain / record_case credit candidate chains by gold label, independent
    of whether the search resolved or refused.

Inert substrate: not wired into the gate, any scorer, or the search (CP-2/CP-3).
Imported by nothing outside its own tests (asserted by a source-tree scan).

Tests (tests/test_adr_0177_cp1_ledger.py, 27 passing): pattern validation;
unit_shape classification; cold ledger -> 0 reliability; credit assignment;
refusals-not-counted; reliability earned by volume; determinism/replay;
immutability; inertness scan. Smoke suite green (67 passed).
2026-05-29 09:38:51 -07:00