Commit graph

4 commits

Author SHA1 Message Date
Shay
5a0423cb36
feat(derivation): Gate A2b case 0002 fractional rest composition (#810)
* feat(derivation): Gate A2b case 0002 fractional rest composition

Add fraction_portion operation for "gives N/M of that" and "half of the
rest" subtract semantics chained after unit_partition, plus keep-on-hand
question binding with partition-noun unit inference. Live train_sample
moves 6/44/0 to 7/43/0 with wrong=0 preserved; confuser-v1-0007 still
refuses without "of that".

* test(derivation): add A2b sibling and confuser anti-overfit cases

Cover Bob/Alice partition+fraction chains, slash-without-referent refusal,
partition-less "of that", and multi-actor pronoun ambiguity.

* chore(derivation): normalize A2b fraction test EOF
2026-06-17 19:08:24 -07:00
Shay
ec1dcf6e78 feat(realizer): ADR-0123 comparison-phrasing surface (closes substrate)
ADR-0123-parser-comparison-phrasing as the **surface increment** on
PR #155's substrate (commit c9bd5d4). Closes the last architectural
gap in the comparison-phrasing class: before this commit, the
substrate's solver evaluated comparison problems successfully but
realize() crashed with `unknown operation_kind 'compare_additive'`
when asked for show-your-work prose.

Substrate (PR #155) already shipped:
- `Comparison` typed graph operand
- `compare_additive` / `compare_multiplicative` operation kinds
- parser patterns for the four canonical surfaces
  (N more / N fewer / twice / N times / half)
- solver + verifier wiring + pack lemmas
  (en-arith-006 compare_additive, en-arith-007 compare_multiplicative)

This surface adds:
- `_compare_additive_sentence(step)` rendering `direction='more'|'fewer'`
- `_compare_multiplicative_sentence(step, entity_units)` rendering
  `direction='times'|'fraction'`
- two new branches in `_step_sentence` dispatch
- `_step_sentence` signature widened with optional `entity_units` map
  (derived once-per-trace in `realize()` from `graph_initial_state`)
- ADR-0123-parser-comparison-phrasing.md (~15 invariants, substrate
  + surface decomposition rationale, multi-construction barrier
  inheritance)
- 26 invariants pinned across canonical surfaces, plurality
  independence, byte-determinism, refusal discipline, and
  backwards-compatibility with the pre-comparison realizer templates

End-to-end pipeline now operates on all four canonical comparison
shapes:

  parse_problem(
    "Alice has 5 apples. Bob has 3 more apples than Alice. "
    "How many apples does Bob have?"
  ) -> solve() -> realize().as_prose() ->
  "Alice has 5 apples. Bob has 3 more apples than Alice, giving Bob
   a total of 8 apples. Bob has 8 apples."

Measurement (this PR):
- 26/28 direct ADR-0123 tests pass; 2 skipped (CORE_HOLDOUT_KEY)
- `core eval cognition` byte-identical: 100/100/100/100
- ADR-0118 stepped-realizer templates re-render byte-identically
- Substrate measurements continue to hold

Honest non-result: sealed `correct_rate` stays at 0/1319. The
realizer cannot create matches the parser refuses; the multi-
construction barrier the substrate ADR documented holds at the
surface layer too. Cumulative lift signal expected only after the
3rd/4th foundational class lands (per ADR-0121's revised
sequencing). `wrong == 0` holds by construction — realizer only
renders successful traces.

Pre-existing failure noted (not introduced by this PR):
`tests/test_adr_0085_gloss_aware_cause.py::test_flag_off_metrics_byte_identical`
fails on substrate base (c9bd5d4) without these changes — an
ADR-0085 cognition baseline drift unrelated to the realizer.
2026-05-23 02:03:49 -07:00
Shay
6582df3bae feat(parser): ADR-0122 rate/per-unit grammar (substrate-only; lift deferred)
First parser-expansion ADR after ADR-0121's deferral. Adds the rate
algebra substrate (Rate dataclass + apply_rate operation kind + parser
pattern + solver/verifier/realizer + en_arithmetic_v1 pack lemma)
mirroring the deferral pattern that ADR-0121 demonstrated for
capability promotion: substrate complete, gate refuses honestly.

Substrate
- Rate(value, numerator_unit, denominator_unit) frozen dataclass with
  strict positive-value + non-empty-distinct-unit refusal at construction
- apply_rate operation kind admitted in VALID_OPERATION_KINDS;
  Operation.operand widened to Quantity | Rate with kind-discriminated
  type enforcement
- Parser: _RATE_COST_EACH_RE + _RATE_COST_EACH_TRAILING_RE +
  _Q_RATE_AGGREGATE_RE patterns; actor_units state tracking;
  first-declaration-wins on redeclaration (ParseError); orphan-rate
  refusal at end of parse; three refusal paths in rate-aggregate question
- Solver: _apply_rate() reads denominator-unit state, multiplies by
  rate.value, writes numerator-unit state (denom preserved)
- Verifier: _verify_apply_rate_step() byte-equal replay
- Realizer: 'At {N} {numer} per {denom_singular}, {actor} spends ...'
  template containing required tokens
- Pack: en-arith-006 apply_rate lemma + gloss; SHA-256 checksums
  refreshed; manifest version 1.0.0 -> 1.1.0; provenance tagged
  adr-0122:rate_extension:2026-05-22

Measurement (honest)
- Sealed GSM8K correct_rate: 0/1319 (substrate matches zero real cases
  alone). Multi-construction barrier documented in the ADR: all 14 sealed
  cases matching 'each \w+ costs?' combine rate with at least one other
  class (aggregation 6, comparison 3, unit conversion 2, multi-actor 2,
  conditional 1)
- Sealed GSM8K wrong: 0 (load-bearing positive claim; grammar adds zero
  misparses across 1,319 real test problems)
- Anti-overfit lanes unchanged: OOD ratio, perturbation invariance
  preserving/breaking 1.0, adversarial wrong 0
- ADR-0121 invariants byte-equal preserved (6/6)
- 41 new ADR-0122 invariants pinned in tests/test_adr_0122_rate_per_unit.py
- 670 existing math + pack regression tests pass

Roadmap update
- Per-ADR lift expectation corrected: no single parser-expansion ADR
  will move sealed correct_rate alone. First lift signal will come
  from cumulative composition after 3rd or 4th class lands (rate +
  comparison + aggregation foundational set)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 21:24:28 -07:00
Shay
7ad3f72cb4 feat: ADR-0118 — stepped realizer (SolutionTrace → show-your-work prose)
Phase 4 of the ADR-0114 GSM8K-math roadmap. Consumes a SolutionTrace
and emits one sentence per step plus setup + answer sentences. Pure
function; same trace → byte-equal RealizedTrace.

What landed

generate/math_realizer.py
  - realize(initial_state, trace) -> RealizedTrace
  - Frozen RealizedTrace dataclass with canonical_bytes() + as_prose()
  - Per-kind sentence rules (add / subtract / transfer / multiply×2 /
    multiply×3 / multiply-general / divide)
  - Singular/plural surface rule matches parser canonicalization
  - Typed RealizerError on unrecognized step kinds

tests/test_math_realizer.py — 60 cases pinning five invariants:
  1. All 50 dev-set cases realize without error
  2. Determinism: byte-equal RealizedTrace across two calls
  3. Setup sentence count == initial_state count
  4. Step sentence count == operation count
  5. Answer sentence contains the resolved value + unit

ADR-0114a obligation discharge update

ADR-0118 hardens determinism (#9) across a third layer (realizer)
and makes #3 / #10 human-inspectable via the prose surface. No
obligation is directly newly discharged by ADR-0118; it's substrate
for ADR-0119 GSM8K eval lane.

Round-trippability of the prose through the parser is explicitly
out of scope for this phase. The trace is the verifiable artifact
(ADR-0117); the prose is human-readable documentation.

Tests: 60 new realizer cases; 546 total green across realizer +
parser + solver + verifier + OOD; 67/67 smoke green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:11:10 -07:00