MS-2 of multi-step composition. Extends the derivation model so a chain mixes
text-quantity operands and COMPARATIVE-scalar operands (twice->x2, 'N times'->xN,
half->x0.5), self-verifying the whole chain with completeness over body+question
and question-target matching.
- model.py: Step gains comparative flag.
- comparatives.py: ComparativeScalar gains number_token (the '<N> times' number,
so completeness counts the consumed body quantity); comparative_step(cs) bridges
a scalar into a Step (operand grounded by cue, not a text value token).
- verify.py: self_verifies exempts comparative operands from value-grounding
(clause 1) — they are cue-grounded (clause 2); completeness (Counter) counts a
digit comparative's number_token as consuming the body quantity. Adds target_units
to select_self_verified: a chain whose answer_unit isn't the asked unit is dropped
(question-target match; empty target_units imposes no constraint).
Proves the multi-step shapes from the gold structures: 0024 (text sum then 'three
times' scale -> 438), 0033 father-chain (digit-comparative '7 times' + fixed 'half'
+ text add -> 47). Full 0033 DAG (quantity reuse + the question's 25) deferred.
25 MS-2 tests; full derivation surface 69/69 (3a/3b/comparatives/ms1/ms2); ruff
clean; smoke 67. Not wired into serving (model ready for MS-3 target-guided search).
MS-1 of multi-step composition. Turns the question into a Target = what the
problem asks for, the search's pruning signal + stopping criterion (MS-3).
Lexeme-level only (ADR-0165): the existing question parser returns nothing on
these GSM8K questions, and 0165 forbids new question-shape grammar regex. Three
robust signals:
- quantities: numbers stated IN the question (0033's 'when she is 25') via the
body's lexeme extractor — they participate in the derivation.
- aggregation: presence of an aggregation lexeme (total/altogether/combined/sum/
'in all'/'in total') — soft hint the final step is a sum.
- units: asked units resolved by INTERSECTION with the body's known units
(precise lexeme match, e.g. 'jumping'). Superordinates (weight<->pounds) are
NOT faked — deferred to a curated superordinate-units pack; until then the unit
signal is precise-but-incomplete and the search leans on completeness.
Refuse-preferring: empty target field is not an error, just a weaker prune.
generate/derivation/target.py: Target + extract_target(question, known_units=()).
12 MS-1 tests (question-quantity, aggregation, body-unit intersection,
superordinate-not-faked, determinism, frozen). Verified: derivation suite 57/57;
ruff clean; smoke 67. Not wired into serving (Target ready for MS-2/MS-3).
The curated, irreducible world-fact primitives multi-step composition needs
(ADR-0175 section 10: the engine can't derive 'twice = 2' from arithmetic). The
microscope flagged these via the 0015/0025/0024/0033 wrongs.
language_packs/data/en_core_comparatives_v1/: 9 closed-set multiplicative
comparatives (twice/double/triple/quadruple/half/quarter + inflections) -> scalar
ops. manifest.json with sha256 of the bytes on disk (CLAUDE.md pack rule).
Refusal-preferring: non-terminating/ambiguous comparatives (a third, several)
deliberately excluded; expansion via HITL corridor.
generate/derivation/comparatives.py: extract_comparative_scalars() ->
ComparativeScalar(op, scalar, span, cue). Fixed lexemes + the '<number> times'
pattern (digit or word-number via WORD_NUMBERS). Lexeme-level (ADR-0165);
deterministic (text-order); supplies only the SCALAR primitive — referent
binding is the multi-step search's job (ADR-0176).
14 tests incl. refusal-preferring discipline + pack integrity (manifest checksum
matches bytes on disk). Verified: derivation suite 45/45; ruff clean; smoke 67;
packs 141. Not wired into serving (data + extractor ready for ADR-0176 MS phases).
ADR-0175 Phase 3b — the first live attempt generator. Runs only in the sealed
practice lane, only on cases the engine refused; every proposal is gated by the
Phase 3a self-verification gate.
generate/derivation/:
- extract.py: extract_quantities() — lexeme-level (number + unit word; ADR-0165).
- search.py: search_multiplicative() — one in-clause product candidate per
sentence with >=2 quantities + a present multiplicative cue; gated by
select_self_verified. Per-sentence scope + multi-candidate disagreement give
the uniqueness gate real teeth (two qualifying sentences -> refuse). The cue
set {each,every,for,per,times} is an explicit PROVISIONAL hypothesis the
practice loop refines, not a claimed-correct grammar.
evals/gsm8k_math/practice/v1/search_runner.py: search_augmented_scorer +
build_search_report — base scorer, then a practice-only attempt on refusals.
MEASUREMENT (the deliverable, per the breadth-of-impact test):
practice with search: correct=4 wrong=9 refused=37 (baseline 3/0/47)
- Flips +1 (0021, the clean in-clause aggregate) and its renumbered/reworded
variants (ADR-0114a perturbation guard) -> a real capability, not memorisation.
- 9 wrong attempts -> elimination records (§9), the learning signal. The naive
full-product cue model over-attempts; the eliminations are exactly the signal
that refines it.
HONEST FINDING: self-verification (grounding ∧ cue ∧ unit ∧ uniqueness) is
NECESSARY but NOT SUFFICIENT — 9/13 self-verified attempts were wrong vs gold.
The gap is cue PRECISION / which-quantities-compose (the knowledge axis), not
'can we multiply' (skill). This is why the search runs sealed: gold catches the
9, and case 0050 (canary) attempted-and-failed IN PRACTICE without touching
serving -> validates the seal.
Invariants: #1 seal (serving still 3/47/0; 0050 refuses in serving; no
generate/chat import of the lane), #3 determinism. Serving wrong=0 untouched.
Verified: 3a+3b 31/31; ruff clean; serving lane 4/4; smoke 67/67.
ADR-0175 Phase 3 splits wrong=0-first: build the gate (3a) and PROVE invariant #2
before the bounded search (3b) that could exploit gaps.
generate/derivation/:
- model.py: Quantity / Step / GroundedDerivation. A derivation is a left-fold over
text-sourced quantities; each Step carries its licensing cue (the lexeme the
search claims licenses the op).
- verify.py: self_verifies() — grounded operands ∧ grounded operation cues ∧ unit
consistency ∧ no divide-by-zero. Grounding REUSES the canonical primitives from
math_roundtrip (_tokens/_token_in/_value_grounds) so the gate cannot drift from
the round-trip contract. select_self_verified() adds the uniqueness rule:
unique self-verifying answer resolves; zero or disagreeing refuse (wrong=0).
INVARIANT #2 proven (TestInvariant2_NoSpuriousSelfVerification): the gate refuses
to self-verify a derivation that is not grounded+unit-consistent+unique even when
its value coincides with gold — the 20/5==4 class:
- invented operand not in text -> refused
- operation cue not in text -> refused (division not licensed by any present cue)
- value coincidence (20/5=4) with ungrounded op -> still refused
- add across units (pounds + reps) -> refused
- divide-by-zero -> refused
Plus uniqueness: disagreeing grounded derivations -> refuse; agreeing -> resolve.
Phase 3a is inert (nothing wires generate.derivation into serving). 3b is the
bounded search that produces derivations for this gate + measures the flip-curve
in the practice lane under perturbation.
Verified: 16/16; ruff clean; smoke 67/67; no serving import.
