Commit graph

2 commits

Author SHA1 Message Date
Shay
c13d7e14c4 feat(ADR-0127/0128 integration): pack-aware parser + Path-B trigger evidence
Integrates en_units_v1 (#164) + en_numerics_v1 (#163) into the
ADR-0126 candidate-graph parser. Loader merge (re-exports from
numerics_loader.py give single import path), pack-aware unit
canonicalization (handles irregular plurals like feet/children
via lookup_unit), indefinite-quantifier refusal (ADR-0128.4 —
'some'/'many' emit no candidates, preserving wrong==0), and
widened initial-possession shapes:
  - <Entity> has N <unit> [of <substance>]  (ADR-0127 substance qualifier)
  - There are N <unit> [in <place>]         (implicit-subject shape)

Plus: pack-backed cardinal grounding in math_roundtrip._value_grounds
(widens word-number coverage from hard-coded 0-12 to full numerics
pack cardinal table + compound rule). Op-pattern trailing prep
alternation gains of/for/with for substance qualifiers.

REGRESSION: 1050/1050 tests green across math + ADR-0126 + ADR-0127
ratification + ADR-0128 ratification + runner.

EMPIRICAL RESULT (the Path-B trigger ADR-0126/0127/0128 named):
  correct =  0/50  wrong =  0/50  refused = 50/50
  on evals/gsm8k_math/train_sample/v1/cases.jsonl

Per ADR-0127's exit criterion (correct >= 10/50, wrong == 0):
**MISSED** — the full deterministic design (candidate-graph
topology + units pack + numerics pack + pack-aware parser) does
not move the GSM8K-math lane. This is the real Path-B trigger.

WHAT WORKS (synthetic verification, 6/6 cases solve end-to-end):
  - 'Jan has 5 apples. Jan buys 3 apples. ...' -> 8
  - 'Sam has 10 feet of rope. Sam uses 3 feet of rope. ...' -> 7
  - 'There are 5 kids in camp. ...' -> 5
  - 'Sam has 10 children. Sam loses 2 children. ...' -> 8
  - (money + time-dimension variants pass)

WHY GSM8K STAYS AT ZERO: real GSM8K problems carry compound
linguistic structure (pronouns across statements, possessives,
subordinate clauses, multi-word entities, multi-step inference)
that no amount of pack vocabulary addresses. Per-sentence parse
rate improved measurably on simple shapes; joint problem-level
pass rate stayed at zero because every real problem contains at
least one sentence the parser still cannot handle.

Full results + Path-B recommendation in
docs/decisions/ADR-0127-0128-RESULTS.md. The substrate
(architecture + packs) stays load-bearing in main; the math
expert promotion path retargets to a benchmark where exact
recall and determinism are the discriminators (proposed
ADR-0131).
2026-05-23 07:41:50 -07:00
Shay
661d67002e feat(ADR-0126 P1): round-trip admissibility primitive + 26 tests
The wrong-answer firewall for the candidate-graph parser topology.

A CandidateOperation carries an Operation plus source-span provenance
for every content slot the parser claimed (verb, value, unit, actor,
transfer target, comparison reference). roundtrip_admissible() checks
each slot grounds in the source span AND the matched verb is
registered for the claimed kind.

Two consequences:
- A regex that mis-reads 'loses' as add fails (loses not in ADD_VERBS).
- A regex that hallucinates a number/unit not in source fails to ground.

KIND_TO_VERBS is the new single source of truth for {kind -> verbs};
P2 will refactor math_parser to consume it. Verb tables are
permissive by design (much wider than current narrow regex tables)
because the filter rejects wrong candidates downstream — narrowness
is no longer the safety mechanism.

Deterministic: pure byte/regex containment. No randomness, no
learning, no approximation. Preserves wrong==0, trace_hash byte-
equality, replay determinism.
2026-05-23 06:36:13 -07:00