core/evals/gsm8k_math/train_sample
Shay fde62d713d docs(ADR-0127): units pack + units-aware parser + conversion graph (proposed)
Diagnostic from ADR-0126's first train-sample run (0/0/50): every
refusal happens at the first statement of each problem, and every
refused first statement fails on the unit-of-measurement construction,
not on the operation grammar. Adding more verb regexes is the per-axis
treadmill that produced 4 zero-lift ADRs. Units form a finite, externally
well-defined ontology (NIST SI tables, currency, English container nouns)
that is semantic substrate the candidate-graph parser was designed to
consume.

Scope:
- en_units_v1 pack: dimensions, units (<=60), containers, rate connectors
- conversions.jsonl: directed weighted graph of within-dimension unit pairs
- 3 new initial-possession shapes + rate-declaration extractor in the
  candidate parser
- Round-trip filter gains optional pack-typed-unit check
- Solver gains dimensional canonicalization helper (shortest path through
  conversion graph); fired edges join SolutionTrace.steps for replay
- Pack ratification invariants: round-trip identity, per-dimension
  connectivity, path consistency, canonical unit per dimension

Wire the same train-sample exit criterion as ADR-0126 (correct >=10/50,
wrong==0). If passed -> sealed holdout. If still missed -> Path B
trigger is REAL (full deterministic design with units substrate failed),
demote GSM8K, re-target math expert promotion.

Also commits the empirical evidence: train_sample/v1/runner.py swapped
_score_one -> _score_one_candidate_graph; report.json baseline 0/0/50
confirming the candidate-graph topology refuses cleanly without units
substrate.
2026-05-23 06:42:39 -07:00
..
v1 docs(ADR-0127): units pack + units-aware parser + conversion graph (proposed) 2026-05-23 06:42:39 -07:00
__init__.py feat(gsm8k): ADR-0126 P6 — train-sample runner + exit-criterion gate 2026-05-23 06:33:06 -07:00