First Phase of ADR-0114's expert-capability roadmap. Decomposed into four
sub-phases so each lands as its own auditable step:
1.1 schema + 5 seed cases + invariants ← this commit
1.2 45 more dev-set cases ← delegated (Codex)
1.3 the parser itself ← exit: ≥0.90 on dev set
1.4 runtime binding ← if non-trivial
What landed
- generate/math_problem_graph.py — typed dataclasses (Quantity,
InitialPossession, Operation, Unknown, MathProblemGraph) + frozen
validation + canonical_bytes() byte-deterministic serialization +
graph_from_dict roundtrip.
- evals/gsm8k_parser_dev/cases.jsonl — 5 seed cases (gpd-001..005)
covering single-add, single-subtract, multi-step, two-entity
transfer, and multi-entity sum constructions. Every case carries a
ground_truth_graph and the documented patterns it exercises.
- evals/gsm8k_parser_dev/README.md — authoring contract: schema,
pattern registry, canonicalization rules, Phase 1.1 scope boundary,
hand-solving rubric, distribution target for the remaining 45
cases. This is the spec Phase 1.2 authors work against.
- tests/test_math_problem_graph.py — 26 cases pinning four invariants:
round-trip byte equality, canonical_bytes() determinism, schema
rejection of malformed graphs, and ground_truth_graph ↔
expected_answer agreement (a hand-solver inside the test module
falsifies mis-authored cases).
Why this is sticky
The Phase 1.1 schema is load-bearing for Phase 1.2 (the 45 authored
cases will be written against it) AND Phase 1.3 (the parser will be
graded byte-equal against ground-truth graphs in this schema). Changing
the schema after Phase 1.2 lands requires an amendment ADR + rewriting
authored cases. The schema choices here are intentionally conservative.
Tests: 26/26 new; 67/67 smoke green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
57b257ca1d