core/docs/handoffs/CLEANUP-C2-run-lane-migration.md

C2 — Wire run_lane to _score_one_candidate_graph

Classification: Unfinished migration (audit finding C2)
Risk: Medium — must verify baseline comparison files still reproduce
Scope: evals/gsm8k_math/runner.py only

---

Problem

run_lane (line 449) calls _score_one, the legacy regex path (generate/math_parser.py → MathProblemGraph → solve). Only the per-lane override in train_sample/v1/runner.py:33 uses _score_one_candidate_graph. Any caller that goes through run_lane — including the public/, dev/, and holdout lanes — is evaluated on the legacy parser, not the candidate-graph architecture we've been measuring.

Impact: lift numbers on train_sample/v1 are not comparable to what run_lane would report on the other slices. The three-slice concern from the audit is real.

---

What _score_one_candidate_graph does differently

_score_one_candidate_graph (line 241) calls parse_and_solve from generate/math_candidate_graph, which layers:

1. Whole-problem comprehension reader (ADR-0164 Phase 2, flag-gated)
2. Per-statement recognizer candidate graph (ME-1..ME-5 matchers)
3. Question-sentence hybrid reader (ADR-0164 Phase 1)
4. Injector dispatch (ADR-0170 + recognizer_anchor_inject.py)
5. Cartesian-product branch enumeration + admissibility gate
6. Existing solver + verifier (same as _score_one)

_score_one calls the regex parser directly, skipping steps 1–5.

---

Migration plan

Step 1 — Baseline snapshot (before changing run_lane)

# Capture current run_lane output on all slices that have unsealed cases
uv run python -m evals.gsm8k_math.runner --lane public  > /tmp/pre_public.json
uv run python -m evals.gsm8k_math.runner --lane dev     > /tmp/pre_dev.json
# Do NOT touch holdout; it is sealed.

Step 2 — Wire run_lane

In evals/gsm8k_math/runner.py line 449, replace:

outcomes = [_score_one(c) for c in cases]

with:

outcomes = [_score_one_candidate_graph(c) for c in cases]

Add a --legacy-parser CLI flag that restores _score_one for the comparison baseline file. The flag is only needed to reproduce baselines/frontier.json and baselines/comparison_v1.json, both of which were captured under the legacy path.

# In run_lane signature:
def run_lane(
    cases: list[dict[str, Any]],
    *,
    config: Any = None,
    legacy_parser: bool = False,
) -> LaneReport:
    score_fn = _score_one if legacy_parser else _score_one_candidate_graph
    outcomes = [score_fn(c) for c in cases]

Step 3 — Verify wrong == 0 on public/dev

uv run python -m evals.gsm8k_math.runner --lane public  > /tmp/post_public.json
uv run python -m evals.gsm8k_math.runner --lane dev     > /tmp/post_dev.json
# Assert wrong == 0 in both outputs

Step 4 — Update baselines if counts differ

If correct counts differ (expected — the new path refuses differently), update baselines/frontier.json and baselines/comparison_v1.json by re-running with --legacy-parser so the files still reflect legacy baseline numbers:

uv run python -m evals.gsm8k_math.runner --lane public --legacy-parser \
    > evals/gsm8k_math/baselines/frontier.json

Step 5 — Run smoke + packs suites

uv run core test --suite smoke -q
uv run core test --suite packs -q

---

Invariant gates

• wrong == 0 must hold on public + dev after the migration.
• _score_one_candidate_graph already enforces this by construction (same verifier as _score_one, additive reader, refusal-preferring injector).
• The baseline snapshots captured in Step 1 give you a before/after diff to explain any count change in a PR description.

---

Relation to other findings

• C4 (compositions compile): C4 should land first or simultaneously so that run_lane immediately benefits from the registry being non-empty.
• C5 (reader zero-delta): confirmed that the reader is inert until statement-level injector gaps close. C2 does not depend on C5.