core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
core/evals/gsm8k_math/train_sample
History
Shay 9d19b8176f feat(gsm8k): ADR-0126 P6 — train-sample runner + exit-criterion gate 
Wraps existing math pipeline (parser -> solver -> verifier) against
PR #159's 50-case train sample. Emits deterministic report.json with
per-case verdicts. CLI exit code reflects exit criterion
(correct >= 10 AND wrong == 0).

Baseline against current parser: 0 correct / 0 wrong / 50 refused.
This baseline is the inner-loop gradient signal for ADR-0126's
candidate-graph parser (in flight on feat/adr-0126-candidate-graph).

Registers tests/test_adr_0126_train_sample_runner.py under
'core test --suite math' so the wrong == 0 invariant becomes a hard
CI gate per ADR-0114a Obligation #4 (refuse rather than confabulate).

Depends on PR #159 (gemini/adr-0126-train-sample). Rebase onto main
after #159 lands.
2026-05-23 06:33:06 -07:00
..
v1 feat(gsm8k): ADR-0126 P6 — train-sample runner + exit-criterion gate 2026-05-23 06:33:06 -07:00
__init__.py feat(gsm8k): ADR-0126 P6 — train-sample runner + exit-criterion gate 2026-05-23 06:33:06 -07:00