core-labs/core
core-labs/core
1
0
Fork 0
Code Issues 1 Pull requests 1 Projects Releases Packages Wiki Activity

docs: reconcile train sample README metric

Browse source
This commit is contained in:
Shay 2026-06-03 22:42:35 -07:00
parent e7392f118e
commit 7dc9dbaa6a
1 changed files with 13 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

18
evals/gsm8k_math/train_sample/v1/README.md
View file

@ -52,20 +52,28 @@ Each case in `cases.jsonl` preserves:
## Current Report ## Current Report
`report.json` currently records **6 correct / 44 refused / 0 wrong**. The `report.json` currently records **7 correct / 43 refused / 0 wrong**. The
ADR-0126 exit criterion remains unmet (`correct >= 10`, `wrong == 0`), so the ADR-0126 exit criterion remains unmet (`correct >= 10`, `wrong == 0`), so the
runner still exits nonzero. runner still exits nonzero.
The two most recent lifts are ADR-0195 product-promotion cases: The current correct set is:
- `gsm8k-train-sample-v1-0003` — complete revenue product - `gsm8k-train-sample-v1-0003` — complete revenue product
(`48 boxes x 24 erasers x $0.75`). (`48 boxes x 24 erasers x $0.75`).
- `gsm8k-train-sample-v1-0014` — fast-path serving case.
- `gsm8k-train-sample-v1-0018` — fast-path serving case.
- `gsm8k-train-sample-v1-0021` — complete total-work product - `gsm8k-train-sample-v1-0021` — complete total-work product
(`15 pounds x 10 reps x 3 sets`). (`15 pounds x 10 reps x 3 sets`).
- `gsm8k-train-sample-v1-0024` — comparative product control retained green.
- `gsm8k-train-sample-v1-0037` — ADR-0207 R4 goal-residual production
(`goal - Σprogress`) promoted through `generate/derivation/goal_residual.py`.
- `gsm8k-train-sample-v1-0042` — fast-path serving case.
Both are exposed through a guarded product bridge over the pooled derivation The product cases are exposed through a guarded product bridge over the pooled
reader. The bridge does **not** promote the pooled reader wholesale; the eight derivation reader. The R4 case is exposed through the guarded goal-residual
known pooled-reader wrong commits remain refused. bridge. Neither bridge promotes the pooled/general composer wholesale; known
wrong commits remain refused, and the no-reference `<N> times` completeness
hazard remains a hard refusal control.
## ADR-0164 Reader — Zero-Delta Diagnosis ## ADR-0164 Reader — Zero-Delta Diagnosis