7 commits

Author	SHA1	Message	Date
Shay	d34883b61a	feat(evals): implement safe metadata and fail-closed evaluator check for GSM1K	2026-06-23 07:52:32 -07:00
Shay	db081fa5b3	feat(evals): add GSM1K local audit adapter	2026-06-23 07:40:53 -07:00
Shay	f823eb437b	feat(evals): add generalization audit runner skeleton	2026-06-23 07:23:43 -07:00
Shay	ed8c0b0e43	feat(evals): verify local generalization benchmark cache	2026-06-23 07:03:56 -07:00
Shay	83ba9a2e9b	feat(evals): implement manifest schema and policy linter (#886)	2026-06-23 06:30:12 -07:00
Shay	67f9b63916	fix(evals): align benchmark manifests and gitignore with sealed-holdout policy ... - Adds benchmark cache ignore rule to extend ADR-0119.7 discipline - Removes sealed slice references from manifest smoke fixture lines - Clarifies openbookqa and para_mawps license gates	2026-06-23 06:12:50 -07:00
Shay	34dc0d64c5	feat(evals): add generalization benchmark manifests + policy (PR-1 of 2) ... Adds policy document and sealed manifest records for the first 8 external audit datasets. No data is vendored. Local cache paths are gitignored. Fetch/verify scripts and smoke fixtures come in PR-2. Datasets: GSM1K, ASDiv, SVAMP, PARA-MAWPS, ARC-Easy, ARC-Challenge, OpenBookQA, CLUTRR	2026-06-23 05:59:36 -07:00