core/scripts
Shay 25580e18b0
fix(adr-0191): candidate-graph completeness guard — real-corpus wrong 5→0 (#496)
* fix(adr-0191): candidate-graph completeness guard — real-corpus wrong 5->0

The candidate-graph reader (serving) checked grounding + round-trip but had
no completeness obligation, so problems whose later clauses failed to parse
emitted a partial reading. Over the full 7,473-question real GSM8K train
split this confabulated 5 answers (wrong!=0) the 47-case train_sample hid;
2 were regressions from #488.

Add the missing admissibility leg (mirrors the derivation reader's verify.py):
every source quantity (all statements + question) must be consumed by the
chosen reading, else refuse. Refusal-only -> cannot create a wrong answer.
Number-sense is pack-authoritative (en_numerics_v1 parse_compound_cardinal +
lookup_multiplier + all 6 currency symbols) so it never disagrees with the
engine; aggregating initials expose consumed_value_tokens provenance.

Evidence: real-corpus wrong 5->0, correct held at 4; train_sample byte-
identical 4/46/0; G1-G5+S1+G3.1 green; smoke 67 passed; math_teaching_corpus
lane byte-identical.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(adr-0191): committed full-corpus GSM8K microscope (standing wrong=0 + coverage instrument)

Promotes the throwaway tmp/ microscope that found the 5 confabulations into a
committed tool. Runs the canonical serving reader over any GSM8K corpus and
reports failures-first: correct/wrong/refused, every wrong answer by name,
refusal families, and the no-injection per-category coverage map that ranks
which injector to build next by real frequency.

Default corpus is the committed 47-case train_sample (always available);
--corpus path runs the full real split. This is the ADR-0191 follow-up: re-run
after every capability PR, not just train_sample — a flip is only real if it
does not widen the confabulation surface.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 15:45:07 -07:00
..
__init__.py
build_refusal_taxonomy_cases.py
calibrate_ratification_threshold.py
dump-api-schemas.py
dump-enums.py
generate_claims.py
generate_discourse_paragraph.py
generate_en_numerics_v1.py
generate_english_fluency_ood.py
generate_grammatical_v2.py
generate_identity_curriculum.py
generate_identity_test_cases.py
generate_monotonic_cases.py
generate_phase5_domain_lanes.py
generate_phase5_language_lanes.py
gsm8k_microscope.py
publish_pack_measurements.py
ratify_anchor_lens_packs.py
ratify_ethics_pack.py
ratify_identity_packs.py
ratify_register_packs.py
ratify_rhetorical_style_packs.py
ratify_safety_pack.py
review_trace.py
run_examples.py
run_pulse.py
scaffold_phase5_domain_lanes.py
score_phase5_holdouts.py
seal_gsm8k_test.py
seal_holdouts.py
verify_definitional_closure.py
verify_lane_shas.py