core/scripts
Shay f9e9cc0c6a fix(evals): the deduction lane hashes the prose it serves
The pinned artifact carried verdict counts only. The runner's docstring
justified that: prose is "presentation, not decision", so the pinned bytes
"stay stable against wording-only changes; wording is covered by
tests/test_deduction_surface.py."

Measured, that justification was false. With render._display_noun sabotaged
so every clause reads "all SABOTAGE_dogs are SABOTAGE_animals":

  11 lane SHA pins                     -> 11/11 byte-identical, blind
  test_deduction_serve_lane + _license -> 20 passed, blind
  tests/test_deduction_surface.py      -> 41 passed, blind
                                          (the named wording guard)
  evals/grammar_roundtrip              -> RED, the only witness

So CORE's user-visible output was unguarded by its own hash pins, which is
how the ratified v1b band served "all dog are mammal" for the entire arc
with wrong=0 intact.

build_report now emits surface_sha256 + per-case surfaces from the real
deduction_grounded_surface — the same call chat serving makes, so what is
hashed is what a user reads. Note build_combined_report re-projects five
named fields per split, so a field added to build_report alone never reaches
the pinned bytes; both had to change. The payload is not the report.

Surfaces are recorded, not just digested, so a moved pin shows the exact
sentence that changed in review instead of an opaque hash to go re-derive.
2,766 -> 37,280 bytes.

  deduction_serve_v1 under sabotage: BEFORE byte-identical (blind)
                                     AFTER  52370b73 vs c855d55c (RED)

Re-pinned surgically, one line, old hash recorded beside it, never --update.
Verdicts untouched: 166/166 correct, wrong=0. The hash moved because the
payload grew.

test_surface_hash_moves_when_the_renderer_is_sabotaged makes it permanent:
it corrupts the renderer, requires a digest to move, AND asserts the
aggregate counts are unchanged — proving the digest tracks PROSE rather than
smuggling in a decision change. A pin that cannot fail guards nothing.

Accepted cost: wording-only changes now move this pin. That is the intent —
a wording change IS a user-visible change and should require a deliberate
re-pin. The other 10 lanes are untouched; deduction_serve was fixed because
it is the one demonstrably serving prose to users.

[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
smoke 621 unchanged; deductive 405 (403 + 2 new);
scripts/verify_lane_shas.py 11/11 with the new pin, and 10/11 (RED on
deduction_serve_v1) under the sabotage it previously could not see.
2026-07-26 19:45:48 -07:00
..
benchmarks
ci ci(local): one-command local-first runner; fix a lane roster red on main 2026-07-25 12:51:17 +00:00
hooks feat(provenance,teaching): close the lateral gaps the assessment actually found 2026-07-25 04:51:15 +00:00
ops feat(telemetry): discovery-yield-per-served-turn baseline (ADR-0255) 2026-07-23 07:27:00 -07:00
probes
__init__.py
agent_startup.sh
build_refusal_taxonomy_cases.py
calibrate_ratification_threshold.py
dump-api-schemas.py
dump-enums.py
dump-schemas.py
generate_claims.py feat(generalization): curriculum-grounded serving — exams answered from ratified curriculum (ADR-0262) 2026-07-24 14:38:08 -07:00
generate_discourse_paragraph.py
generate_en_numerics_v1.py Lane 4: Registry Consolidation (language_packs to packs) 2026-07-04 15:11:28 -07:00
generate_english_fluency_ood.py
generate_grammatical_v2.py
generate_gsm8k_public_corpus.py feat(evals): Author 200-case GSM8K math evaluation corpus and verification script 2026-07-04 16:03:51 -07:00
generate_identity_curriculum.py
generate_identity_test_cases.py
generate_monotonic_cases.py
generate_phase5_domain_lanes.py
generate_phase5_language_lanes.py Lane 4: Registry Consolidation (language_packs to packs) 2026-07-04 15:11:28 -07:00
gsm8k_experience_flywheel.py
gsm8k_frontier_report.py
gsm8k_microscope.py
gsm8k_post_gate_a1_frontier_microscope.py
gsm8k_problem_frame_adequacy.py
gsm8k_sealed_attempt_scout.py
gsm8k_substrate_morphology.py Lane 4: Registry Consolidation (language_packs to packs) 2026-07-04 15:11:28 -07:00
measure_grammar_seam.py refactor(generate): one owner per linguistic fact (Phase 2A) 2026-07-26 17:45:27 -07:00
measure_trackb_inc2.py fix(trackb): pure-S1 extract refuse multi-clause overmatch (0361 wrong=0) 2026-07-19 20:04:27 -07:00
measure_trackb_s1_slice.py feat(trackb): S1 symbolic structure-mapping vertical slice (off-serving) 2026-07-19 19:01:23 -07:00
package_apple_uma_demo.py
publish_pack_measurements.py
ratify_anchor_lens_packs.py Lane 4: Registry Consolidation (language_packs to packs) 2026-07-04 15:11:28 -07:00
ratify_ethics_pack.py
ratify_identity_packs.py
ratify_register_packs.py
ratify_rhetorical_style_packs.py
ratify_safety_pack.py
review_trace.py
run_examples.py
run_pulse.py Lane 4: Registry Consolidation (language_packs to packs) 2026-07-04 15:11:28 -07:00
scaffold_phase5_domain_lanes.py
score_phase5_holdouts.py
seal_gsm8k_test.py
seal_holdouts.py
setup_codex_environment.sh
start-codex-session.zsh
start-grok-session.zsh
test_ffi_layout.py
verify_definitional_closure.py Lane 4: Registry Consolidation (language_packs to packs) 2026-07-04 15:11:28 -07:00
verify_lane_shas.py fix(evals): the deduction lane hashes the prose it serves 2026-07-26 19:45:48 -07:00
verify_semantic_equivalence.py
workbench