core/evals/deduction_serve
Shay f9e9cc0c6a fix(evals): the deduction lane hashes the prose it serves
The pinned artifact carried verdict counts only. The runner's docstring
justified that: prose is "presentation, not decision", so the pinned bytes
"stay stable against wording-only changes; wording is covered by
tests/test_deduction_surface.py."

Measured, that justification was false. With render._display_noun sabotaged
so every clause reads "all SABOTAGE_dogs are SABOTAGE_animals":

  11 lane SHA pins                     -> 11/11 byte-identical, blind
  test_deduction_serve_lane + _license -> 20 passed, blind
  tests/test_deduction_surface.py      -> 41 passed, blind
                                          (the named wording guard)
  evals/grammar_roundtrip              -> RED, the only witness

So CORE's user-visible output was unguarded by its own hash pins, which is
how the ratified v1b band served "all dog are mammal" for the entire arc
with wrong=0 intact.

build_report now emits surface_sha256 + per-case surfaces from the real
deduction_grounded_surface — the same call chat serving makes, so what is
hashed is what a user reads. Note build_combined_report re-projects five
named fields per split, so a field added to build_report alone never reaches
the pinned bytes; both had to change. The payload is not the report.

Surfaces are recorded, not just digested, so a moved pin shows the exact
sentence that changed in review instead of an opaque hash to go re-derive.
2,766 -> 37,280 bytes.

  deduction_serve_v1 under sabotage: BEFORE byte-identical (blind)
                                     AFTER  52370b73 vs c855d55c (RED)

Re-pinned surgically, one line, old hash recorded beside it, never --update.
Verdicts untouched: 166/166 correct, wrong=0. The hash moved because the
payload grew.

test_surface_hash_moves_when_the_renderer_is_sabotaged makes it permanent:
it corrupts the renderer, requires a digest to move, AND asserts the
aggregate counts are unchanged — proving the digest tracks PROSE rather than
smuggling in a decision change. A pin that cannot fail guards nothing.

Accepted cost: wording-only changes now move this pin. That is the intent —
a wording change IS a user-visible change and should require a deliberate
re-pin. The other 10 lanes are untouched; deduction_serve was fixed because
it is the one demonstrably serving prose to users.

[Verification]: in-worktree on CPython 3.12.13, uv sync --locked —
smoke 621 unchanged; deductive 405 (403 + 2 new);
scripts/verify_lane_shas.py 11/11 with the new pin, and 10/11 (RED on
deduction_serve_v1) under the sabotage it previously could not see.
2026-07-26 19:45:48 -07:00
..
practice fix(deduction-serve): resolve assert_corpus_sound name collision 2026-07-25 16:47:01 -07:00
v1 feat(deduction-serve): Band v2-EN — natural-English argument serving, earned licenses (ADR-0257) 2026-07-23 14:37:50 -07:00
v2_condmem feat(deduction-serve): Band v4-CM — conditional-membership fusion, decided (ADR-0259) 2026-07-24 10:14:00 -07:00
v2_en feat(deduction-serve): Band v3-MEM — member-chain band, Socrates syllogism decided (ADR-0258) 2026-07-23 19:21:41 -07:00
v2_exist feat(deduction-serve): Band v6-EX — existential witnesses, decided (ADR-0261) 2026-07-24 14:14:37 -07:00
v2_member feat(deduction-serve): Band v6-EX — existential witnesses, decided (ADR-0261) 2026-07-24 14:14:37 -07:00
v2_verb feat(deduction-serve): Band v5-VP — verb-predicate arguments, decided (ADR-0260) 2026-07-24 12:51:36 -07:00
__init__.py feat(deduction-serve): Phase 2 — end-to-end eval lane, SHA-pinned wrong=0 gate 2026-07-23 12:36:59 -07:00
contract.md feat(deduction-serve): Band v6-EX — existential witnesses, decided (ADR-0261) 2026-07-24 14:14:37 -07:00
report.json fix(evals): the deduction lane hashes the prose it serves 2026-07-26 19:45:48 -07:00
runner.py fix(evals): the deduction lane hashes the prose it serves 2026-07-26 19:45:48 -07:00