core/tests/test_lane_sha_verifier.py
Claude 7bc7131fd9
ci(local): one-command local-first runner; fix a lane roster red on main
Two things, both surfaced by finally being able to run the tree.

## A pre-existing silent red

tests/test_lane_sha_verifier.py::TestExpectedLaneCoverage was RED on clean
main — confirmed by running it in a pristine worktree at origin/main, and the
file is untouched by any of this branch's work. deduction_serve_v1 (ADR-0256)
and curriculum_serve_v1 (ADR-0262) shipped during the generalization arc and
neither was added to EXPECTED_LANES.

The roster's `extra` assertion is a deliberate tripwire: a new lane is supposed
to fail it once so an author acknowledges the addition. Nobody did, and nothing
caught it, because that file is in neither `smoke` nor `deductive` — no local
gate and no CI job runs it. Same silent-red family as the exact-tuple pin S5
found in passing, and as the register-axis e2e tests that sat red for two days.
This commit is the acknowledgement the tripwire was asking for.

That it took running the full tree to find is the argument for the rest of this
commit.

## scripts/ci/local-ci.sh

AGENTS.md says the merge bar IS the local run. There was no single entry point
for it, and on any host without Python 3.12.13 exactly there was no local run
at all: pyproject pins requires-python == "3.12.13", so uv sync --locked fails
outright and every gate becomes unrunnable.

  sh scripts/ci/local-ci.sh --tier smoke|gate|full

`gate` is the three pre-push steps; `full` is the whole tree in parallel. Suite
membership is read from core/cli_test.py::TEST_SUITES through the CLI and never
restated, so this runner cannot drift from the hook the way smoke.yml drifted
from the local list.

Interpreter contract, deliberately fail-closed: the pinned interpreter is
CANONICAL, and the runner refuses on anything else with instructions.
--allow-interpreter-fallback opts into any 3.12.x and stamps every run
NON-CANONICAL. Same discipline as ratified_ledger's missing_ok and the MLX
skip path: a degraded mode is legitimate, a degraded mode reporting itself as
the real thing is not.

Evidence recorded in the script header so the flag is not cargo-culted: the
committed-SHA / trace_hash / content_sha256 / lane-SHA pin set (119 tests) was
run on 3.12.11 and 118 passed, the one failure being the stale roster above,
which fails on 3.12.13 too. That is evidence the exact pin is not load-bearing
for bit-exactness — it is NOT a ruling that it should be relaxed. Relaxing it
is a reproducibility decision and belongs to a human.

## Observed, not fixed

Something in the suite writes a real proposal artifact into
teaching/proposals/derived_close_facts/ instead of a tmp_path. The three test
files that name that sink all override it correctly, so the leak is elsewhere —
likely an idle_tick/contemplation path exercising DEFAULT_SINK. Untracked
residue, removed here, flagged for follow-up. Notable because it is only
visible now that the parents[3] bug is fixed and the sink resolves inside the
repo.

[Verification]: gate tier via the new runner — smoke 236, warmed_session 10,
deductive 285, all passed, correctly stamped NON-CANONICAL. Fail-closed path
verified (exit 3 with instructions). lane_sha_verifier 6 passed. Full tests/
tree in progress at 63% with 0 failures at time of commit.
NON-CANONICAL: Python 3.12.11, not the pinned 3.12.13. Not merge evidence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx
2026-07-25 12:51:17 +00:00

115 lines
4.4 KiB
Python

"""Pin scripts/verify_lane_shas.py's pin-block shape.
The full verify-all suite (which re-runs every lane) is expensive
(~30s). It is exercised in CI by the lane-shas workflow. Locally
``pytest`` covers two cheaper guarantees:
1. Every shipped ADR lane has a pin in ``PINNED_SHAS``.
2. Every ``LaneSpec.runner_module`` actually exists on disk and is
marked as a Python file.
These two together mean no lane silently drops out of CI coverage.
"""
from __future__ import annotations
import importlib.util
import re
import sys
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parent.parent
def _load_verifier_module():
"""Load scripts/verify_lane_shas.py as a module.
The scripts directory is not on the default import path; load
directly so the pins/specs remain authoritative without forcing a
package reshuffle.
"""
path = REPO_ROOT / "scripts" / "verify_lane_shas.py"
spec = importlib.util.spec_from_file_location("_verify_lane_shas", path)
assert spec is not None and spec.loader is not None
module = importlib.util.module_from_spec(spec)
sys.modules["_verify_lane_shas"] = module
spec.loader.exec_module(module)
return module
verifier = _load_verifier_module()
class TestPinBlockShape:
def test_every_lane_spec_has_a_pin(self) -> None:
lane_ids = {spec.lane_id for spec in verifier.LANE_SPECS}
pinned = set(verifier.PINNED_SHAS.keys())
missing = lane_ids - pinned
assert not missing, f"lanes without pinned SHA: {missing}"
def test_no_orphan_pins(self) -> None:
"""A pin without a matching LaneSpec is dead code."""
lane_ids = {spec.lane_id for spec in verifier.LANE_SPECS}
pinned = set(verifier.PINNED_SHAS.keys())
orphans = pinned - lane_ids
assert not orphans, f"orphan pins (no matching LaneSpec): {orphans}"
def test_every_pin_is_64_hex_chars(self) -> None:
for lane_id, sha in verifier.PINNED_SHAS.items():
assert re.fullmatch(r"[0-9a-f]{64}", sha), (
f"pin for {lane_id!r} is not a 64-char hex SHA-256: {sha!r}"
)
def test_every_lane_spec_runner_exists(self) -> None:
for spec in verifier.LANE_SPECS:
assert spec.runner_path.exists(), (
f"lane {spec.lane_id!r} runner not found at {spec.runner_path}"
)
assert spec.runner_path.suffix == ".py"
def test_every_lane_spec_canonical_report_path_under_repo(self) -> None:
for spec in verifier.LANE_SPECS:
assert spec.canonical_report.is_relative_to(REPO_ROOT)
class TestExpectedLaneCoverage:
"""The verifier MUST cover all six ADR-0092..0099 lanes.
Hard-code the canonical lane ids so silently dropping any one
fails this test. ADR-0094 and ADR-0097 are schema/ratification
only — no eval lane — and intentionally absent.
"""
EXPECTED_LANES = frozenset(
{
"reviewer_registry", # ADR-0092
"miner_loop_closure", # ADR-0095
"domain_contract_validation", # ADR-0093
"fabrication_control_summary", # ADR-0096
"demo_composition", # ADR-0098
"public_demo", # ADR-0099
"curriculum_loop_closure",
"math_teaching_corpus_v1", # ADR-0131
"deductive_logic_v1", # ADR-0206 — independent-oracle entailment lane
# Added 2026-07-25. Both lanes shipped during the deduction-serve
# generalization arc and neither was added here, so this roster's
# `extra` tripwire — which exists precisely to make a new lane an
# explicit acknowledgement rather than a silent addition — has been
# RED on clean main ever since. It went unnoticed because this file
# is in neither `smoke` nor `deductive`; nothing local or in CI runs
# it. This entry is the acknowledgement the tripwire was asking for.
"deduction_serve_v1", # ADR-0256
"curriculum_serve_v1", # ADR-0262
}
)
def test_all_expected_lanes_covered(self) -> None:
actual = {spec.lane_id for spec in verifier.LANE_SPECS}
missing = self.EXPECTED_LANES - actual
extra = actual - self.EXPECTED_LANES
assert not missing, f"missing expected lanes: {missing}"
assert not extra, (
f"unexpected extra lanes: {extra} (add to EXPECTED_LANES in test "
"if intentional)"
)