fix(tests): the membership ratchet caught its author — baseline 749 -> 748
PR-6 promoted tests/test_safety_pack.py onto the gate and left it listed in tests/full_only_baseline.txt. test_baseline_has_no_stale_entries failed the very next smoke run, naming the file and the direction. That is the both-directions half of the ratchet doing exactly the work it was written for, roughly an hour after it landed, against the person who wrote it. A baseline that silently keeps entries it no longer needs stops being a measurement and becomes a story about one — so the failure is the feature. Recorded in the pin's own docstring and in G-7, because a mechanism catching its author is better evidence than a sabotage test: the sabotage was contrived, this was not. Baseline: 749 -> 748. The ratchet only turns down. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Wcw2pnMBwyvmNyQg4uPEt4
This commit is contained in:
parent
5f18b7e58f
commit
edf6c2a454
3 changed files with 7 additions and 2 deletions
|
|
@ -70,7 +70,7 @@ Pin 1 exists because the two gates have **already** drifted (see H-12): the loca
|
|||
|
||||
*Pin 1 was already in the repository, in the only direction that protects anything.* `test_cli_smoke_suite_covers_ci_smoke_gate` has pinned `smoke.yml ⊆ TEST_SUITES["smoke"]` throughout (which is why every measurement above finds **CI-only = 0** — a pin, not luck), under a comment reading *"DELIBERATELY ONE-DIRECTIONAL."* The symmetric version was written and reverted the same day in `50fa287d` (2026-07-25), because `AGENTS.md:280` makes GitHub Actions *"billing-locked … dead signals"* and §277 makes the workflows *"secondary observability only."* This entry's "the two gates have already drifted" is therefore a **misreading of a design decision**, and the delta is not a defect to close.
|
||||
|
||||
*Pin 2 shipped, re-specified.* Measured **749 of 877** test files in no curated suite. The mechanism this entry proposes — assign every orphan, or list it with a reason — is hollow at that scale for the same reason the entry itself identifies: glob topic-suites satisfy the assertion while changing nothing about what runs. All four demonstrated incidents were caused by a *newly landed* orphan, never a legacy one, so what shipped is a **ratchet**: `tests/test_suite_membership.py` + `tests/full_only_baseline.txt`, blocking new orphans, enforced in both directions, count pinned. Three sabotages observed red.
|
||||
*Pin 2 shipped, re-specified — and it caught its author within the hour:* PR-6 promoted `tests/test_safety_pack.py` onto the gate and left it in the baseline; the both-directions half failed the very next smoke run, and the count went 749 → 748. *Pin 2 shipped, re-specified.* Measured **749 of 877** test files in no curated suite. The mechanism this entry proposes — assign every orphan, or list it with a reason — is hollow at that scale for the same reason the entry itself identifies: glob topic-suites satisfy the assertion while changing nothing about what runs. All four demonstrated incidents were caused by a *newly landed* orphan, never a legacy one, so what shipped is a **ratchet**: `tests/test_suite_membership.py` + `tests/full_only_baseline.txt`, blocking new orphans, enforced in both directions, count pinned. Three sabotages observed red.
|
||||
|
||||
*One real gap the entry was circling, now fixed:* the parity pin lived in `fast`, which the pre-push gate does not run — **the pin guarding the gate did not run on the gate**. It is now in `smoke`.
|
||||
**Evidence:** `tests/test_suite_membership.py`; `tests/full_only_baseline.txt`; `tests/test_cli_test_suites.py:49`; `50fa287d`; `AGENTS.md:275-280`. · **Authority:** discharged mechanically; **no R-14 ruling is owed** for any of it.
|
||||
|
|
|
|||
|
|
@ -674,7 +674,6 @@ tests/test_rust_backend.py
|
|||
tests/test_rust_scalar_zero_copy_boundary.py
|
||||
tests/test_safe_display.py
|
||||
tests/test_safety_check.py
|
||||
tests/test_safety_pack.py
|
||||
tests/test_safety_refusal.py
|
||||
tests/test_salience.py
|
||||
tests/test_salience_vectorize_parity.py
|
||||
|
|
|
|||
|
|
@ -22,6 +22,12 @@ caused all four. So the pin with teeth is the one that blocks new orphans and
|
|||
makes the legacy set a declared, monotonically shrinking number.
|
||||
|
||||
Recorded as N-9 in ``docs/assessment/50-execution-plan.md``.
|
||||
|
||||
It caught its author within the hour: PR-6 promoted ``tests/test_safety_pack.py``
|
||||
onto the gate and left it listed here, and ``test_baseline_has_no_stale_entries``
|
||||
failed the smoke run. That is the both-directions half doing the work it was
|
||||
written for — a baseline that silently keeps entries it no longer needs stops
|
||||
being a measurement and becomes a story about one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
|
|||
Loading…
Reference in a new issue