From edf6c2a4540691bfb50ed9a2fbb04b6a8c5157ac Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 28 Jul 2026 04:22:28 +0000 Subject: [PATCH] =?UTF-8?q?fix(tests):=20the=20membership=20ratchet=20caug?= =?UTF-8?q?ht=20its=20author=20=E2=80=94=20baseline=20749=20->=20748?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit PR-6 promoted tests/test_safety_pack.py onto the gate and left it listed in tests/full_only_baseline.txt. test_baseline_has_no_stale_entries failed the very next smoke run, naming the file and the direction. That is the both-directions half of the ratchet doing exactly the work it was written for, roughly an hour after it landed, against the person who wrote it. A baseline that silently keeps entries it no longer needs stops being a measurement and becomes a story about one — so the failure is the feature. Recorded in the pin's own docstring and in G-7, because a mechanism catching its author is better evidence than a sabotage test: the sabotage was contrived, this was not. Baseline: 749 -> 748. The ratchet only turns down. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01Wcw2pnMBwyvmNyQg4uPEt4 --- docs/assessment/30-gap-register.md | 2 +- tests/full_only_baseline.txt | 1 - tests/test_suite_membership.py | 6 ++++++ 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/docs/assessment/30-gap-register.md b/docs/assessment/30-gap-register.md index 4bdbaca0..f226e07d 100644 --- a/docs/assessment/30-gap-register.md +++ b/docs/assessment/30-gap-register.md @@ -70,7 +70,7 @@ Pin 1 exists because the two gates have **already** drifted (see H-12): the loca *Pin 1 was already in the repository, in the only direction that protects anything.* `test_cli_smoke_suite_covers_ci_smoke_gate` has pinned `smoke.yml ⊆ TEST_SUITES["smoke"]` throughout (which is why every measurement above finds **CI-only = 0** — a pin, not luck), under a comment reading *"DELIBERATELY ONE-DIRECTIONAL."* The symmetric version was written and reverted the same day in `50fa287d` (2026-07-25), because `AGENTS.md:280` makes GitHub Actions *"billing-locked … dead signals"* and §277 makes the workflows *"secondary observability only."* This entry's "the two gates have already drifted" is therefore a **misreading of a design decision**, and the delta is not a defect to close. -*Pin 2 shipped, re-specified.* Measured **749 of 877** test files in no curated suite. The mechanism this entry proposes — assign every orphan, or list it with a reason — is hollow at that scale for the same reason the entry itself identifies: glob topic-suites satisfy the assertion while changing nothing about what runs. All four demonstrated incidents were caused by a *newly landed* orphan, never a legacy one, so what shipped is a **ratchet**: `tests/test_suite_membership.py` + `tests/full_only_baseline.txt`, blocking new orphans, enforced in both directions, count pinned. Three sabotages observed red. +*Pin 2 shipped, re-specified — and it caught its author within the hour:* PR-6 promoted `tests/test_safety_pack.py` onto the gate and left it in the baseline; the both-directions half failed the very next smoke run, and the count went 749 → 748. *Pin 2 shipped, re-specified.* Measured **749 of 877** test files in no curated suite. The mechanism this entry proposes — assign every orphan, or list it with a reason — is hollow at that scale for the same reason the entry itself identifies: glob topic-suites satisfy the assertion while changing nothing about what runs. All four demonstrated incidents were caused by a *newly landed* orphan, never a legacy one, so what shipped is a **ratchet**: `tests/test_suite_membership.py` + `tests/full_only_baseline.txt`, blocking new orphans, enforced in both directions, count pinned. Three sabotages observed red. *One real gap the entry was circling, now fixed:* the parity pin lived in `fast`, which the pre-push gate does not run — **the pin guarding the gate did not run on the gate**. It is now in `smoke`. **Evidence:** `tests/test_suite_membership.py`; `tests/full_only_baseline.txt`; `tests/test_cli_test_suites.py:49`; `50fa287d`; `AGENTS.md:275-280`. · **Authority:** discharged mechanically; **no R-14 ruling is owed** for any of it. diff --git a/tests/full_only_baseline.txt b/tests/full_only_baseline.txt index 862e74ac..334542a5 100644 --- a/tests/full_only_baseline.txt +++ b/tests/full_only_baseline.txt @@ -674,7 +674,6 @@ tests/test_rust_backend.py tests/test_rust_scalar_zero_copy_boundary.py tests/test_safe_display.py tests/test_safety_check.py -tests/test_safety_pack.py tests/test_safety_refusal.py tests/test_salience.py tests/test_salience_vectorize_parity.py diff --git a/tests/test_suite_membership.py b/tests/test_suite_membership.py index 52b52c49..fd7d1f46 100644 --- a/tests/test_suite_membership.py +++ b/tests/test_suite_membership.py @@ -22,6 +22,12 @@ caused all four. So the pin with teeth is the one that blocks new orphans and makes the legacy set a declared, monotonically shrinking number. Recorded as N-9 in ``docs/assessment/50-execution-plan.md``. + +It caught its author within the hour: PR-6 promoted ``tests/test_safety_pack.py`` +onto the gate and left it listed here, and ``test_baseline_has_no_stale_entries`` +failed the smoke run. That is the both-directions half doing the work it was +written for — a baseline that silently keeps entries it no longer needs stops +being a measurement and becomes a story about one. """ from __future__ import annotations