feat(tests): pin 3 — a curated suite no gate invokes is not a guarantee (PR-4, G-7)

PR-4's membership ratchet closed "no new orphans". Within the hour this arc's
first full-tree run found tests/test_ratification_ceremony.py RED, in the
curated suite `teaching`, invoked by NO gate tier. Membership was satisfied and
bought nothing — which is exactly the objection PR-4 itself raised against
G-7's original formulation: a curated suite nobody runs is the same
non-guarantee as `full` wearing a different name. The fix had the same blind
spot as the thing it fixed.

Measured: 21 curated suites, 2 gate-reachable (smoke, deductive). The gap is
NINETEEN, not the handful first estimated.

Shipped as baseline + ratchet, sized to the measurement rather than the rule:
19 `post-merge-only` justifications written in one sitting is the ceremony
failure already rejected for the 749-file membership baseline. The pin freezes
the gap, fails on growth, enforces both directions (a promoted suite must leave
the baseline), pins the count, and — critically — VERIFIES ITS DECLARED GATE
SET AGAINST THE SHELL (pre-push / local-ci.sh), so it cannot go vacuous the way
a scan that silently matches nothing does (N-1's class, three false conclusions
in this repo so far).

Two sabotages observed red: a new curated suite with no gate caller, and a
GATE_SUITES value drifted from the scripts it claims to describe. The second
correctly cascaded into two failures, which is the anti-vacuity guard working.

What pin 3 deliberately does NOT do: decide which of the 19 belong on the gate.
That costs gate time and is a decision, not an engineering task. Recorded as the
open half of G-7. `teaching` is the concrete first candidate — it just shipped a
red ratification pin that no gate ran.

Process note: a gate run was invalidated by editing the tree mid-run for the
third time today. Stopped with TaskStop rather than pkill this time, which
reaped cleanly instead of leaving the orphaned wait-loop that ran for 2h16m
earlier. All edits now land before any run starts.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wcw2pnMBwyvmNyQg4uPEt4

core/cli_test.py

@@ -119,6 +119,16 @@ TEST_SUITES: dict[str, tuple[str, ...]] = {
         # full-only files stay a declared, shrinking number rather than 749
         # invented justifications (N-9). Filesystem + glob only, <1s.
         "tests/test_suite_membership.py",
+        # PR-4 pin 3 / G-7 — MEMBERSHIP WAS NEVER THE GUARANTEE; EXECUTION IS.
+        # The membership ratchet closed "no new orphans" and, within the hour,
+        # this arc's full-tree run found test_ratification_ceremony.py RED, in
+        # the curated suite `teaching`, invoked by no gate tier. Measured: 21
+        # curated suites, 2 gate-reachable. This freezes the 19-suite gap so it
+        # cannot grow silently, enforced in both directions, with the declared
+        # gate set verified against the shell so the pin cannot go vacuous.
+        # It deliberately does NOT decide which of the 19 belong on the gate —
+        # that costs gate time and is Shay's call. <1s.
+        "tests/test_suite_reachability.py",
         # G-22 — CLAIMS.md is a PUBLISHED capability artifact, deterministically
         # regenerable from the capability ledger + PINNED_SHAS. It published a
         # superseded evidence digest for deduction_serve_v1 for two days after

docs/assessment/30-gap-register.md

@@ -74,7 +74,11 @@ Pin 1 exists because the two gates have **already** drifted (see H-12): the loca
 
 **REOPENED, PARTLY — 2026-07-28, by the arc's own full-tree run.** The ratchet closes *membership*. It does **not** close *execution*, and the difference is not academic: the first full-tree run of this arc found `tests/test_ratification_ceremony.py` **red on `main`, in a curated suite (`teaching`), and run by no gate** — the pre-push gate invokes `smoke` + the warmed_session lane + `deductive`, and nothing invokes `teaching`. That is exactly the hollow-gate failure this entry's own re-specification warned about — *"a curated suite nobody runs is the same non-guarantee as `full`, wearing a different name"* — demonstrated live, on a pin guarding ratification-corpus byte compatibility.
 
-**What PR-4 still owes, therefore, is a third pin: reachability.** Every curated suite must be reachable from a gate tier (`scripts/hooks/pre-push` / `local-ci.sh`), or be declared post-merge-only with a stated reason. `teaching`, `packs`, `algebra`, `sensorium`, `cognition`, `runtime` and the rest are currently in the unreachable set — they have members, and no gate calls them. Membership was never the guarantee; execution is.
+**PIN 3 LANDED 2026-07-28** — `tests/test_suite_reachability.py`, on the gate. **What PR-4 owed was a third pin: reachability.** Every curated suite must be reachable from a gate tier (`scripts/hooks/pre-push` / `local-ci.sh`), or be declared post-merge-only with a stated reason. `teaching`, `packs`, `algebra`, `sensorium`, `cognition`, `runtime` and the rest are currently in the unreachable set — they have members, and no gate calls them. Membership was never the guarantee; execution is.
+
+**Shipped as baseline + ratchet**, sized to the measurement rather than to the rule: **21 curated suites, 2 gate-reachable** (`smoke`, `deductive`). Demanding 19 `post-merge-only` justifications in one sitting is the ceremony failure already rejected for the 749-file membership baseline. So the pin freezes the 19-suite gap, fails on growth, enforces both directions (a promoted suite must leave the baseline), **verifies its declared gate set against the shell** so it cannot go vacuous, and pins the count. Two sabotages observed red — a new curated suite with no gate caller, and a `GATE_SUITES` value drifting from `pre-push`/`local-ci.sh`.
+
+**What pin 3 deliberately does not do — and what is now Shay's to decide:** *which* of the 19 belong on the gate. That costs gate time and is a real decision. `teaching` is the obvious first candidate: it just shipped a red ratification pin that no gate ran. Recorded as the open half of this entry.
 
 *One real gap the entry was circling, now fixed:* the parity pin lived in `fast`, which the pre-push gate does not run — **the pin guarding the gate did not run on the gate**. It is now in `smoke`.
 **Evidence:** `tests/test_suite_membership.py`; `tests/full_only_baseline.txt`; `tests/test_cli_test_suites.py:49`; `50fa287d`; `AGENTS.md:275-280`. · **Authority:** discharged mechanically; **no R-14 ruling is owed** for any of it.

docs/assessment/50-execution-plan.md

@@ -225,7 +225,9 @@ The assessment directory's own corrections (N-1…N-7 applied to `30-gap-registe
 
 **Verification:** three sabotages, each observed red — a new unregistered test file, a stale baseline entry, and the parity pin demoted out of `smoke`. Remediation half (adding the ten to `smoke.yml`) is **not owed**: it buys observability, not gating.
 
-**STILL OWED — a third pin, found by this arc's own full-tree run (G-7 reopened-partly).** The ratchet closes membership; it does not close **execution**. The run found `tests/test_ratification_ceremony.py` red on `main`, **in a curated suite (`teaching`) that no gate invokes** — the gate runs `smoke` + warmed_session + `deductive`. Membership was satisfied and bought nothing, which is this item's own hollow-gate objection turned on its own fix. **Pin 3: every curated suite must be reachable from a gate tier, or declared post-merge-only with a reason.** Currently unreachable: `teaching`, `packs`, `algebra`, `sensorium`, `cognition`, `runtime`, and the rest — members, no caller.
+**STILL OWED — a third pin, found by this arc's own full-tree run (G-7 reopened-partly).** The ratchet closes membership; it does not close **execution**. The run found `tests/test_ratification_ceremony.py` red on `main`, **in a curated suite (`teaching`) that no gate invokes** — the gate runs `smoke` + warmed_session + `deductive`. Membership was satisfied and bought nothing, which is this item's own hollow-gate objection turned on its own fix. **Pin 3 — LANDED 2026-07-28**, `tests/test_suite_reachability.py`. Measured **21 curated suites, 2 gate-reachable**; the gap is **19**, not the handful first estimated. Shipped as baseline + ratchet for the same reason the membership pin was: 19 justifications written in one sitting is ceremony, not a decision. Freezes the gap, fails on growth, both directions enforced, declared gate set verified against `pre-push`/`local-ci.sh` so it cannot go vacuous. Two sabotages observed red.
+
+**Still open, and it is a decision rather than an engineering task:** which of the 19 go on the gate. `teaching` first — it shipped a red ratification pin that no gate ran (G-22).
 
 ### PR-5 · The flag-default register · **M** · G-8/H-6
 `docs/specs/flag_register.md`: all **28** default-off flags plus the 4 default-on — current default, deliberate-posture vs accumulated-hesitancy, **what evidence flips it**, and named profiles (one-shot / eval / continuous-life) per R-4. Declared in the table, not the call site (ADR-0263 Rule 5). A pin asserts the register lists exactly the flags `core/config.py` defines — so a new flag cannot land unregistered. Includes the F-6 accrual resolution from R-3 and the N-6 docstring correction.

tests/test_suite_reachability.py

@@ -0,0 +1,183 @@
+"""PR-4 pin 3 / G-7 — a curated suite that no gate invokes is not a guarantee.
+
+PR-4's ratchet closed *membership*: a new test file cannot land outside every
+curated suite. Within the hour it turned out membership was never the property
+that mattered.
+
+This arc's first full-tree run found ``tests/test_ratification_ceremony.py``
+**red on main, in the curated suite `teaching`, and invoked by no gate tier**.
+Membership was satisfied and bought nothing — which is precisely the objection
+PR-4 itself raised against G-7's original formulation: *a curated suite nobody
+runs is the same non-guarantee as ``full``, wearing a different name.* The fix
+had the same blind spot as the thing it fixed.
+
+Measured at 2026-07-28: **21 curated suites, 2 gate-reachable.** ``smoke`` and
+``deductive`` are invoked by ``scripts/hooks/pre-push`` and
+``scripts/ci/local-ci.sh --tier gate``; the other nineteen have members and no
+caller. That is why this is a **baseline plus ratchet** and not a flat rule —
+demanding nineteen ``post-merge-only`` justifications in one sitting is the
+ceremony failure rejected for the 749-file membership baseline, and it would
+buy a wall of text rather than a decision.
+
+What this pin does:
+  * freezes the currently-unreachable set, so it cannot grow silently;
+  * fails when a **new** curated suite is added with no gate caller and no
+    explicit declaration;
+  * fails when a suite leaves the unreachable set (promoted to a gate) and its
+    baseline line is not removed — the same both-directions discipline that
+    caught this file's author twice already.
+
+What it deliberately does **not** do: decide which of the nineteen belong on
+the gate. That is a real decision with a real time cost, and it is Shay's.
+This pin makes the state visible and stops it worsening.
+"""
+
+from __future__ import annotations
+
+import ast
+import re
+from pathlib import Path
+
+_ROOT = Path(__file__).resolve().parents[1]
+
+#: Suites invoked by a gate tier today. Parsed rather than hard-coded would be
+#: nicer, but the invocations are shell (``step "…" suite smoke``) and a parse
+#: that silently matched nothing would make this pin vacuous — the exact
+#: failure mode (N-1) that has produced three false conclusions in this repo.
+#: So the set is declared here and *verified* against the shell below.
+GATE_SUITES: frozenset[str] = frozenset({"smoke", "deductive"})
+
+#: Curated suites with members and no gate caller, measured 2026-07-28.
+#: This list may only SHRINK. A suite leaves it by being invoked from a gate
+#: tier — not by being deleted from this file.
+UNREACHABLE_BASELINE: frozenset[str] = frozenset(
+    {
+        "adr-0024",
+        "algebra",
+        "cognition",
+        "fast",
+        "formation",
+        "full",
+        "inner-loop",
+        "margin",
+        "math",
+        "packs",
+        "phase5",
+        "phase6",
+        "proof",
+        "pulse",
+        "refusal",
+        "rotor",
+        "runtime",
+        "sensorium",
+        "teaching",
+    }
+)
+
+
+def _curated_suites() -> dict[str, tuple[str, ...]]:
+    src = (_ROOT / "core" / "cli_test.py").read_text(encoding="utf-8")
+    match = re.search(r"TEST_SUITES\s*(?::[^=]+)?=\s*\{", src)
+    assert match is not None, "TEST_SUITES literal not found"
+    start = match.end() - 1
+    depth = 0
+    end = len(src)
+    for i, ch in enumerate(src[start:], start):
+        if ch == "{":
+            depth += 1
+        elif ch == "}":
+            depth -= 1
+            if depth == 0:
+                end = i + 1
+                break
+    return ast.literal_eval(src[start:end])
+
+
+def _suites_invoked_by_gate_tiers() -> set[str]:
+    """Suites named by a ``suite <name>`` step in the hook or the gate tier."""
+    found: set[str] = set()
+    for rel in ("scripts/hooks/pre-push", "scripts/ci/local-ci.sh"):
+        text = (_ROOT / rel).read_text(encoding="utf-8")
+        found.update(re.findall(r"--suite\s+([a-z0-9-]+)", text))
+        found.update(re.findall(r"\bsuite\s+([a-z0-9-]+)\s*$", text, re.MULTILINE))
+    # ``full`` is the arc-boundary tier, not a gate; it is not a merge bar.
+    found.discard("full")
+    found.discard("only")  # prose match guard
+    return found
+
+
+def test_declared_gate_suites_match_the_shell() -> None:
+    """GATE_SUITES must be what the hook and gate tier actually invoke.
+
+    Without this, the pin below could pass while pointing at a fiction — a
+    declared set that drifted from the scripts it claims to describe.
+    """
+    invoked = _suites_invoked_by_gate_tiers()
+    assert invoked, "parsed no suite invocations — the pin would be vacuous"
+    assert invoked == set(GATE_SUITES), (
+        f"gate tiers invoke {sorted(invoked)}, but GATE_SUITES declares "
+        f"{sorted(GATE_SUITES)} — update the constant with the change that "
+        "moved the gate, and move any promoted suite out of "
+        "UNREACHABLE_BASELINE in the same edit"
+    )
+
+
+def test_no_new_gate_unreachable_suite() -> None:
+    """A new curated suite must be gate-reachable or explicitly baselined.
+
+    If this fails on a suite you just added: either invoke it from a gate tier
+    (``scripts/hooks/pre-push`` / ``local-ci.sh --tier gate``), or add it to
+    UNREACHABLE_BASELINE with the reason in your commit message. The second is
+    available and is a decision, not a default — a suite nobody runs guarantees
+    nothing.
+    """
+    suites = set(_curated_suites())
+    unreachable = suites - set(GATE_SUITES)
+    undeclared = sorted(unreachable - UNREACHABLE_BASELINE)
+    assert not undeclared, (
+        "these curated suites have members and no gate caller, and are not in "
+        f"the recorded baseline: {undeclared}"
+    )
+
+
+def test_baseline_has_no_stale_entries() -> None:
+    """Both directions: a promoted or deleted suite must leave the baseline.
+
+    Same discipline as ``test_suite_membership.py`` — a baseline that keeps
+    entries it no longer needs stops being a measurement and becomes a story
+    about one.
+    """
+    suites = set(_curated_suites())
+
+    vanished = sorted(s for s in UNREACHABLE_BASELINE if s not in suites)
+    assert not vanished, (
+        f"baseline names suites that no longer exist: {vanished}"
+    )
+
+    promoted = sorted(s for s in UNREACHABLE_BASELINE if s in GATE_SUITES)
+    assert not promoted, (
+        "these suites are now gate-reachable (good) but are still listed as "
+        f"unreachable — remove them from UNREACHABLE_BASELINE: {promoted}"
+    )
+
+
+def test_recorded_gate_gap_matches_the_measurement() -> None:
+    """Pin the number so the gap moving is a reviewed decision, not a drift.
+
+    19 of 21 at 2026-07-28. Shrinking this is the point — it shrinks by putting
+    a suite on the gate, which costs gate time and is therefore a decision
+    someone has to make deliberately.
+    """
+    assert len(UNREACHABLE_BASELINE) <= 19, (
+        f"gate-unreachable suites grew to {len(UNREACHABLE_BASELINE)} (was 19). "
+        "The ratchet only turns one way."
+    )
+
+
+def test_the_two_gate_suites_actually_exist() -> None:
+    """A gate that names a suite the CLI does not define would fail closed at
+    run time, but silently here. Pin it."""
+    suites = _curated_suites()
+    for name in GATE_SUITES:
+        assert name in suites, f"gate tier invokes undefined suite {name!r}"
+        assert suites[name], f"gate suite {name!r} is empty"