docs(assessment): main is red, was red before this arc, and membership was not the guarantee (G-22, G-7)

The arc's first full-tree run — 13,403 passed, 3 FAILED — and the three
failures reproduce identically at 797ebad5, the arc's starting commit. NOT
arc-caused. main has been red for an unknown period and nothing surfaced it.

  tests/test_ratification_ceremony.py::test_row_is_byte_compatible_with_the_committed_corpus
  tests/test_ratification_ceremony.py::test_unreviewed_status_is_refused
  tests/test_claims_md_is_current.py::test_claims_md_matches_generator_output

DIAGNOSIS — the code is right and the pins are stale, same class as H-8e.
ChainRecord.__slots__ gained "polarity" (ADR-0264 R1), and the serializer
DELIBERATELY OMITS it for affirmative rows (teaching/ratification.py:119:
"polarity is OMITTED when affirmative"). A test that rebuilds
ChainRecord(**{k: row[k] for k in __slots__}) therefore KeyErrors on a
CORRECTLY serialized row. Consequence: the ratification-corpus byte-
compatibility guarantee and the unreviewed-status refusal have been UNVERIFIED
since ADR-0264 R1 landed.

THE STRUCTURAL FINDING, which is bigger than the three tests. test_ratification
_ceremony.py IS in a curated suite — `teaching` — and no gate invokes it. The
pre-push gate runs smoke + the warmed_session lane + deductive. So a curated,
red, governance-path pin sat unrun.

That is this arc's own hollow-gate objection turned back on its own fix. PR-4's
ratchet closes MEMBERSHIP. It does not close EXECUTION, and I closed G-7 as
though it did. G-7 is reopened-partly and PR-4 now owes a third pin:

  PIN 3 — every curated suite must be reachable from a gate tier
  (scripts/hooks/pre-push / local-ci.sh), or be declared post-merge-only with a
  stated reason. Currently unreachable: teaching, packs, algebra, sensorium,
  cognition, runtime and the rest — they have members and no caller.

Membership was never the guarantee. Execution is.

Recorded as G-22 with the diagnosis; repair is test-side and mechanical, but
these guard governance-path behavior and a published artifact (CLAIMS.md), so
the disposition is Shay's rather than mine.

Docs only.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wcw2pnMBwyvmNyQg4uPEt4
This commit is contained in:
Claude 2026-07-28 04:58:21 +00:00
parent 3650331b5b
commit 913111c0dc
No known key found for this signature in database
2 changed files with 7 additions and 0 deletions

View file

@ -72,6 +72,10 @@ Pin 1 exists because the two gates have **already** drifted (see H-12): the loca
*Pin 2 shipped, re-specified — and it caught its author within the hour:* PR-6 promoted `tests/test_safety_pack.py` onto the gate and left it in the baseline; the both-directions half failed the very next smoke run, and the count went 749 → 748. *Pin 2 shipped, re-specified.* Measured **749 of 877** test files in no curated suite. The mechanism this entry proposes — assign every orphan, or list it with a reason — is hollow at that scale for the same reason the entry itself identifies: glob topic-suites satisfy the assertion while changing nothing about what runs. All four demonstrated incidents were caused by a *newly landed* orphan, never a legacy one, so what shipped is a **ratchet**: `tests/test_suite_membership.py` + `tests/full_only_baseline.txt`, blocking new orphans, enforced in both directions, count pinned. Three sabotages observed red.
**REOPENED, PARTLY — 2026-07-28, by the arc's own full-tree run.** The ratchet closes *membership*. It does **not** close *execution*, and the difference is not academic: the first full-tree run of this arc found `tests/test_ratification_ceremony.py` **red on `main`, in a curated suite (`teaching`), and run by no gate** — the pre-push gate invokes `smoke` + the warmed_session lane + `deductive`, and nothing invokes `teaching`. That is exactly the hollow-gate failure this entry's own re-specification warned about — *"a curated suite nobody runs is the same non-guarantee as `full`, wearing a different name"* — demonstrated live, on a pin guarding ratification-corpus byte compatibility.
**What PR-4 still owes, therefore, is a third pin: reachability.** Every curated suite must be reachable from a gate tier (`scripts/hooks/pre-push` / `local-ci.sh`), or be declared post-merge-only with a stated reason. `teaching`, `packs`, `algebra`, `sensorium`, `cognition`, `runtime` and the rest are currently in the unreachable set — they have members, and no gate calls them. Membership was never the guarantee; execution is.
*One real gap the entry was circling, now fixed:* the parity pin lived in `fast`, which the pre-push gate does not run — **the pin guarding the gate did not run on the gate**. It is now in `smoke`.
**Evidence:** `tests/test_suite_membership.py`; `tests/full_only_baseline.txt`; `tests/test_cli_test_suites.py:49`; `50fa287d`; `AGENTS.md:275-280`. · **Authority:** discharged mechanically; **no R-14 ruling is owed** for any of it.
@ -132,6 +136,7 @@ Own `use_salience`, the two underived constants, the self-narrowing budget feedb
- **G-17 · Non-text ingest** — 59 sensorium modules, no serving path, no entry criterion; projection heads do not exist. Position paper is honest about this. Needs either an entry criterion or an explicit deferral ruling (the falsification bench is the standard the track should be held to when it moves). **Authority:** ruling.
- **G-18 · Identity-divergence curriculum may still bypass formation's gates** — known gap since 2026-05-17 (`teaching_order.md`); unverified at this SHA. **Authority:** Phase-3-style verification pass, then a routing PR.
- **G-19 · Wilson/replay evidence shortfall** — 21/25 ratified bands short if replays were counted as independent trials (see H-1 for the mechanism). Recorded here as *evidence debt on existing licenses*; the counting fix is the hindrance entry. **Note (Phase 6):** the exposure is **already pinned in-repo**`tests/test_volume_honesty.py` (ADR-0264 R9, in the local smoke gate) pins the 21-of-25 shortfall *"in BOTH directions"* and calls its inventory *"an EXPOSURE INVENTORY, not an approved baseline."* So the open work is **applying** the demotions, not discovering them, and that pin moves in the same PR. **Authority:** ADR amendment + re-count, authorized by R-13.
- **G-22 · Three tests are red on `main`, and were red before this arc** *(new, 2026-07-28)* — found by the arc's first full-tree run (`3 failed, 13403 passed`), and reproduced identically at `797ebad5`, so **not arc-caused**. `tests/test_ratification_ceremony.py::{test_row_is_byte_compatible_with_the_committed_corpus, test_unreviewed_status_is_refused}` and `tests/test_claims_md_is_current.py::test_claims_md_matches_generator_output`. **Diagnosis:** the code is correct and the pins are stale. `ChainRecord.__slots__` gained `polarity` (ADR-0264 R1), and the serializer **deliberately omits it for affirmative rows** (`teaching/ratification.py:119`) — so a test that builds `ChainRecord(**{k: row[k] for k in __slots__})` `KeyError`s on a correctly-serialized row. Same class as H-8e: ratified design right, pin not updated. **Consequence:** the ratification-corpus byte-compatibility guarantee and the unreviewed-status refusal have been **unverified since ADR-0264 R1 landed**. `test_ratification_ceremony.py` is curated (`teaching`) but ungated; `test_claims_md_is_current.py` is an orphan. **Authority:** the repair is test-side and mechanical, but these guard governance-path behavior and a published artifact (`CLAIMS.md`), so the disposition is Shay's.
- **G-21 · The math reader decides 1.0% of `holdout_dev/v1`** *(new, 2026-07-28)* — measured while building the §5 corpus: `parse_and_solve` returns a selected graph for **5 of 500** held-out cases, all carrying the same relational skeleton (`compare_multiplicative`); on the public lane, 24/150. This is a sharper measurement of the comprehension frontier than G-3's construction count, on the corpus the project already treats as its held-out standard, and it is why ADR-0252 §5.1's four-structure corpus is not extractable today. Distinct from G-3 (which counts *constructions* the general reader admits); this counts *cases decided* on a standing eval corpus. **Authority:** the widening program (G-3) + a ruling on whether holdout decision-rate becomes a tracked lane metric.
- **G-20 · The `refusal_reason` materialisation** — typed refusal evidence exists and is discarded at the public `str` boundary; the plumbing for materialisation already landed. Cross-listed as H-3. **Authority:** small ADR (anticipated by the ADR-0024 chain).

View file

@ -225,6 +225,8 @@ The assessment directory's own corrections (N-1…N-7 applied to `30-gap-registe
**Verification:** three sabotages, each observed red — a new unregistered test file, a stale baseline entry, and the parity pin demoted out of `smoke`. Remediation half (adding the ten to `smoke.yml`) is **not owed**: it buys observability, not gating.
**STILL OWED — a third pin, found by this arc's own full-tree run (G-7 reopened-partly).** The ratchet closes membership; it does not close **execution**. The run found `tests/test_ratification_ceremony.py` red on `main`, **in a curated suite (`teaching`) that no gate invokes** — the gate runs `smoke` + warmed_session + `deductive`. Membership was satisfied and bought nothing, which is this item's own hollow-gate objection turned on its own fix. **Pin 3: every curated suite must be reachable from a gate tier, or declared post-merge-only with a reason.** Currently unreachable: `teaching`, `packs`, `algebra`, `sensorium`, `cognition`, `runtime`, and the rest — members, no caller.
### PR-5 · The flag-default register · **M** · G-8/H-6
`docs/specs/flag_register.md`: all **28** default-off flags plus the 4 default-on — current default, deliberate-posture vs accumulated-hesitancy, **what evidence flips it**, and named profiles (one-shot / eval / continuous-life) per R-4. Declared in the table, not the call site (ADR-0263 Rule 5). A pin asserts the register lists exactly the flags `core/config.py` defines — so a new flag cannot land unregistered. Includes the F-6 accrual resolution from R-3 and the N-6 docstring correction.