diff --git a/AGENTS.md b/AGENTS.md index 3a9b3d83..1a4a696f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -263,6 +263,7 @@ To optimize server resources and bypass external CI billing dependencies, all ag uv run core test --suite smoke -q ``` Ensure all smoke tests pass (parity with the smoke gate is pinned by `test_cli_smoke_suite_covers_ci_smoke_gate`). Pushing broken code is a critical protocol violation. +- **Automated pre-push hook (2026-07-22):** `sh scripts/hooks/install.sh` installs a `core.hooksPath` pre-push gate that runs the `smoke` suite **plus** the `warmed_session` consistency lane pin (`tests/test_warmed_session_lane.py`). The lane pin catches the T13 telemetry-consistency regression class that `smoke` does **not** cover (the #96 fail-closed resolve + #97 morph override escaped the smoke files and surfaced only in the warmed_session lane). The full ~12k fast-lane stays **async CI** by design; the hook is deliberately targeted so it blocks the regression class without gridlocking the push cycle. Emergency bypass (discouraged): `git push --no-verify`. - **Pre-Merge Gate:** Before proposing a merge or requesting a review on a PR, you **MUST** run the larger validation suite relevant to your changes (e.g. `uv run core test --suite cognition` or `uv run core test --suite algebra`). - **No Docker CI for merge:** Do not run, wait on, or re-provision Docker-container CI to green a merge. Fix and prove in-worktree; merge on local evidence. - **PR Documentation:** When creating a PR on Forgejo (via the Forgejo MCP tools), document the local test execution in the PR description, matching this format: diff --git a/docs/analysis/weekly-audit-2026-07-22-stragglers-todo.md b/docs/analysis/weekly-audit-2026-07-22-stragglers-todo.md index 18aaf470..0e575278 100644 --- a/docs/analysis/weekly-audit-2026-07-22-stragglers-todo.md +++ b/docs/analysis/weekly-audit-2026-07-22-stragglers-todo.md @@ -144,17 +144,29 @@ fail-closed determinism). Execution status: ### New findings from the execution pass -- [ ] **T10 (fixed in `fix/test-engine-state-isolation`, needs merge): the test +- [x] **T10 (FIXED + MERGED — PR #100 @ `80100c18`): the test suite wrote the live life-store.** Module-scoped fixtures (e.g. `tests/test_achat.py`) construct `ChatRuntime()` BEFORE the function-scoped isolation fixture exists (pytest sets up wider scopes first), binding the real `engine_state/`: smoke runs loaded it and committed generations (turn_count 14989→14990 observed). Fix: session-scoped baseline fixture + regression pin; `docs/issues/default-engine-state-test-hygiene.md` addendum. -- [ ] **T11 (ruling needed): live-store provenance review.** Historical - `turn_count` (14990) includes an unknown number of pre-fix test turns and - the 465 persisted discovery candidates predate the isolation fix. Decide: - accept-as-lived vs audit/relabel candidates. +- [x] **T11 (RULED + EXECUTED 2026-07-22): live-store provenance reset.** + DECISIVE FINDING: candidates carry NO timestamp / run-id / test-prod field + (only a `source_turn_trace` hash + `review_state`), and gen-21701 == gen-21702 + were byte-identical — "the 465" was the ENTIRE live candidate corpus, not a + separable tainted subset, so a surgical per-record quarantine (the original + `TEST_TAINTED` spec) was impossible. The candidates were inert (proposal / + contemplation only, never the serving surface). **Ruling (Shay): Option (a) — + clear, don't asterisk.** Executed via + `scripts/ops/reset_candidate_corpus_t11_20260722.py`: an ADR-0219 atomic + generation reset (begin → carry recognizers/session/identity+turn_count → + empty candidate ledger → commit `keep=1`). Result: candidates 465 → 0, + `turn_count` 14990 preserved, gen-21701/21702 physically GC'd, `current` → + gen-21703 @ `9a428d84`. Idle-tick discovery repopulates organically. + **`turn_count` itself left as-is** (honest lived count; the pre-fix test turns + are not separably identifiable, and resetting the counter would fabricate a + number — the candidate corpus was the provenance hazard, not the scalar). - [ ] **T12 (pattern watch): experiment scaffolding scope over-reach.** The T6 glob hazard suggests a cheap architecture pin: `rnd`/scripts corpus readers should be lint-checked against sealed/practice eval paths. @@ -183,8 +195,8 @@ fail-closed determinism). Execution status: surface and `trace_hash` in the durable stream. Deferred (default sink is `None`, so no live consumer is affected today); own PR when a sink consumer lands. -- [x] **T13 (main regression, serving-provenance — telemetry half FIXED): - fast lane is red on main** (was 0-red 2026-07-16): +- [x] **T13 (main regression, serving-provenance — telemetry FIXED + MERGED #101): + fast lane was red on main** (was 0-red 2026-07-16): `tests/test_warmed_session_lane.py::TestPipelineOverrideGateInvariants::test_telemetry_consistency_rate_is_one` fails at 0.9444 (17/18). Verified pre-existing on unmodified main (fails plain AND state-isolated; unrelated to the T10 isolation fix). Root cause: @@ -206,20 +218,25 @@ fail-closed determinism). Execution status: not cover this lane, and GitHub Actions are dead signals — nothing gates fast-lane red on main right now. - **Resolution (2026-07-22, PR `fix/telemetry-serve-boundary`):** + **Resolution (2026-07-22, PR #101 `fix/telemetry-serve-boundary` @ `9a428d84`, MERGED):** - **(1) FIXED** — `ChatRuntime.finalize_turn_surface` back-stamps the pipeline's resolved surface onto `turn_log[-1]`, a sibling to `finalize_turn_trace_hash`, called from `pipeline.py` immediately after the trace-hash back-stamp. Lane green: `telemetry_consistency_rate` 0.9444 → 1.0 (10/10 warmed_session tests pass). `trace_hash` byte-identity preserved — it folds the pre-decoration surface, not the served surface. - - **(2) OPEN — needs Shay's ruling.** Orthogonal to the red (telemetry now - matches whatever is served, refusal or not). Recommend **AGAINST** a - query-type "definitional/epistemic" classifier bypass: that fails a - geometric coherence gate OPEN on a lexical cue — the fail-open / cue-table - pattern ADR-0252 retired and INV-34 / fail-closed forbid. Principled - alternative: route open-geometry-but-**pack-grounded** surfaces to the - existing hedge-injection arm (`authoritative=False`, honestly hedged), - discriminated by grounding *provenance* (structural), not question type. - Own focused PR + real-data (not synthetic) validation. + - **(2) RULED (Shay, 2026-07-22): hedge-arm routing, own PR.** The query-type + "definitional/epistemic" classifier bypass is REJECTED — it fails a + geometric coherence gate OPEN on a lexical cue (fail-open / cue-table; + ADR-0252 + INV-34). Ruling: fix the READING, not the question — route + open-geometry-but-**pack-grounded** surfaces to the existing hedge-injection + arm (`authoritative=False`, honestly hedged), discriminated by grounding + *provenance* (structural), never by question type. Its own dedicated PR + + GSM8K-style (real-data) validation. Scoped next; deliberately NOT in the + telemetry PR or the infra PR. + - **(3) RULED (Shay, 2026-07-22): local pre-push gate — IMPLEMENTED.** + `scripts/hooks/pre-push` + `scripts/hooks/install.sh` (`core.hooksPath`) run + the smoke suite **plus** the warmed_session lane pin on every push — the + targeted gate (NOT the ~9-min full fast-lane, which stays async CI). + AGENTS.md §Local-First CI Validation Protocol updated. (This PR.) - Durable-sink residual tracked as **R7** above. diff --git a/scripts/hooks/install.sh b/scripts/hooks/install.sh new file mode 100755 index 00000000..3428233f --- /dev/null +++ b/scripts/hooks/install.sh @@ -0,0 +1,22 @@ +#!/bin/sh +# Install the CORE local-first pre-push gate for THIS clone. +# +# Points git at the tracked hooks directory (scripts/hooks) via core.hooksPath. +# The relative path resolves per-worktree, so every linked worktree gets the +# same gate. Idempotent — safe to re-run. Run once after cloning: +# +# sh scripts/hooks/install.sh +# +# Uninstall: git config --unset core.hooksPath + +set -eu + +repo_root="$(git rev-parse --show-toplevel)" +cd "${repo_root}" + +chmod +x scripts/hooks/pre-push +git config core.hooksPath scripts/hooks + +echo "[install] core.hooksPath -> scripts/hooks" +echo "[install] pre-push gate active: smoke suite + warmed_session lane pin." +echo "[install] emergency bypass (discouraged): git push --no-verify" diff --git a/scripts/hooks/pre-push b/scripts/hooks/pre-push new file mode 100755 index 00000000..4dde729f --- /dev/null +++ b/scripts/hooks/pre-push @@ -0,0 +1,52 @@ +#!/bin/sh +# CORE local-first pre-push gate (weekly-audit T13, 2026-07-22). +# +# Automates the AGENTS.md §Local-First CI Validation Protocol "Pre-Push Gate": +# (1) the `smoke` suite — exact CI-gate parity, and +# (2) the warmed_session consistency lane pin — catches the T13 +# telemetry-consistency regression class, which `smoke` does NOT cover +# (the #96 fail-closed resolve + #97 morph override escaped smoke and +# surfaced only in the warmed_session lane). +# The full ~12k fast-lane stays an ASYNC CI concern by design — this gate is +# deliberately targeted so it prevents the regression class without gridlocking +# the push cycle. +# +# Install (per clone): scripts/hooks/install.sh (sets core.hooksPath). +# Emergency bypass: git push --no-verify (defeats the gate — avoid). + +set -u + +z40="0000000000000000000000000000000000000000" +read_any=0 +only_deletes=1 +while read -r _local_ref local_sha _remote_ref _remote_sha; do + read_any=1 + if [ "${local_sha}" != "${z40}" ] && [ -n "${local_sha}" ]; then + only_deletes=0 + fi +done + +# Fail-closed: run the gate unless we positively identified a deletion-only push. +if [ "${read_any}" -eq 1 ] && [ "${only_deletes}" -eq 1 ]; then + echo "[pre-push] deletion-only push — nothing to validate." + exit 0 +fi + +echo "[pre-push] CORE local-first gate: smoke suite + warmed_session lane pin" + +echo "[pre-push] (1/2) smoke suite (core test --suite smoke) ..." +if ! uv run core test --suite smoke -q; then + echo "[pre-push] BLOCKED: smoke suite failed. Fix before pushing." >&2 + echo "[pre-push] (emergency bypass, discouraged: git push --no-verify)" >&2 + exit 1 +fi + +echo "[pre-push] (2/2) warmed_session consistency lane ..." +if ! uv run python -m pytest tests/test_warmed_session_lane.py -q; then + echo "[pre-push] BLOCKED: warmed_session lane failed (T13 regression class)." >&2 + echo "[pre-push] (emergency bypass, discouraged: git push --no-verify)" >&2 + exit 1 +fi + +echo "[pre-push] gate PASSED — push proceeding." +exit 0 diff --git a/scripts/ops/reset_candidate_corpus_t11_20260722.py b/scripts/ops/reset_candidate_corpus_t11_20260722.py new file mode 100644 index 00000000..8cb0a4b6 --- /dev/null +++ b/scripts/ops/reset_candidate_corpus_t11_20260722.py @@ -0,0 +1,120 @@ +#!/usr/bin/env python +"""One-off: reset the live discovery-candidate corpus (weekly-audit T11 ruling). + +Weekly audit 2026-07-22, Ruling T11 (Shay): the 465 discovery candidates in the +live engine_state store were generated while pre-#100 smoke runs could WRITE the +live store (the module-scope conftest escape #100 sealed). They are provenance- +tainted. They are *inert* — candidates feed only the config-gated proposal / +contemplation loop, never the serving surface — but Absolute Provenance does not +tolerate synthetically-tainted records lingering in a production epistemic corpus. +Ruling: Option (a) — do not asterisk the ledger, clear it. Let idle-tick +discovery repopulate the corpus organically. + +WHY THIS SCRIPT AND NOT `rm`: +The store is an ADR-0219 atomic-generation checkpoint. A raw file delete would +itself violate the checkpoint contract (torn state, orphaned pointer). This +routes the reset through the store's own two-phase commit so the wipe is itself +an atomic, auditable committed generation: + + 1. begin_generation() -> allocate the next gen dir + 2. carry forward recognizers + lived session_state + manifest (identity + lineage + turn_count UNCHANGED — only the candidate ledger is reset) + 3. save_discovery_candidates([]) -> the wipe (empty ledger) + 4. commit_generation(keep=1) -> atomic `current` pointer swap, then GC + every older generation so the tainted candidate files are PHYSICALLY gone + (keep=1 is deliberate: the point of the reset is to eliminate the taint, + not retain it as a rollback generation). + +Idempotent-ish guard: refuses to run unless the live store holds exactly the +EXPECTED_TAINTED count, so it cannot be misfired against a repopulated store. + +EXECUTED once, 2026-07-22, against the live store @ 9a428d8466c3: + pre: candidates=465 recognizers=0 session_state=absent turn_count=14990 + post: candidates=0 turn_count=14990 remaining_generations=[gen-21703] + (gen-21701 / gen-21702 GC'd; identity lineage c9e5968a… preserved) +Retained in-tree as the provenance record of the reset. +""" + +from __future__ import annotations + +import os +import sys +from pathlib import Path + +from engine_state import EngineStateStore + +EXPECTED_TAINTED = 465 # the audited pre-#100 candidate count + + +def main() -> int: + store_dir = os.environ.get("CORE_ENGINE_STATE_DIR") + store = EngineStateStore(Path(store_dir)) if store_dir else EngineStateStore() + print(f"[T11] target store: {store.path}") + + if not store.exists(): + print("[T11] ABORT: no committed checkpoint at target store.", file=sys.stderr) + return 2 + + candidates = store.load_discovery_candidates() + manifest = store.load_manifest() or {} + recognizers = store.load_recognizers() + session_state = store.load_session_state() + turn_count = int(manifest.get("turn_count", 0)) + + print( + f"[T11] pre-reset: candidates={len(candidates)} recognizers={len(recognizers)} " + f"session_state={'present' if session_state is not None else 'absent'} " + f"turn_count={turn_count}" + ) + + if len(candidates) != EXPECTED_TAINTED: + print( + f"[T11] ABORT: expected {EXPECTED_TAINTED} tainted candidates, found " + f"{len(candidates)}. Refusing to run against an unexpected store.", + file=sys.stderr, + ) + return 3 + + # --- atomic generation-dir reset (ADR-0219) ------------------------------- + gen_num, gen_dir = store.begin_generation() + gen_store = EngineStateStore(gen_dir) + gen_store.save_recognizers(recognizers) # carry forward (faithful) + gen_store.save_discovery_candidates([]) # THE WIPE + if session_state is not None: + gen_store.save_session_state(session_state) # carry forward lived state + gen_store.save_manifest( + turn_count, # UNCHANGED + engine_identity=manifest.get("engine_identity", ""), + parent_engine_identity=manifest.get("parent_engine_identity", ""), + identity_scheme=int(manifest.get("identity_scheme", 2)), + ) + store.commit_generation(gen_num, keep=1) # swap + GC every older gen + print(f"[T11] committed reset generation gen-{gen_num:04d} (keep=1)") + + # --- verify --------------------------------------------------------------- + verify = EngineStateStore(store.path) + post_candidates = verify.load_discovery_candidates() + post_manifest = verify.load_manifest() or {} + post_turn = int(post_manifest.get("turn_count", -1)) + remaining_gens = sorted( + p.name for p in store.path.iterdir() if p.is_dir() and p.name.startswith("gen-") + ) + + ok = ( + len(post_candidates) == 0 + and post_turn == turn_count + and remaining_gens == [f"gen-{gen_num:04d}"] + ) + print( + f"[T11] post-reset: candidates={len(post_candidates)} turn_count={post_turn} " + f"remaining_generations={remaining_gens}" + ) + if not ok: + print("[T11] VERIFY FAILED — inspect the store manually.", file=sys.stderr) + return 4 + print("[T11] OK — candidate corpus cleared; lived state + identity + turn_count intact.") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main())