core-labs/core
core-labs/core
1
0
Fork 0
Code Issues Pull requests 2 Projects Releases Packages Wiki Activity

chore(ci): local pre-push gate (smoke + warmed_session pin) + T11 reset record

Browse source 
Weekly-audit 2026-07-22 closeout — T13 decision (3) CI gate + T11 execution.

- scripts/hooks/pre-push + install.sh: a core.hooksPath pre-push gate that runs
  the smoke suite PLUS the warmed_session consistency lane pin — the T13
  telemetry-consistency regression class that smoke does NOT cover. The full
  ~12k fast-lane stays async CI by design; the gate is deliberately targeted so
  it blocks the regression class without gridlocking pushes. Fail-closed on
  empty stdin; skips deletion-only pushes; emergency bypass git push --no-verify.

- scripts/ops/reset_candidate_corpus_t11_20260722.py: provenance record of the
  T11 live-store reset. The 465 discovery candidates predated the #100
  isolation fix and carried no per-record test/prod discriminator, so a surgical
  quarantine was impossible; ruling was to clear (not asterisk). Executed as an
  ADR-0219 atomic generation reset (empty candidate ledger, keep=1) — candidates
  465 -> 0, turn_count 14990 and identity lineage preserved, tainted generations
  GC'd. Data lives in gitignored engine_state/; this script is the audit trail.

- AGENTS.md: document the automated pre-push gate under the Local-First CI
  Validation Protocol.

- weekly-audit ledger: T10 merged (#100), T11 ruled+executed, T13 (2) hedge-arm
  ruling + (3) pre-push gate recorded.

[Verification]: smoke 180 passed (132.9s) + warmed_session lane 10 passed
(140.5s), exit 0 — worktree core-wt-infra @ base 9a428d84.
This commit is contained in:
Shay 2026-07-22 22:28:55 -07:00
parent 9a428d8466
commit 4bf8f3be99
5 changed files with 229 additions and 17 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
AGENTS.md
View file

@ -263,6 +263,7 @@ To optimize server resources and bypass external CI billing dependencies, all ag
uv run core test --suite smoke -q
```
Ensure all smoke tests pass (parity with the smoke gate is pinned by `test_cli_smoke_suite_covers_ci_smoke_gate`). Pushing broken code is a critical protocol violation.
- **Automated pre-push hook (2026-07-22):** `sh scripts/hooks/install.sh` installs a `core.hooksPath` pre-push gate that runs the `smoke` suite **plus** the `warmed_session` consistency lane pin (`tests/test_warmed_session_lane.py`). The lane pin catches the T13 telemetry-consistency regression class that `smoke` does **not** cover (the #96 fail-closed resolve + #97 morph override escaped the smoke files and surfaced only in the warmed_session lane). The full ~12k fast-lane stays **async CI** by design; the hook is deliberately targeted so it blocks the regression class without gridlocking the push cycle. Emergency bypass (discouraged): `git push --no-verify`.
- **Pre-Merge Gate:** Before proposing a merge or requesting a review on a PR, you **MUST** run the larger validation suite relevant to your changes (e.g. `uv run core test --suite cognition` or `uv run core test --suite algebra`).
- **No Docker CI for merge:** Do not run, wait on, or re-provision Docker-container CI to green a merge. Fix and prove in-worktree; merge on local evidence.
- **PR Documentation:** When creating a PR on Forgejo (via the Forgejo MCP tools), document the local test execution in the PR description, matching this format:

51
docs/analysis/weekly-audit-2026-07-22-stragglers-todo.md
View file

@ -144,17 +144,29 @@ fail-closed determinism). Execution status:
### New findings from the execution pass
- [ ] **T10 (fixed in `fix/test-engine-state-isolation`, needs merge): the test
- [x] **T10 (FIXED + MERGED — PR #100 @ `80100c18`): the test
suite wrote the live life-store.** Module-scoped fixtures (e.g.
`tests/test_achat.py`) construct `ChatRuntime()` BEFORE the function-scoped
isolation fixture exists (pytest sets up wider scopes first), binding the
real `engine_state/`: smoke runs loaded it and committed generations
(turn_count 14989→14990 observed). Fix: session-scoped baseline fixture +
regression pin; `docs/issues/default-engine-state-test-hygiene.md` addendum.
- [ ] **T11 (ruling needed): live-store provenance review.** Historical
`turn_count` (14990) includes an unknown number of pre-fix test turns and
the 465 persisted discovery candidates predate the isolation fix. Decide:
accept-as-lived vs audit/relabel candidates.
- [x] **T11 (RULED + EXECUTED 2026-07-22): live-store provenance reset.**
DECISIVE FINDING: candidates carry NO timestamp / run-id / test-prod field
(only a `source_turn_trace` hash + `review_state`), and gen-21701 == gen-21702
were byte-identical — "the 465" was the ENTIRE live candidate corpus, not a
separable tainted subset, so a surgical per-record quarantine (the original
`TEST_TAINTED` spec) was impossible. The candidates were inert (proposal /
contemplation only, never the serving surface). **Ruling (Shay): Option (a) —
clear, don't asterisk.** Executed via
`scripts/ops/reset_candidate_corpus_t11_20260722.py`: an ADR-0219 atomic
generation reset (begin → carry recognizers/session/identity+turn_count →
empty candidate ledger → commit `keep=1`). Result: candidates 465 → 0,
`turn_count` 14990 preserved, gen-21701/21702 physically GC'd, `current` →
gen-21703 @ `9a428d84`. Idle-tick discovery repopulates organically.
**`turn_count` itself left as-is** (honest lived count; the pre-fix test turns
are not separably identifiable, and resetting the counter would fabricate a
number — the candidate corpus was the provenance hazard, not the scalar).
- [ ] **T12 (pattern watch): experiment scaffolding scope over-reach.** The T6
glob hazard suggests a cheap architecture pin: `rnd`/scripts corpus readers
should be lint-checked against sealed/practice eval paths.
@ -183,8 +195,8 @@ fail-closed determinism). Execution status:
surface and `trace_hash` in the durable stream. Deferred (default sink is
`None`, so no live consumer is affected today); own PR when a sink consumer
lands.
- [x] **T13 (main regression, serving-provenance — telemetry half FIXED):
fast lane is red on main** (was 0-red 2026-07-16):
- [x] **T13 (main regression, serving-provenance — telemetry FIXED + MERGED #101):
fast lane was red on main** (was 0-red 2026-07-16):
`tests/test_warmed_session_lane.py::TestPipelineOverrideGateInvariants::test_telemetry_consistency_rate_is_one`
fails at 0.9444 (17/18). Verified pre-existing on unmodified main (fails
plain AND state-isolated; unrelated to the T10 isolation fix). Root cause:
@ -206,20 +218,25 @@ fail-closed determinism). Execution status:
not cover this lane, and GitHub Actions are dead signals — nothing gates
fast-lane red on main right now.
**Resolution (2026-07-22, PR `fix/telemetry-serve-boundary`):**
**Resolution (2026-07-22, PR #101 `fix/telemetry-serve-boundary` @ `9a428d84`, MERGED):**
- **(1) FIXED** — `ChatRuntime.finalize_turn_surface` back-stamps the
pipeline's resolved surface onto `turn_log[-1]`, a sibling to
`finalize_turn_trace_hash`, called from `pipeline.py` immediately after
the trace-hash back-stamp. Lane green: `telemetry_consistency_rate`
0.9444 → 1.0 (10/10 warmed_session tests pass). `trace_hash` byte-identity
preserved — it folds the pre-decoration surface, not the served surface.
- **(2) OPEN — needs Shay's ruling.** Orthogonal to the red (telemetry now
matches whatever is served, refusal or not). Recommend **AGAINST** a
query-type "definitional/epistemic" classifier bypass: that fails a
geometric coherence gate OPEN on a lexical cue — the fail-open / cue-table
pattern ADR-0252 retired and INV-34 / fail-closed forbid. Principled
alternative: route open-geometry-but-**pack-grounded** surfaces to the
existing hedge-injection arm (`authoritative=False`, honestly hedged),
discriminated by grounding *provenance* (structural), not question type.
Own focused PR + real-data (not synthetic) validation.
- **(2) RULED (Shay, 2026-07-22): hedge-arm routing, own PR.** The query-type
"definitional/epistemic" classifier bypass is REJECTED — it fails a
geometric coherence gate OPEN on a lexical cue (fail-open / cue-table;
ADR-0252 + INV-34). Ruling: fix the READING, not the question — route
open-geometry-but-**pack-grounded** surfaces to the existing hedge-injection
arm (`authoritative=False`, honestly hedged), discriminated by grounding
*provenance* (structural), never by question type. Its own dedicated PR +
GSM8K-style (real-data) validation. Scoped next; deliberately NOT in the
telemetry PR or the infra PR.
- **(3) RULED (Shay, 2026-07-22): local pre-push gate — IMPLEMENTED.**
`scripts/hooks/pre-push` + `scripts/hooks/install.sh` (`core.hooksPath`) run
the smoke suite **plus** the warmed_session lane pin on every push — the
targeted gate (NOT the ~9-min full fast-lane, which stays async CI).
AGENTS.md §Local-First CI Validation Protocol updated. (This PR.)
- Durable-sink residual tracked as **R7** above.

22
scripts/hooks/install.sh Executable file
View file

@ -0,0 +1,22 @@
#!/bin/sh
# Install the CORE local-first pre-push gate for THIS clone.
#
# Points git at the tracked hooks directory (scripts/hooks) via core.hooksPath.
# The relative path resolves per-worktree, so every linked worktree gets the
# same gate. Idempotent — safe to re-run. Run once after cloning:
#
# sh scripts/hooks/install.sh
#
# Uninstall: git config --unset core.hooksPath
set -eu
repo_root="$(git rev-parse --show-toplevel)"
cd "${repo_root}"
chmod +x scripts/hooks/pre-push
git config core.hooksPath scripts/hooks
echo "[install] core.hooksPath -> scripts/hooks"
echo "[install] pre-push gate active: smoke suite + warmed_session lane pin."
echo "[install] emergency bypass (discouraged): git push --no-verify"

52
scripts/hooks/pre-push Executable file
View file

@ -0,0 +1,52 @@
#!/bin/sh
# CORE local-first pre-push gate (weekly-audit T13, 2026-07-22).
#
# Automates the AGENTS.md §Local-First CI Validation Protocol "Pre-Push Gate":
# (1) the `smoke` suite — exact CI-gate parity, and
# (2) the warmed_session consistency lane pin — catches the T13
# telemetry-consistency regression class, which `smoke` does NOT cover
# (the #96 fail-closed resolve + #97 morph override escaped smoke and
# surfaced only in the warmed_session lane).
# The full ~12k fast-lane stays an ASYNC CI concern by design — this gate is
# deliberately targeted so it prevents the regression class without gridlocking
# the push cycle.
#
# Install (per clone): scripts/hooks/install.sh (sets core.hooksPath).
# Emergency bypass: git push --no-verify (defeats the gate — avoid).
set -u
z40="0000000000000000000000000000000000000000"
read_any=0
only_deletes=1
while read -r _local_ref local_sha _remote_ref _remote_sha; do
read_any=1
if [ "${local_sha}" != "${z40}" ] && [ -n "${local_sha}" ]; then
only_deletes=0
fi
done
# Fail-closed: run the gate unless we positively identified a deletion-only push.
if [ "${read_any}" -eq 1 ] && [ "${only_deletes}" -eq 1 ]; then
echo "[pre-push] deletion-only push — nothing to validate."
exit 0
fi
echo "[pre-push] CORE local-first gate: smoke suite + warmed_session lane pin"
echo "[pre-push] (1/2) smoke suite (core test --suite smoke) ..."
if ! uv run core test --suite smoke -q; then
echo "[pre-push] BLOCKED: smoke suite failed. Fix before pushing." >&2
echo "[pre-push] (emergency bypass, discouraged: git push --no-verify)" >&2
exit 1
fi
echo "[pre-push] (2/2) warmed_session consistency lane ..."
if ! uv run python -m pytest tests/test_warmed_session_lane.py -q; then
echo "[pre-push] BLOCKED: warmed_session lane failed (T13 regression class)." >&2
echo "[pre-push] (emergency bypass, discouraged: git push --no-verify)" >&2
exit 1
fi
echo "[pre-push] gate PASSED — push proceeding."
exit 0

120
scripts/ops/reset_candidate_corpus_t11_20260722.py Normal file
View file

@ -0,0 +1,120 @@
#!/usr/bin/env python
"""One-off: reset the live discovery-candidate corpus (weekly-audit T11 ruling).
Weekly audit 2026-07-22, Ruling T11 (Shay): the 465 discovery candidates in the
live engine_state store were generated while pre-#100 smoke runs could WRITE the
live store (the module-scope conftest escape #100 sealed). They are provenance-
tainted. They are *inert* — candidates feed only the config-gated proposal /
contemplation loop, never the serving surface — but Absolute Provenance does not
tolerate synthetically-tainted records lingering in a production epistemic corpus.
Ruling: Option (a) — do not asterisk the ledger, clear it. Let idle-tick
discovery repopulate the corpus organically.
WHY THIS SCRIPT AND NOT `rm`:
The store is an ADR-0219 atomic-generation checkpoint. A raw file delete would
itself violate the checkpoint contract (torn state, orphaned pointer). This
routes the reset through the store's own two-phase commit so the wipe is itself
an atomic, auditable committed generation:
1. begin_generation() -> allocate the next gen dir
2. carry forward recognizers + lived session_state + manifest (identity
lineage + turn_count UNCHANGED — only the candidate ledger is reset)
3. save_discovery_candidates([]) -> the wipe (empty ledger)
4. commit_generation(keep=1) -> atomic `current` pointer swap, then GC
every older generation so the tainted candidate files are PHYSICALLY gone
(keep=1 is deliberate: the point of the reset is to eliminate the taint,
not retain it as a rollback generation).
Idempotent-ish guard: refuses to run unless the live store holds exactly the
EXPECTED_TAINTED count, so it cannot be misfired against a repopulated store.
EXECUTED once, 2026-07-22, against the live store @ 9a428d8466c3:
pre: candidates=465 recognizers=0 session_state=absent turn_count=14990
post: candidates=0 turn_count=14990 remaining_generations=[gen-21703]
(gen-21701 / gen-21702 GC'd; identity lineage c9e5968a… preserved)
Retained in-tree as the provenance record of the reset.
"""
from __future__ import annotations
import os
import sys
from pathlib import Path
from engine_state import EngineStateStore
EXPECTED_TAINTED = 465 # the audited pre-#100 candidate count
def main() -> int:
store_dir = os.environ.get("CORE_ENGINE_STATE_DIR")
store = EngineStateStore(Path(store_dir)) if store_dir else EngineStateStore()
print(f"[T11] target store: {store.path}")
if not store.exists():
print("[T11] ABORT: no committed checkpoint at target store.", file=sys.stderr)
return 2
candidates = store.load_discovery_candidates()
manifest = store.load_manifest() or {}
recognizers = store.load_recognizers()
session_state = store.load_session_state()
turn_count = int(manifest.get("turn_count", 0))
print(
f"[T11] pre-reset: candidates={len(candidates)} recognizers={len(recognizers)} "
f"session_state={'present' if session_state is not None else 'absent'} "
f"turn_count={turn_count}"
)
if len(candidates) != EXPECTED_TAINTED:
print(
f"[T11] ABORT: expected {EXPECTED_TAINTED} tainted candidates, found "
f"{len(candidates)}. Refusing to run against an unexpected store.",
file=sys.stderr,
)
return 3
# --- atomic generation-dir reset (ADR-0219) -------------------------------
gen_num, gen_dir = store.begin_generation()
gen_store = EngineStateStore(gen_dir)
gen_store.save_recognizers(recognizers) # carry forward (faithful)
gen_store.save_discovery_candidates([]) # THE WIPE
if session_state is not None:
gen_store.save_session_state(session_state) # carry forward lived state
gen_store.save_manifest(
turn_count, # UNCHANGED
engine_identity=manifest.get("engine_identity", ""),
parent_engine_identity=manifest.get("parent_engine_identity", ""),
identity_scheme=int(manifest.get("identity_scheme", 2)),
)
store.commit_generation(gen_num, keep=1) # swap + GC every older gen
print(f"[T11] committed reset generation gen-{gen_num:04d} (keep=1)")
# --- verify ---------------------------------------------------------------
verify = EngineStateStore(store.path)
post_candidates = verify.load_discovery_candidates()
post_manifest = verify.load_manifest() or {}
post_turn = int(post_manifest.get("turn_count", -1))
remaining_gens = sorted(
p.name for p in store.path.iterdir() if p.is_dir() and p.name.startswith("gen-")
)
ok = (
len(post_candidates) == 0
and post_turn == turn_count
and remaining_gens == [f"gen-{gen_num:04d}"]
)
print(
f"[T11] post-reset: candidates={len(post_candidates)} turn_count={post_turn} "
f"remaining_generations={remaining_gens}"
)
if not ok:
print("[T11] VERIFY FAILED — inspect the store manually.", file=sys.stderr)
return 4
print("[T11] OK — candidate corpus cleared; lived state + identity + turn_count intact.")
return 0
if __name__ == "__main__":
raise SystemExit(main())