assessment: independent Tier-S verification, ratification ceremony, local-first CI runner (recovered from cloud session) #113
3 changed files with 196 additions and 0 deletions
|
|
@ -264,6 +264,8 @@ To optimize server resources and bypass external CI billing dependencies, all ag
|
|||
```
|
||||
Ensure all smoke tests pass (parity with the smoke gate is pinned by `test_cli_smoke_suite_covers_ci_smoke_gate`). Pushing broken code is a critical protocol violation.
|
||||
- **Automated pre-push hook (2026-07-22):** `sh scripts/hooks/install.sh` installs a `core.hooksPath` pre-push gate that runs the `smoke` suite **plus** the `warmed_session` consistency lane pin (`tests/test_warmed_session_lane.py`). The lane pin catches the T13 telemetry-consistency regression class that `smoke` does **not** cover (the #96 fail-closed resolve + #97 morph override escaped the smoke files and surfaced only in the warmed_session lane). The full ~12k fast-lane stays **async CI** by design; the hook is deliberately targeted so it blocks the regression class without gridlocking the push cycle. Emergency bypass (discouraged): `git push --no-verify`.
|
||||
- **One-command local CI (2026-07-25):** `sh scripts/ci/local-ci.sh [--tier smoke|gate|full]` runs the tiers from a single entry point. `gate` is the three pre-push steps; `full` runs the whole `tests/` tree in parallel. Suite membership is read from `core/cli_test.py::TEST_SUITES` through the CLI — never restated — so the runner cannot drift from the hook.
|
||||
- **Interpreter contract.** `pyproject.toml` pins `requires-python == "3.12.13"` exactly, so on a host without that patch version `uv sync --locked` fails and *every* gate becomes unrunnable. The runner is fail-closed about this: it refuses and tells you to `uv python install 3.12.13`. `--allow-interpreter-fallback` opts into any 3.12.x and stamps the run **NON-CANONICAL** on every line — a degraded run is legitimate, a degraded run reporting itself as the real thing is not. Only a canonical run is merge evidence.
|
||||
- **Pre-Merge Gate:** Before proposing a merge or requesting a review on a PR, you **MUST** run the larger validation suite relevant to your changes (e.g. `uv run core test --suite cognition` or `uv run core test --suite algebra`).
|
||||
- **No Docker CI for merge:** Do not run, wait on, or re-provision Docker-container CI to green a merge. Fix and prove in-worktree; merge on local evidence.
|
||||
- **PR Documentation:** When creating a PR on Forgejo (via the Forgejo MCP tools), document the local test execution in the PR description, matching this format:
|
||||
|
|
|
|||
185
scripts/ci/local-ci.sh
Executable file
185
scripts/ci/local-ci.sh
Executable file
|
|
@ -0,0 +1,185 @@
|
|||
#!/bin/sh
|
||||
# CORE local-first CI runner.
|
||||
#
|
||||
# AGENTS.md: "The real CI is local-first. Run validation in the worktree...
|
||||
# Do not treat remote Actions / Docker job containers as the merge gate."
|
||||
# This script is that runner — one entry point for the gate tiers, usable on
|
||||
# any host, with an honest statement of which interpreter produced the result.
|
||||
#
|
||||
# sh scripts/ci/local-ci.sh # gate tier (what pre-push runs)
|
||||
# sh scripts/ci/local-ci.sh --tier smoke
|
||||
# sh scripts/ci/local-ci.sh --tier full # the whole tests/ tree, parallel
|
||||
# sh scripts/ci/local-ci.sh --list
|
||||
#
|
||||
# WHY THIS EXISTS
|
||||
# ---------------
|
||||
# `pyproject.toml` pins `requires-python == "3.12.13"` — an EXACT patch
|
||||
# version. On a host that does not have precisely 3.12.13, `uv sync --locked`
|
||||
# fails outright and every gate becomes unrunnable. For a project whose merge
|
||||
# bar IS the local run, an unrunnable local gate is the failure mode that
|
||||
# matters most: it is how `tests/test_lane_sha_verifier.py` sat red on clean
|
||||
# main from the deduction-serve arc until 2026-07-25 with nobody noticing.
|
||||
#
|
||||
# THE INTERPRETER CONTRACT (the part to read before trusting a green run)
|
||||
# ----------------------------------------------------------------------
|
||||
# The pinned interpreter is CANONICAL. A run on any other interpreter is
|
||||
# NON-CANONICAL and this script says so, loudly, on every such run.
|
||||
#
|
||||
# Fallback is opt-in, never automatic — the same discipline as
|
||||
# `core.ratified_ledger`'s `missing_ok`: a degraded mode is legitimate, and a
|
||||
# degraded mode that reports itself as the real thing is not. Default is
|
||||
# fail-closed with instructions.
|
||||
#
|
||||
# --allow-interpreter-fallback run on any 3.12.x, stamped NON-CANONICAL
|
||||
#
|
||||
# Measured note, so the flag is not cargo-culted: on 2026-07-25 the full
|
||||
# committed-SHA / trace_hash / content_sha256 / lane-SHA pin set (119 tests)
|
||||
# was run on 3.12.11 and 118 passed — the single failure was a stale lane
|
||||
# roster that fails on 3.12.13 too. That is evidence the exact pin is not
|
||||
# load-bearing for bit-exactness, NOT a ruling that it should be relaxed.
|
||||
# Relaxing it is a reproducibility decision and belongs to a human.
|
||||
|
||||
set -eu
|
||||
|
||||
repo_root="$(git rev-parse --show-toplevel)"
|
||||
cd "${repo_root}"
|
||||
|
||||
TIER="gate"
|
||||
ALLOW_FALLBACK=0
|
||||
PYTEST_EXTRA=""
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
usage: local-ci.sh [--tier TIER] [--allow-interpreter-fallback] [--list] [-- ARGS]
|
||||
|
||||
tiers:
|
||||
smoke smoke suite only (~1 min) = CI smoke.yml parity
|
||||
gate smoke + warmed_session + deductive (~2 min) = the pre-push gate
|
||||
full the entire tests/ tree, parallel (~10 min) = the merge bar
|
||||
|
||||
options:
|
||||
--allow-interpreter-fallback permit a non-pinned 3.12.x; marks run NON-CANONICAL
|
||||
--list print the tier definitions and exit
|
||||
-- pass remaining args through to pytest
|
||||
EOF
|
||||
}
|
||||
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--tier) TIER="$2"; shift 2 ;;
|
||||
--tier=*) TIER="${1#*=}"; shift ;;
|
||||
--allow-interpreter-fallback) ALLOW_FALLBACK=1; shift ;;
|
||||
--list) usage; exit 0 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
--) shift; PYTEST_EXTRA="$*"; break ;;
|
||||
*) echo "[local-ci] unknown argument: $1" >&2; usage >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
case "${TIER}" in
|
||||
smoke|gate|full) ;;
|
||||
*) echo "[local-ci] unknown tier: ${TIER} (smoke|gate|full)" >&2; exit 2 ;;
|
||||
esac
|
||||
|
||||
# --- interpreter resolution ------------------------------------------------
|
||||
|
||||
pinned="$(cat .python-version 2>/dev/null || echo '')"
|
||||
[ -n "${pinned}" ] || pinned="3.12.13"
|
||||
canonical=0
|
||||
RUNNER=""
|
||||
|
||||
if command -v uv >/dev/null 2>&1 && uv python find "${pinned}" >/dev/null 2>&1; then
|
||||
canonical=1
|
||||
RUNNER="uv run --"
|
||||
echo "[local-ci] interpreter: ${pinned} (CANONICAL — matches the repo pin)"
|
||||
else
|
||||
if [ "${ALLOW_FALLBACK}" -eq 0 ]; then
|
||||
cat >&2 <<EOF
|
||||
[local-ci] BLOCKED: the pinned interpreter (${pinned}) is not available here.
|
||||
|
||||
\`pyproject.toml\` pins requires-python == "${pinned}", so \`uv sync --locked\`
|
||||
cannot resolve and the canonical gate cannot run on this host.
|
||||
|
||||
Either provision it: uv python install ${pinned}
|
||||
or run degraded, knowingly:
|
||||
sh scripts/ci/local-ci.sh --tier ${TIER} \\
|
||||
--allow-interpreter-fallback
|
||||
|
||||
A degraded run is legitimate. A degraded run reported as canonical is not,
|
||||
which is why this is opt-in rather than automatic.
|
||||
EOF
|
||||
exit 3
|
||||
fi
|
||||
fallback=""
|
||||
for cand in python3.12 python3; do
|
||||
if command -v "${cand}" >/dev/null 2>&1 && \
|
||||
"${cand}" -c 'import sys; sys.exit(0 if sys.version_info[:2]==(3,12) else 1)' 2>/dev/null; then
|
||||
fallback="${cand}"; break
|
||||
fi
|
||||
done
|
||||
if [ -z "${fallback}" ]; then
|
||||
echo "[local-ci] BLOCKED: no 3.12.x interpreter found at all." >&2
|
||||
exit 3
|
||||
fi
|
||||
actual="$(${fallback} -c 'import sys; print(".".join(map(str, sys.version_info[:3])))')"
|
||||
RUNNER="${fallback} -m"
|
||||
cat >&2 <<EOF
|
||||
|
||||
############################################################
|
||||
# NON-CANONICAL RUN #
|
||||
# interpreter ${actual} != pinned ${pinned}
|
||||
# Results are indicative, NOT merge evidence. #
|
||||
# Re-run on ${pinned} before merging.
|
||||
############################################################
|
||||
|
||||
EOF
|
||||
fi
|
||||
|
||||
# Two invocation shapes, one interface. Suite membership is NEVER restated
|
||||
# here — it is read from ``core/cli_test.py::TEST_SUITES`` through the CLI, so
|
||||
# this runner cannot drift from the pre-push gate the way smoke.yml drifted
|
||||
# from it.
|
||||
if [ "${canonical}" -eq 1 ]; then
|
||||
suite() { uv run core test --suite "$1" -q; }
|
||||
pytest_() { uv run python -m pytest "$@"; }
|
||||
else
|
||||
suite() { ${RUNNER} core.cli test --suite "$1" -q; }
|
||||
pytest_() { ${RUNNER} pytest "$@"; }
|
||||
fi
|
||||
|
||||
step() {
|
||||
label="$1"; shift
|
||||
echo "[local-ci] ${label} ..."
|
||||
if ! "$@"; then
|
||||
echo "[local-ci] FAILED: ${label}" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# --- gates -----------------------------------------------------------------
|
||||
|
||||
start="$(date +%s)"
|
||||
|
||||
case "${TIER}" in
|
||||
smoke)
|
||||
step "smoke suite" suite smoke
|
||||
;;
|
||||
gate)
|
||||
echo "[local-ci] tier=gate — same three steps as scripts/hooks/pre-push"
|
||||
step "(1/3) smoke suite" suite smoke
|
||||
step "(2/3) warmed_session lane" pytest_ tests/test_warmed_session_lane.py -q --no-header
|
||||
step "(3/3) deductive suite" suite deductive
|
||||
;;
|
||||
full)
|
||||
jobs="$(nproc 2>/dev/null || echo 4)"
|
||||
step "full tests/ tree (-n ${jobs})" \
|
||||
pytest_ tests/ -q --no-header -p no:cacheprovider -n "${jobs}" --dist loadfile
|
||||
;;
|
||||
esac
|
||||
|
||||
elapsed=$(( $(date +%s) - start ))
|
||||
if [ "${canonical}" -eq 1 ]; then
|
||||
echo "[local-ci] PASSED tier=${TIER} in ${elapsed}s — CANONICAL (${pinned})."
|
||||
else
|
||||
echo "[local-ci] PASSED tier=${TIER} in ${elapsed}s — NON-CANONICAL; not merge evidence." >&2
|
||||
fi
|
||||
|
|
@ -92,6 +92,15 @@ class TestExpectedLaneCoverage:
|
|||
"curriculum_loop_closure",
|
||||
"math_teaching_corpus_v1", # ADR-0131
|
||||
"deductive_logic_v1", # ADR-0206 — independent-oracle entailment lane
|
||||
# Added 2026-07-25. Both lanes shipped during the deduction-serve
|
||||
# generalization arc and neither was added here, so this roster's
|
||||
# `extra` tripwire — which exists precisely to make a new lane an
|
||||
# explicit acknowledgement rather than a silent addition — has been
|
||||
# RED on clean main ever since. It went unnoticed because this file
|
||||
# is in neither `smoke` nor `deductive`; nothing local or in CI runs
|
||||
# it. This entry is the acknowledgement the tripwire was asking for.
|
||||
"deduction_serve_v1", # ADR-0256
|
||||
"curriculum_serve_v1", # ADR-0262
|
||||
}
|
||||
)
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue