576c2e13e3
Root cause of the simultaneous lane-shas failures on PRs #46 and #47: workflows installed with unlocked 'uv pip install -e .' against floor-only specs (numpy>=1.26, ...) while the repo maintains uv.lock — so a PyPI release landing between runs shifts the runner's resolved set, and the byte-exact lane SHA pins move with it. Timeline: main lane run green at 23:40Z, both PR runs (diffs no lane executes) red after 00:02Z; the identical trees verify 9/9 locally against the locked venv, and 'uv lock --check' is clean. All six workflows now 'uv sync --locked' (--extra dev where [dev] was installed) — CI tests the locked universe, so lane pins can only move when code moves. Locked-sync also fails loudly on a stale lock instead of silently resolving fresh. Self-validating: this PR's own lane-shas run uses this branch's workflow. [Verification]: smoke suite passed locally (175 passed); lane repro on the failing PRs' tree: 9/9 pinned SHAs; uv lock --check clean.
113 lines
4 KiB
YAML
113 lines
4 KiB
YAML
name: lane-shas
|
|
|
|
# Verify that every ADR-0092..0104 lane produces its pinned SHA-256
|
|
# report. A failing job means a lane's deterministic output changed
|
|
# without an explicit ADR-tracked pin update via:
|
|
#
|
|
# python scripts/verify_lane_shas.py --update
|
|
#
|
|
# Single source of truth for the pinned values is scripts/verify_lane_shas.py.
|
|
#
|
|
# PR path policy (job-level, not workflow-level):
|
|
# Pin bytes can move from Python, packs, eval fixtures/corpora, teaching
|
|
# corpora, dependency pins, CLAIMS.md, or this workflow. When none of those
|
|
# paths change on a PR, we skip the multi-minute runners and exit success so
|
|
# a required check never sits "Waiting" forever (workflow-level `paths:`
|
|
# would omit the job entirely and hang required status checks).
|
|
# Main pushes always verify.
|
|
|
|
on:
|
|
push:
|
|
branches: [main]
|
|
pull_request:
|
|
branches: [main]
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
concurrency:
|
|
group: lane-shas-${{ github.ref }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
verify:
|
|
name: verify pinned lane SHAs
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 45
|
|
|
|
steps:
|
|
- name: checkout
|
|
uses: actions/checkout@v4
|
|
with:
|
|
# Need base..head for PR path detection.
|
|
fetch-depth: 0
|
|
|
|
- name: detect pin-relevant paths
|
|
id: paths
|
|
shell: bash
|
|
run: |
|
|
set -euo pipefail
|
|
# Always verify on main pushes.
|
|
if [ "${{ github.event_name }}" = "push" ]; then
|
|
echo "run=true" >> "$GITHUB_OUTPUT"
|
|
echo "Pin-relevant path check: always run on push"
|
|
exit 0
|
|
fi
|
|
|
|
BASE="${{ github.event.pull_request.base.sha }}"
|
|
HEAD="${{ github.event.pull_request.head.sha }}"
|
|
# Paths that can change pin bytes or CLAIMS generation inputs.
|
|
# Keep in sync with docs/testing-lanes.md § CI policy.
|
|
PATTERN='(\.py$|^packs/|^evals/|^teaching/|^CLAIMS\.md$|^pyproject\.toml$|^uv\.lock$|^\.github/workflows/lane-shas\.yml$)'
|
|
if git diff --name-only "$BASE" "$HEAD" | grep -E "$PATTERN" >/dev/null; then
|
|
echo "run=true" >> "$GITHUB_OUTPUT"
|
|
echo "Pin-relevant paths changed; running lane SHA verification"
|
|
git diff --name-only "$BASE" "$HEAD" | grep -E "$PATTERN" || true
|
|
else
|
|
echo "run=false" >> "$GITHUB_OUTPUT"
|
|
echo "No pin-relevant paths changed; skipping lane SHA verification (job still green)"
|
|
fi
|
|
|
|
# setup-uv (not actions/setup-python) provisions Python on the aarch64
|
|
# self-hosted runner; actions/setup-python has no arm64 build for the
|
|
# pinned 3.12.13. Matches smoke.yml / full-pytest.yml.
|
|
- name: set up uv
|
|
if: steps.paths.outputs.run == 'true'
|
|
uses: astral-sh/setup-uv@v5
|
|
with:
|
|
python-version: '3.12.13'
|
|
enable-cache: true
|
|
|
|
- name: install dependencies
|
|
if: steps.paths.outputs.run == 'true'
|
|
run: |
|
|
uv sync --locked
|
|
|
|
- name: verify lane SHAs
|
|
if: steps.paths.outputs.run == 'true'
|
|
env:
|
|
PYTHONPATH: ${{ github.workspace }}
|
|
# public_demo wall-clock is soft by default (see evals/public_demo/runner.py).
|
|
# Do not set CORE_SHOWCASE_HARD_BUDGET here — cold Act runners exceed 60s.
|
|
# Content cases (claims, determinism, pure composition) remain hard gates.
|
|
run: |
|
|
uv run python scripts/verify_lane_shas.py
|
|
|
|
- name: verify CLAIMS.md is current
|
|
if: steps.paths.outputs.run == 'true'
|
|
env:
|
|
PYTHONPATH: ${{ github.workspace }}
|
|
run: |
|
|
uv run python scripts/generate_claims.py --check
|
|
|
|
- name: emit machine-readable report (on failure)
|
|
if: failure() && steps.paths.outputs.run == 'true'
|
|
env:
|
|
PYTHONPATH: ${{ github.workspace }}
|
|
run: |
|
|
uv run python scripts/verify_lane_shas.py --json || true
|
|
|
|
- name: skip notice
|
|
if: steps.paths.outputs.run != 'true'
|
|
run: |
|
|
echo "::notice title=lane-shas skipped::No pin-relevant paths in this PR; verification skipped (success)."
|