Compare commits

..

8 commits

Author SHA1 Message Date
Shay
44f7258b16 fix(algebra): P11a physics hot paths via algebra.backend (Rust-ready)
Some checks failed
smoke / smoke (-m "not quarantine") (pull_request) Successful in 5m57s
lane-shas / verify pinned lane SHAs (pull_request) Failing after 1h43m22s
Stop wave/Third-Door physics from bypassing native dispatch:

- Route geometric_product / versor_apply / versor_condition / cga_inner
  through algebra.backend in wave_manifold, goldtether, trajectory,
  dynamic_manifold, surprise, holographic_vault, atlas_packing,
  biography, self_authorship.
- Backend: dtype-aware Rust use — f32 workloads use core_rs; f64 wave
  residual pins keep Python SOT until f64 GP parity exists. Coerce
  arrays for PyO3 bindings; fail soft to Python.
- AST hygiene pin: tests/test_physics_backend_dispatch_hygiene.py
- Docs: RUST.md, runtime_contracts, fidelity (ADR-0235 / UMA hygiene).

Verified: wave + cohesion suites green default and CORE_BACKEND=rust
(with core_rs built). MLX still exploratory off-serve.
2026-07-14 21:08:02 -07:00
Shay
db6430ed4e feat(adr-0242): macro-phase V2–V5 + sensorium feed (Drive gap close)
Parallel implementation of remaining Drive ADR-0242 vectors on PR #38:

D3 V2 — multi_scale_energy: E_n(t)=E0·exp(-age/(F_n·τ0)), dyadic baseline,
        comparative helpers; not production FieldEnergyOperator default.
D5 V4 — fibonacci_word_schedule: W0=B,W1=A,W_{n+1}=W_n W_{n-1}; telemetry only.
D6 V5 — algebra/topological_reasoning quarantine + production AST pin.
D7    — sensorium_wave_feed: packet→ψ, superpose, ρ via WaveManifold only.

Serve quarantine extended for all new modules. Fidelity + ADR-0242 +
runtime_contracts honesty pass. Suite: 118 related tests green.

Single macro-phase commit per plan policy (no micro-PRs).
2026-07-14 21:08:02 -07:00
Shay
bbd3b6678f feat(adr-0242): Drive V1 cert discipline + doc align five vectors
Close the gap between cohesion packing/search and Drive ADR-0242:

D0 — Expand ADR-0242 to five-vector + sovereignty thesis (title matches Drive).
D8 — Land docs/analysis/fibonacci_applications_in_core_substrate.md.
D1 — FibonacciSearchCertificate | OptimizationFailure (never bare float);
     content-addressed cert_id; dual-run stable digest.
D2 — propose_kappa_from_search / goldtether.propose_kappa_line_search;
     failure → baseline κ=1.0 (no state mutation).
D4 — ALLOCATOR_VERSION golden_angle_v1 + layout descriptor.

Fidelity §12 honest: V1/V3 green; V2 table-only; V4/V5 staged.
2026-07-14 21:08:02 -07:00
Shay
9d543f6a9c docs(governance): P12 cohesion close — contracts, checklist, ready-for-accept
ADR-0241 cohesion package P12 (governance):
- runtime_contracts: off-serve quarantine, holographic SPECULATIVE/COHERENT,
  hypothesis vs evidence reconstruct, crystallization, entity suite pins.
- docs/audit/adr_0241_cohesion_acceptance_checklist.md maps C0–C8 → tests.
- ADRs remain Proposed + ready for Joshua acceptance (no self-Accept).
- Fidelity §12 honesty: P0–P10 green; human Accepted is the gate.
- tests/test_adr_0241_governance_p12.py pins inventory + status honesty.

CLAIMS.md Tier-2 lanes unchanged (I-01…I-05 are suite pins, not lane SHAs).
2026-07-14 21:08:02 -07:00
Shay
f123e0ea75 feat(wave): P10 Trace B energy boundary + multi-scale Fibonacci τ
Land ADR-0241 cohesion package P10:
- core/physics/wave_energy_boundary.py wires WaveManifold unitary residual
  into energy profiles and trajectory energy gates (no free-floating residual).
- fibonacci_tau_schedule: τ_n = F_n · τ_0 constants table + recency bands.
- crystallization_for_holographic_seal: E0–E1 + closed residual may
  SPECULATIVE-seal; dirty residual or hot classes refuse.
- Public fibonacci_number; serve quarantine includes wave_energy_boundary.
- Fidelity + ADR-0241 status updated.

Lane: pytest tests/test_adr_0241_wave_energy_boundary.py
      tests/test_adr_0242_fibonacci.py tests/test_energy.py
      tests/test_third_door_cohesion.py → green.
2026-07-14 21:08:02 -07:00
Shay
aa86f1ae35 feat(wave): P9 Trace A contemplation → SPECULATIVE holographic seal
Land ADR-0241 cohesion package P9:
- core/contemplation/wave_seam.py seals standing-wave modes only via
  HolographicVaultStore.seal_mode (SPECULATIVE); no seal_mode_reviewed,
  no direct VaultStore.store (INV-21).
- FindingKind.RESONANT_MODE_CANDIDATE + ContemplationFinding for teaching
  review corridor.
- reconstruct_as_hypothesis (full spectrum) vs reconstruct_as_evidence
  (min_status=COHERENT only; SPECULATIVE cannot masquerade).
- HolographicVaultStore.resonant_reconstruct with status filter.
- Serve quarantine extended to wave_seam; fidelity + ADR-0241 status.

Lane: pytest tests/test_adr_0241_wave_contemplation_seam.py
      tests/test_adr_0241_holographic_vault.py
      tests/test_third_door_cohesion.py → 38 passed.
2026-07-14 21:08:02 -07:00
b80f262f1d Merge pull request 'ci: fast-lane main gate, nightly full suite, skip-safe lane-shas' (#39) from optimize-ci-test-suites into main
Some checks failed
full-pytest / fast pytest (-m "not quarantine and not slow" -n 2) (push) Failing after 59s
lane-shas / verify pinned lane SHAs (push) Successful in 17m25s
ci: land multi-lane CI policy — fast on main, full nightly, skip-safe lane-shas
2026-07-15 04:04:22 +00:00
Shay
82f9b021fb ci: fast-lane main gate, nightly full suite, skip-safe lane-shas
Some checks failed
smoke / smoke (-m "not quarantine") (pull_request) Failing after 53s
lane-shas / verify pinned lane SHAs (pull_request) Failing after 32m58s
Unblock the single Act runner: post-merge full-pytest runs the fast
marker (not quarantine and not slow); full non-quarantine suite moves
to nightly + workflow_dispatch. lane-shas uses job-level path detection
so docs-only PRs skip green without hanging required checks. Sync
smoke/conftest/docs CI contracts to the PR/main/nightly policy.
2026-07-14 21:02:43 -07:00
7 changed files with 209 additions and 130 deletions

View file

@ -1,20 +1,25 @@
name: full-pytest
# Post-merge gate — runs the **fast** lane on every push to main.
# Post-merge FAST lane — runs on every push to main.
#
# Why not full+slow here:
# Host is a 2vCPU / ~4GiB VM shared with Forgejo + Postgres + Traefik.
# Job containers were capped at 1.5 CPU / 1.2GiB. Running the entire
# ~12k-test suite including the slow registry (one 16min fixture floor)
# monopolized the single Act runner for hours and starved PR smoke /
# lane-shas ("Waiting to run").
# Marker: -m "not quarantine and not slow"
# ~9.5k unit/integration tests; excludes the slow registry in conftest.py
# (soak / bench / proof / register-matrix; ~912 tests including the 16 min
# phase2 fixture floor).
#
# Coverage contract:
# - This workflow: ``not quarantine and not slow`` (make test-fast).
# - Slow/soak/proof matrix: ``nightly-full-pytest.yml`` (schedule + manual).
# - PR gate remains smoke.yml (minutes) + path-filtered lane-shas.yml.
# Why "full-pytest" still names this file:
# Keep the workflow id stable for Forgejo required-check / history matching.
# The job display name and this comment state the true contract: FAST on main.
# The complete non-quarantine suite runs in nightly-full-pytest.yml.
#
# See docs/testing-lanes.md and docs/ci-optimization.md.
# PR gate: smoke.yml (small critical subset).
# Lane pins: lane-shas.yml.
# Full soak: nightly-full-pytest.yml (schedule + workflow_dispatch).
#
# See:
# conftest.py — QUARANTINE + SLOW_FILES / SLOW_TESTS registries
# docs/testing-lanes.md — lane commands and CI policy (SSoT)
# docs/ci-optimization.md — runner bottleneck + capacity notes
on:
push:
@ -29,9 +34,9 @@ concurrency:
jobs:
pytest:
name: full pytest (fast lane, not quarantine and not slow)
name: fast pytest (-m "not quarantine and not slow" -n 2)
runs-on: ubuntu-latest
# Fast lane should finish well under this on a healthy runner.
# Headroom over ~9.5 min parallel on a 10-core host; Act 2-vCPU is slower.
timeout-minutes: 30
steps:
@ -50,17 +55,14 @@ jobs:
run: |
uv pip install -e ".[dev]" pyyaml
- name: pytest fast lane (parallel)
- name: pytest (parallel, quarantine and slow excluded)
env:
PYTHONPATH: ${{ github.workspace }}
CORE_SHOWCASE_SKIP_BUDGET: "1"
# Avoid uv hardlink warning noise across filesystems on Act.
UV_LINK_MODE: copy
run: |
# -n 2 matches prior CI; host is 2 vCPU — do not raise without more cores.
uv run pytest -m "not quarantine and not slow" -n 2 --tb=short -q --maxfail=10
- name: report quarantine + slow registry size (informational)
- name: report quarantine size (informational)
if: always()
env:
PYTHONPATH: ${{ github.workspace }}
@ -68,7 +70,6 @@ jobs:
uv run python -c "
import sys
sys.path.insert(0, '.')
from conftest import QUARANTINE, SLOW_FILES, SLOW_TESTS
print(f'::notice title=Quarantine size::{len(QUARANTINE)} tests quarantined.')
print(f'::notice title=Slow registry::{len(SLOW_FILES)} files + {len(SLOW_TESTS)} exact tests classified slow (nightly only).')
from conftest import QUARANTINE
print(f'::notice title=Quarantine size::{len(QUARANTINE)} tests currently quarantined. Goal: shrink this number.')
"

View file

@ -1,6 +1,6 @@
name: lane-shas
# Verify that every ADR-0092..0099 lane produces its pinned SHA-256
# Verify that every ADR-0092..0104 lane produces its pinned SHA-256
# report. A failing job means a lane's deterministic output changed
# without an explicit ADR-tracked pin update via:
#
@ -8,33 +8,19 @@ name: lane-shas
#
# Single source of truth for the pinned values is scripts/verify_lane_shas.py.
#
# PR path filter: skip the multi-lane wall-clock cost when the change cannot
# affect pin bytes (e.g. docs-only, pure physics off the demo path). Main
# still always verifies after merge.
# PR path policy (job-level, not workflow-level):
# Pin bytes can move from Python, packs, eval fixtures/corpora, teaching
# corpora, dependency pins, CLAIMS.md, or this workflow. When none of those
# paths change on a PR, we skip the multi-minute runners and exit success so
# a required check never sits "Waiting" forever (workflow-level `paths:`
# would omit the job entirely and hang required status checks).
# Main pushes always verify.
on:
push:
branches: [main]
pull_request:
branches: [main]
paths:
- "scripts/verify_lane_shas.py"
- "scripts/generate_claims.py"
- "CLAIMS.md"
- "evals/**"
- "demos/**"
- "core/demos/**"
- "chat/**"
- "generate/**"
- "packs/**"
- "teaching/**"
- "vault/**"
- "algebra/**"
- "core/physics/**"
- "core/cognition/**"
- "core/capability/**"
- "pyproject.toml"
- ".github/workflows/lane-shas.yml"
permissions:
contents: read
@ -53,25 +39,54 @@ jobs:
- name: checkout
uses: actions/checkout@v4
with:
fetch-depth: 1
# Need base..head for PR path detection.
fetch-depth: 0
- name: detect pin-relevant paths
id: paths
shell: bash
run: |
set -euo pipefail
# Always verify on main pushes.
if [ "${{ github.event_name }}" = "push" ]; then
echo "run=true" >> "$GITHUB_OUTPUT"
echo "Pin-relevant path check: always run on push"
exit 0
fi
BASE="${{ github.event.pull_request.base.sha }}"
HEAD="${{ github.event.pull_request.head.sha }}"
# Paths that can change pin bytes or CLAIMS generation inputs.
# Keep in sync with docs/testing-lanes.md § CI policy.
PATTERN='(\.py$|^packs/|^evals/|^teaching/|^CLAIMS\.md$|^pyproject\.toml$|^uv\.lock$|^\.github/workflows/lane-shas\.yml$)'
if git diff --name-only "$BASE" "$HEAD" | grep -E "$PATTERN" >/dev/null; then
echo "run=true" >> "$GITHUB_OUTPUT"
echo "Pin-relevant paths changed; running lane SHA verification"
git diff --name-only "$BASE" "$HEAD" | grep -E "$PATTERN" || true
else
echo "run=false" >> "$GITHUB_OUTPUT"
echo "No pin-relevant paths changed; skipping lane SHA verification (job still green)"
fi
# setup-uv (not actions/setup-python) provisions Python on the aarch64
# self-hosted runner; actions/setup-python has no arm64 build for the
# pinned 3.12.13. Matches smoke.yml / full-pytest.yml.
- name: set up uv
if: steps.paths.outputs.run == 'true'
uses: astral-sh/setup-uv@v5
with:
python-version: '3.12.13'
enable-cache: true
- name: install dependencies
if: steps.paths.outputs.run == 'true'
run: |
uv pip install -e . pyyaml pytest
- name: verify lane SHAs
if: steps.paths.outputs.run == 'true'
env:
PYTHONPATH: ${{ github.workspace }}
UV_LINK_MODE: copy
# public_demo wall-clock is soft by default (see evals/public_demo/runner.py).
# Do not set CORE_SHOWCASE_HARD_BUDGET here — cold Act runners exceed 60s.
# Content cases (claims, determinism, pure composition) remain hard gates.
@ -79,14 +94,20 @@ jobs:
uv run python scripts/verify_lane_shas.py
- name: verify CLAIMS.md is current
if: steps.paths.outputs.run == 'true'
env:
PYTHONPATH: ${{ github.workspace }}
run: |
uv run python scripts/generate_claims.py --check
- name: emit machine-readable report (on failure)
if: failure()
if: failure() && steps.paths.outputs.run == 'true'
env:
PYTHONPATH: ${{ github.workspace }}
run: |
uv run python scripts/verify_lane_shas.py --json || true
- name: skip notice
if: steps.paths.outputs.run != 'true'
run: |
echo "::notice title=lane-shas skipped::No pin-relevant paths in this PR; verification skipped (success)."

View file

@ -1,15 +1,22 @@
name: nightly-full-pytest
# Slow/soak lane — the heavyweight registry excluded from post-merge full-pytest.
# Runs on a schedule so it cannot starve PR smoke / lane-shas on the single
# self-hosted Act runner.
# Nightly FULL lane — complete non-quarantine suite including the slow registry.
#
# Manual: Actions → nightly-full-pytest → Run workflow.
# Marker: -m "not quarantine"
# Includes soak / bench / proof / register-matrix (SLOW_FILES + SLOW_TESTS).
# Intentionally off the PR and post-merge critical path so a single 2-vCPU
# Act runner is not held for 12h after every main push (see docs/ci-optimization.md).
#
# Risk owned here: a main merge can break slow tests until the next nightly
# (or a manual workflow_dispatch). Treat red nightlies as release-blocking
# debt; re-run via Actions → nightly-full-pytest → Run workflow after fixes.
#
# See docs/testing-lanes.md for the full CI policy.
on:
schedule:
# 06:00 UTC daily — adjust if runner is shared with other heavy jobs.
- cron: "0 6 * * *"
# 02:00 UTC daily — off peak for human PR iteration.
- cron: '0 2 * * *'
workflow_dispatch:
permissions:
@ -17,12 +24,15 @@ permissions:
concurrency:
group: nightly-full-pytest
cancel-in-progress: true
cancel-in-progress: false
jobs:
pytest-slow:
name: nightly full (not quarantine, includes slow)
pytest:
name: full pytest (-m "not quarantine" -n 2)
runs-on: ubuntu-latest
# Full suite parallel floor includes ~16 min phase2 fixture; thrashing on a
# 2-vCPU Act host has been observed well past 60 min. Prefer a red timeout
# only after a genuine hang, not under normal soak load.
timeout-minutes: 120
steps:
@ -42,11 +52,21 @@ jobs:
run: |
uv pip install -e ".[dev]" pyyaml
- name: pytest full including slow registry
- name: pytest (parallel, full suite, quarantine excluded)
env:
PYTHONPATH: ${{ github.workspace }}
CORE_SHOWCASE_SKIP_BUDGET: "1"
UV_LINK_MODE: copy
run: |
# Single worker on 2vCPU host avoids xdist thrash with 16min fixtures.
uv run pytest -m "not quarantine" -n 1 --tb=short -q --maxfail=20
uv run pytest -m "not quarantine" -n 2 --tb=short -q --maxfail=10
- name: report quarantine size (informational)
if: always()
env:
PYTHONPATH: ${{ github.workspace }}
run: |
uv run python -c "
import sys
sys.path.insert(0, '.')
from conftest import QUARANTINE
print(f'::notice title=Quarantine size::{len(QUARANTINE)} tests currently quarantined. Goal: shrink this number.')
"

View file

@ -7,9 +7,10 @@ name: smoke
# — ratified packs diverge directionally; pack-invariant refusal floor; no
# fabrication). The falsifiability lane adds ~4 min but blocks-on-regression.
#
# Full pytest runs post-merge to main (see full-pytest.yml).
# Regressions caught here block the PR; anything outside the smoke
# suite is caught on main within minutes of merge.
# Post-merge on main: full-pytest.yml runs the FAST lane
# (-m "not quarantine and not slow"). Soak / proof / register-matrix coverage
# is nightly-full-pytest.yml (not on the PR critical path).
# See docs/testing-lanes.md.
on:
pull_request:

View file

@ -1,19 +1,18 @@
"""Project-root conftest — test classification registries.
The QUARANTINE set is the only allowed registry for known-failing tests.
It is currently empty. If it ever contains nodeids, the CI gate at
.github/workflows/full-pytest.yml runs ``pytest -m "not quarantine"``
so those explicitly tracked failures do not block unrelated PRs. The
suite is a ratchet: a quarantined test removed from this set must pass
It is currently empty. If it ever contains nodeids, CI excludes them via
``-m "not quarantine"`` (smoke, full-pytest fast lane, nightly full).
The suite is a ratchet: a quarantined test removed from this set must pass
on its own merits.
See docs/test-debt-quarantine.md for current policy and historical cluster
diagnoses.
diagnoses. See docs/testing-lanes.md for CI lane policy (PR / main / nightly).
To remove a test from quarantine:
1. Land a PR that makes the test pass.
2. Delete its entry from QUARANTINE in the same PR.
3. The full-pytest CI gate will now require it to keep passing.
3. The main fast gate and nightly full gate will both require it to pass.
Adding a test to QUARANTINE is strongly discouraged. If a new
failure surfaces, the right default is to fix it in the PR that
@ -80,13 +79,14 @@ QUARANTINE: frozenset[str] = frozenset()
# a developer run a fast lane locally. Classification adds the ``slow`` marker
# ONLY — it never skips — so ``-m slow`` SELECTS these tests. Choose a lane:
#
# fast lane: pytest -m "not quarantine and not slow" (make test-fast)
# fast lane: pytest -m "not quarantine and not slow" (make test-fast;
# also full-pytest.yml on main)
# slow lane: pytest -m "slow and not quarantine" (make test-slow)
# full lane: pytest -m "not quarantine" (make test-full; CI)
# full lane: pytest -m "not quarantine" (make test-full;
# also nightly-full-pytest.yml)
#
# CI: full-pytest.yml runs the fast lane (``not quarantine and not slow``).
# Nightly workflow runs full including slow. See docs/testing-lanes.md and
# docs/ci-optimization.md.
# CI policy: PR = smoke subset; main = fast lane; nightly = full including
# slow. See docs/testing-lanes.md.
# ---------------------------------------------------------------------------
# Whole-file: the cost is carried by a module/session-scoped fixture, so marking

View file

@ -1,63 +1,77 @@
# CI optimization (single self-hosted Act runner)
# CI optimization — runner bottleneck and lane policy
## Problem (measured 2026-07-15)
Companion to [testing-lanes.md](./testing-lanes.md) (SSoT for markers and
workflow → lane mapping). This note records **why** the multi-lane CI split
exists and what capacity still limits PR queue time.
| Fact | Value |
|------|--------|
| Host | 2 vCPU, ~3.8GiB RAM, shared Forgejo + Postgres + Traefik |
| Job cgroup (was) | `--cpus=1.5 --memory=1200m` |
| Concurrent capacity | 1 job |
| Suite size | ~12.7k tests (`not quarantine`) |
| Post-merge job | `full-pytest` with **slow included** + `-n 2` |
| Observed | One `full-pytest` ran **2+ hours** at ~25% progress; PR **smoke / lane-shas** stayed **Waiting to run** |
## The bottleneck
That is not “slow tests” alone — it is **queue starvation**: one oversized job owns the only runner.
A single self-hosted Act runner (≈2 vCPU / limited RAM) serializes all workflows
that target `ubuntu-latest` for this repo. When post-merge CI ran the **full**
non-quarantine suite (`-m "not quarantine"`) with `-n 2`, the slow registry
dominated wall-clock:
## Order-of-magnitude strategy
- Module fixture floor: `test_inner_loop_phase2` ≈16 min setup alone
- ~912 tests classified `slow` in `conftest.py` (including the register matrix)
- Thrashing under oversubscription can stretch the job toward **12 hours**
### 1. Right-size the default CI lane (workflow) — largest lever
While that job holds the runner, PR gates (`smoke`, `lane-shas`) sit in
**Waiting**. That is a queue problem first, a test-count problem second.
| Lane | When | Marker expression | Intent |
|------|------|-------------------|--------|
| **smoke** | every PR | fixed file list | minutes; merge gate |
| **lane-shas** | PR (path-filtered) + main | pin script | only when pin surface can change |
| **full-pytest (main)** | every push to main | `not quarantine and not slow` | fast post-merge ratchet |
| **nightly-full** | schedule + manual | `not quarantine` (includes slow) | soak / proof / eval-matrix |
## What we changed (in-repo)
Local doc already measured **~7.7×** from “full serial” → “fast + parallel” on a laptop (`docs/testing-lanes.md`). On a 2vCPU VM the absolute times are worse, but **excluding slow** removes the 16minute parallel floor and hundreds of soak tests — typically multi× wall clock and frees the runner for PR gates.
| Change | Effect |
|---|---|
| `full-pytest.yml` → fast marker | Main push runs `not quarantine and not slow`; frees the runner much sooner after merge |
| `nightly-full-pytest.yml` | Full suite (`not quarantine`) daily at 02:00 UTC + `workflow_dispatch`; timeout 120 min |
| `lane-shas.yml` job-level path skip | PR pin verify runs only when pin-relevant paths change; job still green when skipped (no required-check hang) |
| Contract comments | `smoke.yml`, `conftest.py`, `testing-lanes.md` match the real PR / main / nightly split |
### 2. Path-filter expensive PR jobs
Workflow **id** `full-pytest` is kept on purpose so Forgejo history / required-check
names do not break; the job display name is `fast pytest (...)`.
`lane-shas` is multiminute (demos + showcase). Skip on PR when the diff cannot change pin bytes (docs-only, pure tooling, etc.). Main still always verifies after merge.
## What is out of band (ops, not git)
### 3. Runner cgroup (infra)
Host cgroup limits, hung-container cleanup, and adding a second runner are
**VM/ops** actions. They are not encoded in this repository. Document them in
ops runbooks when you change the host; do not treat this file as a substitute.
`infra/runner/config.yaml` job options should match host reality:
## Expected feedback loop after this change
- Prefer **2 CPUs** for the job when the host is 2vCPU (avoid 1.5 + xdist thrash).
- Memory **≤ available** after Forgejo (~800MiB limit on host stack).
- Keep **capacity 1** until a second runner exists — concurrency >1 on 2 cores makes two jobs slower than one.
| Stage | Typical hold |
|---|---|
| Local | Targeted tests; `make test-fast` before push |
| PR | `smoke` (+ `lane-shas` when pin-relevant paths change) |
| Main | Fast `full-pytest` + always-on `lane-shas` |
| Nightly | Full soak including slow registry |
### 4. What will *not* give 100× on this box
This is roughly **hours of runner hold after merge → tens of minutes** on the
fast lane for the same host class — not infinite parallel capacity. Multiple
open PRs still queue on one runner.
- Throwing more xdist workers at a 2vCPU host.
- Running full+slow on every main push.
- Expecting MLX/Rust to speed pytest collection/runtime of the whole suite without suite redesign.
## Capacity still required
True second order of magnitude needs either **more hardware** (second runner / larger VM) or **further suite partitioning** (shard by package, change-based test selection).
1. **Second Act runner or larger VM** — only real fix for concurrent PR + main
execution.
2. **xdist hermeticity** — see follow-ups in `testing-lanes.md` before raising
`-n` further.
3. **Warm-runtime fixture** — long tail of `ChatRuntime` construction in the
fast lane.
4. **Nightly failure signal** — wire Forgejo notification / issue on red
`nightly-full-pytest` so ≤24h slow-break debt is not silent.
5. **Optional path-triggered slow job** — if main must catch soak breaks same-day
without nightly wait, add a non-blocking or path-filtered slow workflow later.
## Local policy (solo branch work)
## Validation
1. Targeted tests for files you touched.
2. Dual-run only if pin surface changed.
3. Full smoke before merge (or trust green smoke if unchanged smoke paths).
4. Never rely on remote `full-pytest` for PR feedback — it is post-merge / nightly.
```bash
# Local parity with main CI
make test-fast
## Related
# Local parity with nightly
make test-full
- `docs/testing-lanes.md` — fast / slow / full classification
- `conftest.py``SLOW_FILES` / `SLOW_TESTS` / `QUARANTINE`
- `.github/workflows/full-pytest.yml`
- `.github/workflows/nightly-full-pytest.yml`
- `.github/workflows/smoke.yml`
- `.github/workflows/lane-shas.yml`
# Lane pins (same as lane-shas job body)
uv run python scripts/verify_lane_shas.py
uv run python scripts/generate_claims.py --check
```

View file

@ -2,34 +2,56 @@
The full pytest suite is ~10,600 tests and ~73 min serial. A small set of
heavyweight tests dominates that wall-clock, so we classify them and offer a
**fast lane** for local development. Classification is empirical
test-infrastructure metadata, so it lives in one auditable place
**fast lane** for local development and post-merge CI. Classification is
empirical test-infrastructure metadata, so it lives in one auditable place
(`conftest.py`), beside the `QUARANTINE` registry — not as `@pytest.mark.slow`
decorators spread across ~24 files.
Runner capacity notes and the single-Act-runner queue story live in
[ci-optimization.md](./ci-optimization.md). **This file is the SSoT for lane
commands and which workflow runs which marker.**
## Lanes
| Lane | Command | What it runs |
|---|---|---|
| **fast** | `make test-fast``pytest -m "not quarantine and not slow"` | everything except the slow registry |
| **slow** | `make test-slow``pytest -m "slow and not quarantine"` | only the heavyweight registry |
| **full** | `make test-full``pytest -m "not quarantine"` | everything (local + **nightly** CI) |
| **full** | `make test-full``pytest -m "not quarantine"` | everything non-quarantine (local + nightly CI) |
The marker is **classification only** — it never skips. `-m slow` *selects* the
slow tests; you choose a lane with an explicit marker expression. Plain
`pytest` (no `-m`) still runs the full suite.
**CI mapping (2026-07-15 optimization):**
## CI policy
| Workflow | Expression | When |
|----------|------------|------|
| `smoke.yml` | fixed smoke file list | every PR |
| `full-pytest.yml` | `not quarantine and not slow` | every push to **main** |
| `nightly-full-pytest.yml` | `not quarantine` (includes slow) | daily schedule + manual |
| `lane-shas.yml` | pin script | main always; PR only if pin-affecting paths change |
| Surface | Workflow | Marker / scope |
|---|---|---|
| **PR** | `smoke.yml` | Fixed critical subset (`not quarantine` within those files) |
| **PR** | `lane-shas.yml` | Pinned ADR lane SHAs + `CLAIMS.md` check; **skipped green** when the PR does not touch pin-relevant paths |
| **main push** | `full-pytest.yml` | **Fast lane** (`not quarantine and not slow`); workflow *id* kept as `full-pytest` for check stability |
| **main push** | `lane-shas.yml` | Always runs (no path skip on push) |
| **nightly / manual** | `nightly-full-pytest.yml` | **Full lane** (`not quarantine`), includes slow registry |
See `docs/ci-optimization.md` for why the single Act runner cannot run full+slow
on every merge without starving PR gates.
### Pin-relevant paths (`lane-shas` PR skip)
On `pull_request`, verification runs only when the base…head diff touches any of:
- `*.py`
- `packs/**`, `evals/**`, `teaching/**`
- `CLAIMS.md`, `pyproject.toml`, `uv.lock`
- `.github/workflows/lane-shas.yml`
Otherwise the job still starts and exits **success** (skip notice) so a required
status check never sits on “Waiting” forever. Do **not** reintroduce workflow-level
`on.pull_request.paths` for this gate without a always-green companion job.
### Owned risk
Slow/soak/proof tests (including the ~16 min phase2 fixture and the register
matrix) are **not** on the PR or post-merge critical path. A merge can break them
until the next nightly (02:00 UTC) or a manual `workflow_dispatch`. Treat a red
nightly as release-blocking debt.
## Measured timings (10-core macOS, `CORE_BACKEND=numpy`)