core/docs/testing-lanes.md

Testing lanes — fast / slow / full

The full pytest suite is ~10,600 tests and ~73 min serial. A small set of heavyweight tests dominates that wall-clock, so we classify them and offer a fast lane for local development. Classification is empirical test-infrastructure metadata, so it lives in one auditable place (conftest.py), beside the QUARANTINE registry — not as @pytest.mark.slow decorators spread across ~24 files.

Lanes

Lane	Command	What it runs
fast	make test-fast → pytest -m "not quarantine and not slow"	everything except the slow registry
slow	make test-slow → pytest -m "slow and not quarantine"	only the heavyweight registry
full	make test-full → pytest -m "not quarantine"	everything (what CI runs)

The marker is classification only — it never skips. -m slow selects the slow tests; you choose a lane with an explicit marker expression. Plain pytest (no -m) still runs the full suite.

CI is unchanged: .github/workflows/smoke.yml and full-pytest.yml both run -m "not quarantine", which includes the slow tests — so the split costs no CI coverage.

Measured timings (10-core macOS, CORE_BACKEND=numpy)

Lane	Serial	Parallel (-n auto)
full	73 min	25 min
fast	~26 min	9.5 min (9,590 passed)

Combined (split + parallel) = 73 → 9.5 min (7.7×). The parallel fast lane scales ~5.7× because it excludes the 975s parallel-floor monster (see below); the full suite only reaches 2.9× because that one test pins a worker for 16 min.

-n auto is not wired into the make targets yet — see Follow-up: xdist.

The slow registry (conftest.py)

Two registries, by cost shape:

• SLOW_FILES — whole-file: the cost is carried by a module/session-scoped fixture, so marking one test is insufficient (skipping it just shifts the fixture cost to the next test that requests it). 10 files.
• SLOW_TESTS — exact nodeids: mixed files where only specific tests are soak/bench scale; the file's fast predicate/unit tests stay in the fast lane. 26 tests across 14 files.

Honest accounting — the registry marks 912 of 10,596 tests slow. 801 of those are test_cognition_eval_register_matrix.py (a per-register × invariant eval matrix: many cheap parametrized assertions gated behind expensive per-register module-fixture setups). It is classified whole-file because the cost is in the module fixture, but be aware the fast lane therefore omits the register-matrix coverage; CI's full lane still runs it.

Finding: the 975s test_inner_loop_phase2 outlier

test_inner_loop_phase2.py::TestCausalAttribution::test_null_control_matches_boundary_only showed a 975s (16 min) setup — the single largest test, and the parallel floor for the whole suite.

Probed: it is expected proof-scale work, not a bug or runaway. The cost is a module-scoped phase2_report fixture that runs the FSC corpus (9 cases: 1 public/v1 + 8 dev) through run_lane, which executes 4 conditions + 4 determinism reruns = 8 full real-runtime pipeline turns per case, plus a fresh ChatRuntime() per case (~5s each). 9 × 8 heavy pipeline turns ≈ 975s. The fixture is shared across the file's 5 tests, so the cost is paid once.

A possible optimization exists — share the primed runtime across the 4 conditions instead of reconstructing — but it touches the runner's determinism contract, so it is deferred, not done here.

Follow-ups (separate PRs)

1. xdist by default. The fast/full lanes are not xdist-hermetic yet: fresh-env-dict subprocess tests (tests/formation/*, test_identity_packs) write to the repo engine_state/ dir, and other tests write evals/.../report.json and teaching/proposals/ — these race under parallel workers (e.g. test_workbench_replay::test_replay_leaves_no_trace fails under -n auto, passes serially). Isolate those writers, then wire -n auto into make test-fast / test-full. This is the same hermeticity theme as docs/issues/default-engine-state-test-hygiene.md.
2. Warm-runtime fixture. The fast lane's remaining ~9.5 min (parallel) is a long tail of 1–15s ChatRuntime constructions, not outliers. A shared/session-scoped warm-runtime fixture for read-only tests would cut this further.

Recommended determinism / teaching regression invocation (post-Claim-B hardening of CLOSE yardstick)

After any change touching CLOSE flywheel, idle_tick, realize_derived, consolidate_determinations, vault recall of realized facts, determine(), or the derived close proposal bridge, run the hardened yardstick as part of your determinism regression and anti-regression verification:

uv run python -m evals.close_derived_climb
uv run python -m pytest tests/test_derived_close_proposals.py tests/test_architectural_invariants.py -q

(Also available via core demo anti-regression which now embeds the yardstick — see below.)

This is the canonical "standard verification story" invocation for the CLOSE autonomous growth surface. It is the direct follow-up to the Claim-B hardening (#791) and makes the improved measurement recurring rather than isolated.

What the hardened yardstick now exercises and measures (Claim B):

• Real ChatRuntime.idle_tick() + IdleTickResult.derived_close_proposals_emitted (proposal flag gating via the lived runtime path, not a simulation).
• Explicit determine() calls on the post-fixed-point positive probes, asserting Determined(True, rule='direct') ("semantic_positives_determined_direct").
• content_replay_checksum covering canonical closure sets (with structure_key, Derivation, and premise_structure_keys) and full proposal bodies for exact-trajectory fidelity.
• Retained Claim A guarantees: strict/monotone growth (1/5/8), wrong_total == 0, negatives and excluded predicates refused, full determinism, hermetic (no serving, no ratification, SPECULATIVE/proposal_only only, all INV-30/31 etc. preserved).

See:

• evals/close_derived_climb/contract.md (metrics, scenarios, "no side effects")
• docs/analysis/close-derived-climb-yardstick-claim-b-ratification-2026-06-16.md (the hardening ratification)
• docs/analysis/integrate-hardened-close-yardstick-determinism-teaching-regression-ratification-2026-06-16.md (this integration ratification + "why only correct path")
• docs/runtime_contracts.md (determination surface contract exercised by the semantic asserts)
• docs/evals/anti_regression_demo.md (the anti-regression demo now runs the yardstick)
• tests/test_anti_regression_demo.py (contract test that pins the embedding)
• Makefile (comments under test lanes point here)

The yardstick itself remains hermetic per the rules in this document (fresh runtimes, internal temps only for proposal sink during flag test). It introduces no new writers to engine_state/, teaching/proposals/, or evals reports.

This integration (documentation promotion into the lanes + hermetic execution inside the anti-regression demo) is the highest-leverage way to ensure the project actually benefits from the hardened Claim-B measurement surface on every relevant regression run.