fix(tests): isolate xdist polluters blocking -n auto default (W3) #46

Merged
core-labs merged 4 commits from fix/xdist-polluter-isolation into main 2026-07-16 13:53:15 +00:00

4 commits

Author SHA1 Message Date
Shay
df8d35c12f build: commit uv.lock — the locked universe CI now installs
All checks were successful
smoke / smoke (-m "not quarantine") (pull_request) Successful in 5m53s
lane-shas / verify pinned lane SHAs (pull_request) Successful in 17m53s
#48 switched CI to 'uv sync --locked' but uv.lock was gitignored (an early
library-style choice; this repo is an application — the lock belongs in
git). Every run since failed at install: 'Unable to find lockfile'.
Un-ignore and commit the lock (217 packages, matches pyproject via
'uv lock --check'). CI now installs the exact universe local development
and all green verification ran against.

[Verification]: code tree byte-identical to the smoke-gated state
(175 passed); this commit adds a data file CI requires + one .gitignore
line removal.
2026-07-16 06:14:49 -07:00
Shay
fbb4c26e1a Merge remote-tracking branch 'forgejo/main' into tmp-46-update
Some checks failed
smoke / smoke (-m "not quarantine") (pull_request) Failing after 5m46s
lane-shas / verify pinned lane SHAs (pull_request) Failing after 16m48s
2026-07-16 06:00:50 -07:00
Shay
c0cc8c40c4 fix(tests): isolate 4th polluter victim caught in live -n auto run
Some checks failed
smoke / smoke (-m "not quarantine") (pull_request) Successful in 11m38s
lane-shas / verify pinned lane SHAs (pull_request) Failing after 1h39m15s
tests/test_teaching_queue.py::test_read_only_invariant hardcoded
`project_root / "engine_state"` (the real shared dir) in its read-only-
invariant snapshot, instead of reading `engine_state._DEFAULT_DIR` (the
root-conftest autouse fixture's per-test isolated dir). Same bug pattern
as test_workbench_replay.py::test_replay_leaves_no_trace, and this one
was caught red-handed: a full `-n auto` fast-lane run failed with
"Directory .../engine_state was mutated!" and passed cleanly in serial
isolation, the exact order-dependent-flake signature this wave hunts.

The other three snapshotted dirs (teaching/proposals, packs,
contemplation/runs) have no equivalent per-test isolation idiom in the
repo yet and are left as-is — this test's own hitl-queue commands don't
write to them, so they were not implicated in the observed failure.
2026-07-15 16:46:08 -07:00
Shay
531d4aa0d1 fix(tests): isolate three xdist polluter clusters blocking -n auto default
Hunt for the -n auto fast-lane polluters flagged in docs/testing-lanes.md
("Follow-up: xdist by default"). Three root causes, all shared-repo-path
writers with no per-test isolation, following the #782 monkeypatch idiom:

1. tests/test_workbench_replay.py::test_replay_leaves_no_trace hardcoded
   `Path("engine_state")` (the real shared dir) instead of reading
   `engine_state._DEFAULT_DIR`, which the root-conftest autouse fixture
   already redirects per-test. It was a victim, not a polluter: any
   concurrent worker writing the real dir made this snapshot-diff flake.
   Fixed by reading `engine_state._DEFAULT_DIR` dynamically.

2. evals/gsm8k_math/train_sample/v1/runner.py hardcoded its report.json
   output to the committed repo path with no override. Two test files
   (test_rat1_end_to_end_admission.py, test_wave_a_multiplicative_
   aggregation_injector.py; 4 tests total) spawn it as a subprocess and
   read the same file back — a write race under -n auto, and a confirmed
   downstream victim (test_gsm8k_sealed_attempt_scout.py::
   test_report_json_mtime_unchanged_by_scout_import asserts the file's
   mtime is stable). Added an optional CORE_GSM8K_TRAIN_SAMPLE_REPORT_PATH
   env override (default unchanged) and pointed the 4 call sites at
   tmp_path.

3. core/cli.py's `_DEMO_RESULTS_DIR` (evals/forward_semantic_control/
   results/) is written, glob-scanned, and index.json-rebuilt by ~11 tests
   across tests/test_cli_demo.py's TestDemoSubcommand and TestDemoPreambles
   classes. Added an autouse fixture monkeypatching `cli._DEMO_RESULTS_DIR`
   to a per-test tmp dir. This also unmasked a latent order-dependent
   coupling: test_demo_list_results_indexes_reports and
   test_demo_list_results_json_well_formed never wrote their own report,
   relying on a sibling test's leftover file in the shared dir (silently
   correct only because pytest ran the file in declaration order). Made
   both self-contained.

No assertion weakening, no test deletion, no global autouse fixture masking
real bugs. All three fixes preserve the real (non-test) default behavior
byte-for-byte when unpatched/env-unset.

Verification: 8x targeted -n 8 loop over the 7 affected files (before and
after) did not force-reproduce the underlying race live (narrow timing
window, small-scale run) — confirmation is source-level (hardcoded shared
paths bypassing the established isolation idiom) plus the prior documented
flake for test_replay_leaves_no_trace in docs/testing-lanes.md. Full
fast-lane -n auto run recorded in the PR description.
2026-07-15 16:46:08 -07:00