Hunt for the -n auto fast-lane polluters flagged in docs/testing-lanes.md
("Follow-up: xdist by default"). Three root causes, all shared-repo-path
writers with no per-test isolation, following the #782 monkeypatch idiom:
1. tests/test_workbench_replay.py::test_replay_leaves_no_trace hardcoded
`Path("engine_state")` (the real shared dir) instead of reading
`engine_state._DEFAULT_DIR`, which the root-conftest autouse fixture
already redirects per-test. It was a victim, not a polluter: any
concurrent worker writing the real dir made this snapshot-diff flake.
Fixed by reading `engine_state._DEFAULT_DIR` dynamically.
2. evals/gsm8k_math/train_sample/v1/runner.py hardcoded its report.json
output to the committed repo path with no override. Two test files
(test_rat1_end_to_end_admission.py, test_wave_a_multiplicative_
aggregation_injector.py; 4 tests total) spawn it as a subprocess and
read the same file back — a write race under -n auto, and a confirmed
downstream victim (test_gsm8k_sealed_attempt_scout.py::
test_report_json_mtime_unchanged_by_scout_import asserts the file's
mtime is stable). Added an optional CORE_GSM8K_TRAIN_SAMPLE_REPORT_PATH
env override (default unchanged) and pointed the 4 call sites at
tmp_path.
3. core/cli.py's `_DEMO_RESULTS_DIR` (evals/forward_semantic_control/
results/) is written, glob-scanned, and index.json-rebuilt by ~11 tests
across tests/test_cli_demo.py's TestDemoSubcommand and TestDemoPreambles
classes. Added an autouse fixture monkeypatching `cli._DEMO_RESULTS_DIR`
to a per-test tmp dir. This also unmasked a latent order-dependent
coupling: test_demo_list_results_indexes_reports and
test_demo_list_results_json_well_formed never wrote their own report,
relying on a sibling test's leftover file in the shared dir (silently
correct only because pytest ran the file in declaration order). Made
both self-contained.
No assertion weakening, no test deletion, no global autouse fixture masking
real bugs. All three fixes preserve the real (non-test) default behavior
byte-for-byte when unpatched/env-unset.
Verification: 8x targeted -n 8 loop over the 7 affected files (before and
after) did not force-reproduce the underlying race live (narrow timing
window, small-scale run) — confirmation is source-level (hardcoded shared
paths bypassing the established isolation idiom) plus the prior documented
flake for test_replay_leaves_no_trace in docs/testing-lanes.md. Full
fast-lane -n auto run recorded in the PR description.
Replaces the W-026 501 stub. Re-executes a journaled prompt in a sealed
fresh runtime (ChatRuntime(no_load_state=True): no checkpoint load, no
checkpoint write, no proposal lineage) and compares the envelope
leaf-by-leaf against the recorded TurnJournalEntry.
Design correction vs the scoping doc (amended in-doc): journaled turns
each ran in a fresh ChatRuntime(), never one continuous session, so
genesis-PREFIX replay would manufacture spurious divergence; shipped
basis is sealed_fresh_runtime_single_turn (O(1)). origin_state:
"unrecorded" — the journal does not record whether the original turn's
runtime loaded a checkpoint, so divergence is reported as nondeterminism
OR origin-state influence, never disambiguated.
- workbench/replay.py: pure comparison + injected executor; every
TurnJournalEntry field classified critical/informational exactly once
(exhaustiveness enforced by test)
- api.py: route wiring under _CHAT_TURN_LOCK; runtime failure -> 500
runtime_unavailable, no comparison may be fabricated
- schemas: additive TurnReplayComparison/TurnReplayDivergence (W-026
artifact-keyed pair retires with the frontend Replay Moment PR)
- tests: 10 obligations incl. tamper-prompt, tamper-leaf precision,
no-execution-no-comparison, no-trace (journal + engine_state bytes),
wall-clock tolerance, sealed-construction proof
- snapshot regenerated; NOT_YET_MIRRORED debt entries for the two new
classes (mirrors land with the frontend PR)
- api-contract-v1.md § Replay rewritten for the turn-keyed shape
