d94ea7e2d1
Fixes span three subsystems:
math parser / OOD generator:
- Add OOD unit registry words (ingots, shards, crystals, …) to
allowed_nouns so rename_unit variants parse cleanly
- Add scarf/scarves and other -ves→-f irregulars to _PLURAL_IRREGULARS
so _canonical_unit("scarf") → "scarves" (not "scarfs")
- Add _IRREGULAR_SINGULAR dict to _singular() in ood_surface_generator
so "scarves" → "scarf" for n=1 rendering; prevents "scarve" parse error
eval lane drift:
- cold_start_grounding public cases: update 4 expected_grounding_source
values from "pack"/"oov" → "teaching" (cognition chains now cover
truth/memory/recall for DEFINITION prompts)
- gsm8k_math runner: handle fast-path graph=None (capacity/earnings
solvers return is_admitted=True with selected_graph=None)
- coverage probe report: regenerate committed JSON after parser fix
raised admission_rate and changed per_case trace hashes
- test_gsm8k_math_runner: add decoded_unarticulated / _rate to
expected metrics key set
test guards:
- test_composed_surface + test_compound_walkthrough_eval_lanes: skip
holdout-split tests when CORE_HOLDOUT_KEY unset (not a regression)
- test_en_core_action_v1_pack: EXPECTED_TOTAL 26→27, issubset check,
provenance in-check for pack that gained one inflected entry
- test_relations_chains_v1: EXPECTED_CHAIN_IDS 7→21 after seed expansion
conftest: QUARANTINE frozenset emptied — ratchet at zero.
44 lines
1.7 KiB
Python
44 lines
1.7 KiB
Python
"""Project-root conftest — quarantine registry for known-failing tests.
|
|
|
|
The QUARANTINE set lists test IDs that are pre-existing failures
|
|
predating the substrate-liveness audit work (verified via bisect
|
|
against c1a1b7a, the commit immediately before the first W-* PR
|
|
of 2026-05-24). The CI gate at .github/workflows/full-pytest.yml
|
|
runs ``pytest -m "not quarantine"`` so these failures do not block
|
|
PRs, but the suite is a ratchet: a quarantined test removed from
|
|
this set must pass on its own merits.
|
|
|
|
See docs/test-debt-quarantine.md for cluster diagnoses, removal
|
|
policy, and the per-test rationale.
|
|
|
|
To remove a test from quarantine:
|
|
1. Land a PR that makes the test pass.
|
|
2. Delete its entry from QUARANTINE in the same PR.
|
|
3. The full-pytest CI gate will now require it to keep passing.
|
|
|
|
Adding a test to QUARANTINE is strongly discouraged. If a new
|
|
failure surfaces, the right default is to fix it in the PR that
|
|
caused it — not to quarantine. The set should only shrink.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
|
|
QUARANTINE: frozenset[str] = frozenset()
|
|
|
|
|
|
def pytest_collection_modifyitems(config, items):
|
|
"""Stamp the quarantine marker on any test whose nodeid is listed
|
|
in QUARANTINE. Tests not in the set are unaffected.
|
|
|
|
The CI gate runs ``pytest -m "not quarantine"`` so quarantined
|
|
tests are silently skipped in CI. Local ``pytest`` runs include
|
|
them by default (still useful for debugging individual fixes).
|
|
"""
|
|
_ = config # pluggy hook signature requires the name `config`; not used here
|
|
quarantine_marker = pytest.mark.quarantine
|
|
for item in items:
|
|
if item.nodeid in QUARANTINE:
|
|
item.add_marker(quarantine_marker)
|