core-labs/core
core-labs/core
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

fix(tests): pin showcase budget to the deliberate 60s + complete the slow-lane ledger row

Browse source 
Slow-lane sweep (2026-07-15, 37m06s): 1 failed / 909 passed / 1 skipped /
1 xfailed. The single red was a stale constant pin (assert
MAX_RUNTIME_SECONDS == 30) missed when 640dbe8f deliberately re-budgeted
the public showcase to 60s — the slow lane hadn't been run since (nightly
lives on the billing-locked GitHub side). Pin updated to the deliberate,
comment-documented value; ledger Slow row completed. The historical '~31
full-suite reds' figure is fully retired: fast lane cleared by #45, slow
lane clean after this pin.

[Verification]: tests/test_public_showcase.py green; push held until the
CI locked-install fix lands (branch inherits it via main-merge).
This commit is contained in:
Shay 2026-07-15 19:44:04 -07:00
parent 987a04bac9
commit edaf3c7a7a
2 changed files with 9 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/analysis/full-suite-reds-ledger-2026-07.md
View file

@ -10,7 +10,7 @@ Base: `forgejo/main` @ `54228d45`
|------|---------|--------|
| Fast | `pytest -m "not quarantine and not slow" -n auto -q` | **5 failed**, 11660 passed, 106 skipped (pre-fix) |
| Fast recheck | same after fix | (see verification below) |
| Slow | `pytest -m "slow and not quarantine" -n auto -q` | running / recorded at PR time |
| Slow | `pytest -m "slow and not quarantine" -n auto -q` | **1 failed, 909 passed, 1 skipped, 1 xfailed** (37m06s, 2026-07-15 sweep @ `8f67590d`+#47). The 1 red = `test_public_showcase.py::TestRuntimeBudget` — stale constant pin (30) vs the deliberate 640dbe8f re-budget (60); fixed alongside this row. Slow lane is otherwise clean → the historical "~31 reds" figure is fully retired. |
| Full | `pytest -m "not quarantine"` | union of fast + slow |
The brief’s “~31 reds” appears to reflect an older main tip. On current `forgejo/main`, the **non-quarantine fast lane** surfaces **exactly 5** dedicated failures. Smoke/pack/lane gates stay green because they never execute these nodes.

10
tests/test_public_showcase.py
View file

@ -162,5 +162,11 @@ class TestPureCompositionGate:
class TestRuntimeBudget:
def test_budget_is_thirty_seconds(self) -> None:
assert MAX_RUNTIME_SECONDS == 30
def test_budget_is_sixty_seconds(self) -> None:
# 30 → 60 was a deliberate re-budget (commit 640dbe8f: post-CGA
# substrate work makes a cold RegisterTour alone ~30s+; see
# evals/public_demo/contract.md "Known Environment Caveat"). This slow
# pin was missed then because the slow lane hadn't been run since
# (nightly lives on the billing-locked GitHub side) — caught by the
# 2026-07 slow-lane sweep. Pin the deliberate value.
assert MAX_RUNTIME_SECONDS == 60