From edaf3c7a7aa5c7346b7b7457aa88efa27d345b54 Mon Sep 17 00:00:00 2001 From: Shay Date: Wed, 15 Jul 2026 19:44:04 -0700 Subject: [PATCH] fix(tests): pin showcase budget to the deliberate 60s + complete the slow-lane ledger row MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Slow-lane sweep (2026-07-15, 37m06s): 1 failed / 909 passed / 1 skipped / 1 xfailed. The single red was a stale constant pin (assert MAX_RUNTIME_SECONDS == 30) missed when 640dbe8f deliberately re-budgeted the public showcase to 60s — the slow lane hadn't been run since (nightly lives on the billing-locked GitHub side). Pin updated to the deliberate, comment-documented value; ledger Slow row completed. The historical '~31 full-suite reds' figure is fully retired: fast lane cleared by #45, slow lane clean after this pin. [Verification]: tests/test_public_showcase.py green; push held until the CI locked-install fix lands (branch inherits it via main-merge). --- docs/analysis/full-suite-reds-ledger-2026-07.md | 2 +- tests/test_public_showcase.py | 10 ++++++++-- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/docs/analysis/full-suite-reds-ledger-2026-07.md b/docs/analysis/full-suite-reds-ledger-2026-07.md index 12b92c2e..1cfac10e 100644 --- a/docs/analysis/full-suite-reds-ledger-2026-07.md +++ b/docs/analysis/full-suite-reds-ledger-2026-07.md @@ -10,7 +10,7 @@ Base: `forgejo/main` @ `54228d45` |------|---------|--------| | Fast | `pytest -m "not quarantine and not slow" -n auto -q` | **5 failed**, 11660 passed, 106 skipped (pre-fix) | | Fast recheck | same after fix | (see verification below) | -| Slow | `pytest -m "slow and not quarantine" -n auto -q` | running / recorded at PR time | +| Slow | `pytest -m "slow and not quarantine" -n auto -q` | **1 failed, 909 passed, 1 skipped, 1 xfailed** (37m06s, 2026-07-15 sweep @ `8f67590d`+#47). The 1 red = `test_public_showcase.py::TestRuntimeBudget` — stale constant pin (30) vs the deliberate 640dbe8f re-budget (60); fixed alongside this row. Slow lane is otherwise clean → the historical "~31 reds" figure is fully retired. | | Full | `pytest -m "not quarantine"` | union of fast + slow | The brief’s “~31 reds” appears to reflect an older main tip. On current `forgejo/main`, the **non-quarantine fast lane** surfaces **exactly 5** dedicated failures. Smoke/pack/lane gates stay green because they never execute these nodes. diff --git a/tests/test_public_showcase.py b/tests/test_public_showcase.py index 2171dc04..962bc007 100644 --- a/tests/test_public_showcase.py +++ b/tests/test_public_showcase.py @@ -162,5 +162,11 @@ class TestPureCompositionGate: class TestRuntimeBudget: - def test_budget_is_thirty_seconds(self) -> None: - assert MAX_RUNTIME_SECONDS == 30 + def test_budget_is_sixty_seconds(self) -> None: + # 30 → 60 was a deliberate re-budget (commit 640dbe8f: post-CGA + # substrate work makes a cold RegisterTour alone ~30s+; see + # evals/public_demo/contract.md "Known Environment Caveat"). This slow + # pin was missed then because the slow lane hadn't been run since + # (nightly lives on the billing-locked GitHub side) — caught by the + # 2026-07 slow-lane sweep. Pin the deliberate value. + assert MAX_RUNTIME_SECONDS == 60