fix(tests): make frontier_compare viewer test resilient to copy refreshes (#67)
test_frontier_compare_report_viewer_exists was failing on main against
the current report_viewer.html because two verbatim substring checks
no longer matched the viewer's UI copy:
- "Drop report JSON" → viewer now says "Drop JSON report" (order swapped)
- "No network calls" → viewer now says "no network calls" (lowercase)
Both copy refreshes were behavior-preserving — drop-zone affordance
and network-free trust boundary are both intact in the viewer. The
test was coupling to verbatim phrasing rather than to the load-bearing
affordances.
Switch to case-insensitive substring checks that pin what actually
matters:
- "frontier compare" — viewer identity
- "drop" AND "json" together — drop-zone affordance, order-independent
- "no network calls" — trust boundary (case-insensitive)
- fetch(/XMLHttpRequest still hard-banned (case-sensitive — these
are JS API surface, not human-readable copy)
Pre-existing failure flagged in PR #66's body as out-of-scope cleanup;
this is that cleanup.
This commit is contained in:
parent
dedf05565d
commit
4d26e1503b
1 changed files with 13 additions and 3 deletions
|
|
@ -81,8 +81,18 @@ def test_frontier_compare_report_viewer_exists() -> None:
|
|||
viewer = Path("evals/frontier_compare/ui/report_viewer.html")
|
||||
text = viewer.read_text(encoding="utf-8")
|
||||
|
||||
assert "Frontier Compare" in text
|
||||
assert "Drop report JSON" in text
|
||||
assert "No network calls" in text
|
||||
# Case-insensitive substring checks — pin the viewer's load-bearing
|
||||
# affordances and trust-boundary copy without coupling to verbatim
|
||||
# phrasing. Earlier verbatim checks ("Drop report JSON",
|
||||
# "No network calls") broke when the viewer's copy was refreshed
|
||||
# to "Drop JSON report" / "no network calls" without any change in
|
||||
# behavior.
|
||||
lowered = text.lower()
|
||||
assert "frontier compare" in lowered
|
||||
# Drop-zone affordance: order-independent — both tokens must be
|
||||
# present so a user can paste a report into the viewer.
|
||||
assert "drop" in lowered and "json" in lowered
|
||||
# Trust boundary: viewer must remain network-free.
|
||||
assert "no network calls" in lowered
|
||||
assert "fetch(" not in text
|
||||
assert "XMLHttpRequest" not in text
|
||||
|
|
|
|||
Loading…
Reference in a new issue