Commit graph

2 commits

Author SHA1 Message Date
Shay
4d26e1503b
fix(tests): make frontier_compare viewer test resilient to copy refreshes (#67)
test_frontier_compare_report_viewer_exists was failing on main against
the current report_viewer.html because two verbatim substring checks
no longer matched the viewer's UI copy:

  - "Drop report JSON"  →  viewer now says "Drop JSON report" (order swapped)
  - "No network calls"  →  viewer now says "no network calls" (lowercase)

Both copy refreshes were behavior-preserving — drop-zone affordance
and network-free trust boundary are both intact in the viewer. The
test was coupling to verbatim phrasing rather than to the load-bearing
affordances.

Switch to case-insensitive substring checks that pin what actually
matters:
  - "frontier compare" — viewer identity
  - "drop" AND "json" together — drop-zone affordance, order-independent
  - "no network calls" — trust boundary (case-insensitive)
  - fetch(/XMLHttpRequest still hard-banned (case-sensitive — these
    are JS API surface, not human-readable copy)

Pre-existing failure flagged in PR #66's body as out-of-scope cleanup;
this is that cleanup.
2026-05-20 15:13:38 -07:00
Shay
e64ec578eb
feat(evals): frontier comparison benchmark wave 1 (#52)
* feat(evals): add frontier comparison benchmark wave one scaffold

* feat(evals): add frontier comparison runner package

* feat(evals): implement frontier comparison wave one suites

* feat(evals): add frontier comparison CLI entrypoint

* feat(evals): add static frontier benchmark report viewer

* test(evals): cover frontier comparison wave one benchmarks

* fix(evals): record runtime observation failures instead of aborting suites

* docs(evals): document frontier comparison recording UI
2026-05-20 06:27:32 -07:00