Production-line brief pack for the workbench-UI implementation wave. Five briefs in five PRs, gated on ADR-0173 (PR #394) landing first. Dependency DAG: ADR-0173 → W1 → { W2 ‖ W4a ‖ W4b } → W3 (after W2 + #393 CompositionClaim handler) Operator profiles assigned per [[feedback-parallel-dispatch-pattern]]: W1 Codex — Vite/TS/Tailwind/shadcn scaffold from ADR-0162 spec W2 Sonnet — read surfaces (ProposalQueue/Card/Detail + StableJsonViewer) W3 Opus — ratification corridor (load-bearing wrong=0; case 0050 pin) W4a Sonnet — replay surfaces (TraceDrawer/ReplayTheater/Diff) W4b Sonnet — eval surfaces (EvalCenter/LaneList/FailureViewer) Each brief carries: dispatch line, reads-required-first list, outcome inventory, hard requirements, tests, deliverables, forbidden surface. Anti-regression invariants enumerated (wrong=0, ADR-0166 no-new-lanes, partition, case 0050, 127.0.0.1, no remote runtime). Includes copy-paste dispatch sequence and a `status` zsh trap for single-line PR monitoring during the wave. Pain-point retirement matrix maps each of the five scoping-brief operator pain points to the wave PR that retires it.
30 KiB
Workbench UI Wave — Brief Pack (W1..W4)
Goal: Stand up the workbench frontend so the math teaching corridor
is operable by one operator at human-realistic throughput. Implements
the wave shape committed in
docs/handoff/WORKBENCH-UI-WAVE-SCOPING.md under the trust boundary
pinned by ADR-0173.
Blocker. Every brief here depends on ADR-0173 W0 landing first (PR #394). Do not dispatch W1..W4 until that PR merges.
Bundling rule. feedback-batch-during-research. This is implementation mode, not research — one PR per wave (W4 splits into two). No spec churn in the implementation PRs; if doctrine needs to move, it moves in its own ADR PR first.
Production-line discipline. feedback-production-line-pattern.
Dispatch lines are copy-paste-ready; each brief opens with a worktree
add line; brief pack is the single source of truth.
Dependency DAG
ADR-0173 (PR #394)
│
▼
W1 — Scaffold (one frontend PR; blocks W2/W3/W4)
│
├──────────┬──────────┐
▼ ▼ ▼
W2 W4a W4b
(read) (replay) (eval)
│
▼
W3 (ratification corridor — needs W2's queue + detail)
Parallel-safe pairs: {W2 ‖ W4a ‖ W4b} once W1 lands. W3 is sequential after W2 because the RatificationCommandPanel is wired to the ProposalDetailPanel that W2 ships.
Recommended sequence (single-operator): W1 → W2 → W3 → W4a → W4b. Recommended sequence (parallel, 2+ operators): W1 → (W2 ‖ W4a ‖ W4b) → W3.
Operator profiles
Per feedback-parallel-dispatch-pattern:
| Wave | Profile | Why |
|---|---|---|
| W1 | Codex (mechanical) | Vite/TS/Tailwind/shadcn scaffold from ADR-0162 spec; deterministic toolchain bootstrap; minimal judgment. |
| W2 | Sonnet (tight-scope) | Frontend work over a fully-specified API contract; TanStack Query + Zustand; semantic color mapping from ADR-0162 ratified enums. |
| W3 | Opus (load-bearing wrong=0 surface) | Same profile as CC-2; ratification dispatch with hazard-pin discipline, partition tests, case 0050 pin. |
| W4a | Sonnet (tight-scope) | Replay surfaces over existing API routes. |
| W4b | Sonnet (tight-scope) | Eval surfaces over existing API routes. |
Brief W1 — Frontend Scaffold
Operator profile: Codex (mechanical; Vite/TS/Tailwind bootstrap)
Branch: feat/workbench-ui-w1-scaffold
Base: origin/main (post-ADR-0173 merge)
Style: Frontend bootstrap. No app routes yet. Tokens + shell + palette + status footer only.
Dispatch
git fetch origin main && \
git worktree add /tmp/wt-wb-ui-w1 origin/main && \
cd /tmp/wt-wb-ui-w1 && \
git checkout -b feat/workbench-ui-w1-scaffold
Reads required FIRST
docs/decisions/ADR-0160-core-workbench-v1.md§"Architecture choice" + §"Trust boundary"docs/decisions/ADR-0162-workbench-design-system.md§"Token namespace", §"Typography", §"Color semantics", §"Motion", §"Keyboard contract", §"Layout shell", §"Component map — v1 must-ship", §"The no-go list", §"Implementation plan — Branch 1"docs/decisions/ADR-0173-workbench-ratification-trust-boundary.md§"Keyboard contract", §"What the workbench MUST NOT do"docs/handoff/WORKBENCH-UI-WAVE-SCOPING.md§"Guardrails"
Outcome
workbench-ui/ directory at repo root, fully bootstrapped per
ADR-0162 §"Implementation plan — Branch 1":
workbench-ui/package.json— pinned deps:- React 18.x, ReactDOM 18.x
- Vite 5.x
- TypeScript 5.x
- Tailwind 3.x
- shadcn-style primitives (Radix UI under shadcn)
- TanStack Query 5.x
- Zustand 4.x (state)
lucide-reactfor icons (bundled, no CDN)
workbench-ui/vite.config.ts— dev server on127.0.0.1, no network proxies to remote hosts, build output todist/.workbench-ui/tsconfig.json— strict TS.workbench-ui/tailwind.config.js+postcss.config.js.workbench-ui/src/design/tokens/tokens.css— every token from ADR-0162 §1–§4 (color, typography, motion, spacing).workbench-ui/src/design/badges/— four badge primitives:EpistemicStateBadge(15 values fromcore/epistemic_state.py)NormativeClearanceBadge(4 values)ReviewStateBadge(4 values fromteaching/proposals.py)GroundingSourceBadge(6 values) Each binds color + glyph + text label per ADR-0162 §3. No color-only encoding.
workbench-ui/src/shell/—WorkbenchShell.tsx(the outer layout)TopBar.tsxLeftNav.tsxStatusFooter.tsxCommandPalette.tsx(focusable via:per ADR-0173)
workbench-ui/src/keyboard/—KeyboardHelpOverlay.tsxpopulated from ADR-0173 §"Keyboard contract"- global
useKeyboard()hook ?opens the help overlayEscdismisses- No
Cmd/Ctrlchords (ADR-0173 constraint)
workbench-ui/src/api/client.ts— fetch wrapper for the existing backend (http://127.0.0.1:<port>); typed againstworkbench/schemas.pyshapes via hand-written TS types insrc/api/types.ts.workbench-ui/dist/— added to root.gitignore.- CI job —
.github/workflows/workbench-ui-build.yml:- triggers on PR touching
workbench-ui/** - runs
npm ci && npm run build - does not commit
dist/ - fails the PR if build fails
- triggers on PR touching
Hard requirements
- Tokens snapshot test. A snapshot test pins the computed token
values from
tokens.cssso future PRs cannot drift them silently. - Fonts bundled locally per ADR-0173 Q4 — no CDN font fetch, no
@import url(...google...). License compliance is W1's responsibility. - Icons bundled locally —
lucide-reactis installed as a dependency; no remote icon fetches. - No app routes yet. W1 ships the shell;
/proposals,/evals,/traceroutes are W2/W3/W4's job. - Backend binds 127.0.0.1. UI verified to refuse cross-origin
requests; CORS configured restrictively in
workbench/server.pyif not already (read current state first; do not relax). - Keyboard help overlay populated from ADR-0173. Verbatim bindings table.
- Accessibility: no icon-only buttons without
aria-label; all badges carry text + shape, not color alone (ADR-0162 no-go list). - No-go list cross-check. Reviewer must verify: no chat-clone bubbles, no "AI thinking" affordances, no glassmorphism, no purple-neon-cyberpunk, no graph-builder canvas, no dashboard splash metrics.
Tests
workbench-ui/src/**/__tests__/*.test.tsxusing Vitest + React Testing Libraryshell.test.tsx— WorkbenchShell renders with TopBar, LeftNav, StatusFootercommand_palette.test.tsx—:focuses palette,Escdismisseskeyboard_help.test.tsx—?opens overlay with ADR-0173 bindingsbadges.test.tsx— all four badge primitives render correct color + glyph + label for every enum valuetokens.snapshot.test.ts— pins computed token bytes
Deliverables
workbench-ui/directory complete per outcome list- root
.gitignoreupdated (one line:workbench-ui/dist/) .github/workflows/workbench-ui-build.yml- Tests green; CI workbench-ui-build job green
- No backend changes
- No changes to
core/,chat/,teaching/,field/,generate/,algebra/,language_packs/
Forbidden
- Adding app routes (W2/W3/W4 territory)
- Adding ratification UI (W3 territory)
- Adding any ProposalQueue/Card/Detail components (W2 territory)
- Adding any TraceDrawer/Replay/Eval components (W4 territory)
- Touching backend code (
workbench/api.py,workbench/readers.py, etc.) - Committing
dist/ - Remote CDN dependencies (fonts, icons, runtime)
Cmd/Ctrlkeyboard chords- Color-only state encoding
- Auto-dismiss toasts for audit events
Brief W2 — Read Surfaces (Proposal Queue)
Operator profile: Sonnet (tight-scope frontend over fully-specified API)
Branch: feat/workbench-ui-w2-proposal-queue
Base: origin/main (post-W1 merge)
Style: Frontend only. Read-only against existing routes.
Dispatch
git fetch origin main && \
git worktree add /tmp/wt-wb-ui-w2 origin/main && \
cd /tmp/wt-wb-ui-w2 && \
git checkout -b feat/workbench-ui-w2-proposal-queue
Reads required FIRST
workbench/api.py— routes/proposals,/proposals/{id},/math-proposals,/math-proposals/{id}(exact payload shapes)workbench/schemas.py—ProposalSummary,ProposalDetail,MathProposalSummary,MathProposalDetail,MathReasoningStepdocs/decisions/ADR-0162-workbench-design-system.md§"Component map", §"StableJsonViewer", §"Empty/error/loading state contract"docs/decisions/ADR-0173-workbench-ratification-trust-boundary.md§"Keyboard contract" (queue + detail bindings)docs/decisions/ADR-0167-audit-as-teaching-evidence.md§"Partition guarantees"
Outcome
Six components + two routes + two stores.
src/proposals/ProposalQueue.tsx— paginated list of pending proposals. Renders math + cognition in separate tabs (partition discipline). Default sort: oldest pending first. Filter chips: state, source kind, replay status.src/proposals/ProposalCard.tsx— one-row representation:proposal_id(truncated,CopyableHash),EpistemicStateBadge,ReviewStateBadge,age_proposals,replay_equivalenticon,handler_name(if math).src/proposals/ProposalDetailPanel.tsx— full detail view withStableJsonVieweroverevidence,replay_evidence,proposed_chain/proposed_change_payload, andreasoning_trace_stepsrendered as a step list (not a graph — ADR-0162 no graph-builder).src/json/StableJsonViewer.tsx— canonical JSON renderer; sorted keys; no re-ordering; no syntax-highlight that mutates bytes; copy-to-clipboard per top-level node.src/json/CopyableHash.tsx— truncated SHA with full-on-hover- copy button.
src/json/ArtifactLink.tsx— typed link to/artifacts/{id}route (lazy-loads the artifact).src/routes/proposals.tsx—/proposals(cognition) +/math-proposals(math) routes; URL state binds to selected proposal id.src/state/proposals.ts— Zustand store: queue cache, selected id, filter chips. TanStack Query owns the fetch.
Keyboard wires (per ADR-0173)
g p→ navigate to/proposals(or/math-proposalsbased on last-viewed tab)j/↓→ focus next proposal in queuek/↑→ focus previousEnter→ open detail panel for focused proposalEsc→ close detail, return to queue/→ focus filter input
Hard requirements
- Partition discipline. Cognition and math queues are separate
routes, separate Zustand slices, separate component trees.
Math
ProposalCardcannot render a cognition payload and vice versa. Verified by component test. StableJsonViewerdoes not alter bytes. A round-trip (json.stringify(parsed)) of the rendered payload equals the fetched payload byte-for-byte (modulo whitespace which is rendered but not stored).- No ratification UI. No
rkeybind. No "ratify" button. No "accept" or "reject" affordance. That is W3's surface. - Empty/error/loading per ADR-0162 §6. Empty state names what's missing; error state names the failure; loading state is a skeleton, not a spinner-soup splash.
- Replay-state visualization.
replay_equivalent: true|false|nullrenders three distinct states (passed / failed / not-yet-replayed) — not collapsed to a two-state truthy/falsy icon. Per scoping brief pain point #6. suggested_ratify_clirendered (read-only). Operator can copy viaykeybind even thoughr(ratify) is not yet wired. This unblocks audit-fallback workflows during W2 before W3 ships.
Tests
src/proposals/__tests__/queue.test.tsx— renders 0, 1, many proposals; sort/filter behaviorsrc/proposals/__tests__/card.test.tsx— one card per state permutationsrc/proposals/__tests__/detail.test.tsx— full detail render;StableJsonViewerbyte-stabilitysrc/proposals/__tests__/partition.test.tsx— math card refuses cognition payload; vice versasrc/proposals/__tests__/keyboard.test.tsx— j/k/Enter/Esc//behavior per ADR-0173 contractsrc/proposals/__tests__/replay_state.test.tsx— three distinct states render distinctly (passed / failed / not-yet-replayed)src/json/__tests__/stable_json.test.tsx— round-trip byte-equality on a sampleMathProposalDetailpayload- e2e —
e2e/proposal_queue.spec.ts— Playwright; start backend with fixture proposals, open UI, verify the queue populates and the detail panel renders for one math + one cognition proposal
Deliverables
- Components 1–8 above
- Routes + store wired
- Tests + e2e green
- Workbench backend untouched (read-only against existing routes)
Forbidden
- Adding ratification UI (W3)
- Adding trace / replay / eval surfaces (W4)
- Backend route changes
- Mutating any existing component from W1
- Mixing cognition and math proposals in one queue/component
Brief W3 — Ratification Corridor (the throughput multiplier)
Operator profile: Opus (load-bearing wrong=0 surface; ratification handler discipline; mirror CC-2 rigor)
Branch: feat/workbench-ui-w3-ratification-corridor
Base: origin/main (post-W2 merge, and post PR #393 / CC-2 merge for CompositionClaim handler availability)
Style: Frontend + thin backend dispatch test. Load-bearing.
Dispatch
git fetch origin main && \
git worktree add /tmp/wt-wb-ui-w3 origin/main && \
cd /tmp/wt-wb-ui-w3 && \
git checkout -b feat/workbench-ui-w3-ratification-corridor
Reads required FIRST
docs/decisions/ADR-0173-workbench-ratification-trust-boundary.md(entire ADR)docs/decisions/ADR-0167-audit-as-teaching-evidence.md§"Partition guarantees"docs/decisions/ADR-0168-frameclaim-ratification.md§"Acceptance gates"docs/decisions/ADR-0169-compositionclaim-ratification.md§"Acceptance gates"teaching/math_lexical_ratification.py—apply_lexical_claim(), exception typesteaching/math_frame_ratification.py—apply_frame_claim(), exception typesteaching/math_composition_ratification.py—apply_composition_claim(), exception typesworkbench/api.py—POST /math-proposals/{id}/ratifyroute +MathRatifyResultshapetests/test_math_frame_ratification.py::test_case_0050_hazard_pin(the template)tests/test_math_composition_ratification.py::test_case_0050_hazard_pin(same template)
Outcome
One component, one keyboard surface, one e2e per handler.
src/proposals/RatificationCommandPanel.tsx— bottom-sheet panel attached to the focused proposal inProposalDetailPanel.- Visible only when a proposal is focused AND in
pendingstate ANDreplay_equivalent == trueANDhandler_nameis in the admitted set (Lexical / Frame / Composition). - Disabled with a named status message when any precondition fails (per ADR-0173 §"Keyboard contract" — no silent failures).
- Renders: handler name, claim signature digest, target JSONL artifact path, "ratify (r)" / "reject with note (x)" / "defer (d)" / "copy CLI (y)" affordances.
- Visible only when a proposal is focused AND in
src/api/ratify.ts— typed wrapper aroundPOST /math-proposals/{id}/ratify. On 200, returnsMathRatifyResult. On 4xx, surfaces the structured error verbatim into the status footer (do not translate exception messages — operator audit fallback depends on raw text).src/state/ratify.ts— Zustand slice for post-ratify auto-advance (focus next pending after a successful ratify).- Operator-action telemetry wire in
workbench/api.py— emitoperator_ratify/operator_reject/operator_deferevents to the existingchat/telemetry.pysink per ADR-0173 Q3. Event schema:{"event": "operator_ratify", "proposal_id": "...", "handler": "...", "outcome": "applied|rejected_precondition", "ratifier_kind": "workbench", ...} - Ratification record extension —
ratifier_kind: "workbench"is written to the JSONL artifact's provenance record. Audit forensic field only; not a permission gate. Verified by test that stripsratifier_kindand confirms replay still works (per ADR-0161 §5 "Replay invariants").
Keyboard wires (per ADR-0173)
r→ ratify focused proposal (only when preconditions hold; otherwise no-op + status footer message)x→ reject with note (opens a single-line text input;Entercommits,Esccancels)d→ defer (proposal stayspending; operator-defer telemetry emits)y→ copysuggested_ratify_clito clipboard- Post-ratify: focus auto-advances to next
pending+replay_equivalent==trueproposal
Hard requirements
These mirror ADR-0173 §"Acceptance gates for W1..W4 → W3" verbatim.
- Same Python entrypoint. Every
rkeypress executes the sameapply_*_claim()function as the corresponding CLI invocation. Verified by parametrized test (one per handler) that:- constructs a fixture proposal
- ratifies via the UI (HTTP POST through the API)
- ratifies an equivalent fixture via the CLI
- diffs the resulting JSONL artifact rows
- asserts byte-equal except for
ratifier_kind
- Case 0050 hazard pin holds end-to-end. Ratifying any admitted
handler via the UI must not cause case 0050 to admit. Verified
by e2e: ratify a synthetic CompositionClaim under each safe
category, then run
core eval gsm8k_math(via the existing/evals/runroute) and assert case 0050 stays refused. - Exception surface verbatim.
AlreadyRatified,WrongClaimSubType,WrongCompositionCategory,EvidenceTampered, replay-regression errors — all surface verbatim into the status footer. No translation, no smoothing. ris a no-op on any proposal not inpendingANDreplay_equivalent == trueAND admitted handler. The no-op emits a status-footer message naming the failing precondition.- Partition. Math ratifications cannot mutate cognition artifacts and vice versa. Verified by test that attempts a cross-domain ratification and asserts the API returns 400 with a partition-violation reason.
- No auto-ratify. Replay-passed proposals remain
pendinguntil an explicitrkeypress on a focused, expanded detail. Verified by test: poll the queue with replay-passed fixtures; assert no state transitions until a keypress arrives. - No batch ratification. Each
rpress ratifies exactly one focused proposal. Multi-select UI is not introduced. - No CORS relaxation. Backend continues to bind 127.0.0.1 only; cross-origin requests refused. Verified by test.
- Operator-action telemetry redact-by-default. Events do not include surface text or token content. Verified by test: emit, parse, assert no proposal evidence text leaks.
Tests
Mirror tests/test_math_composition_ratification.py rigor.
tests/test_workbench_ratify_lexical.py— end-to-end UI ratify → JSONL artifact byte-diff vs CLI ratifytests/test_workbench_ratify_frame.py— same shape for FrameClaimtests/test_workbench_ratify_composition.py— same shape for CompositionClaimtests/test_workbench_ratify_case_0050_hazard_pin.py— mandatorytests/test_workbench_ratify_partition.py— math/cognition isolationtests/test_workbench_ratify_no_auto.py— auto-ratify forbiddentests/test_workbench_ratify_exception_surface.py— exceptions surface verbatimtests/test_workbench_operator_telemetry.py— events emit; no content leaktests/test_workbench_ratify_idempotent.py— duplicate ratify via UI returnsAlreadyRatifiedsrc/proposals/__tests__/ratification_panel.test.tsx— frontend unit tests for visibility, keyboard, no-op behaviore2e/ratification_corridor.spec.ts— Playwright end-to-end for all three handlers
Deliverables
RatificationCommandPanel.tsx,src/api/ratify.ts,src/state/ratify.ts- Backend: operator-action telemetry wire in
workbench/api.py workbench/readers.pyupdates if needed forratifier_kindsurfacing (read current state first; minimal diff)- All tests above, green
core test --suite teaching -qgreencore test --suite runtime -qgreencore eval gsm8k_mathgreen- Case 0050 remains refused
Forbidden
- Admitting a handler not in {Lexical, Frame, Composition}
- Adding auto-ratify of any flavor
- Adding batch / multi-select ratification
- Bypassing handler preconditions in any code path
- Adding remote operator auth, login, or token-bearer
- Relaxing CORS on the backend
- Adding
Cmd/Ctrlkeyboard chords - Mutating
engine_state/*outside the existing checkpoint path - Treating
ratifier_kindas a permission gate - Adding a parallel telemetry log
Brief W4a — Replay Surfaces
Operator profile: Sonnet (tight-scope frontend over existing routes)
Branch: feat/workbench-ui-w4a-replay-surfaces
Base: origin/main (post-W1; parallel-safe with W2/W3/W4b)
Style: Frontend only. Read-only.
Dispatch
git fetch origin main && \
git worktree add /tmp/wt-wb-ui-w4a origin/main && \
cd /tmp/wt-wb-ui-w4a && \
git checkout -b feat/workbench-ui-w4a-replay-surfaces
Reads required FIRST
workbench/api.py—/trace/...,/replay/...routesworkbench/schemas.py—ReplayComparison,ReplayDivergence,ReplayDivergenceSeverity,ReplayStatusdocs/decisions/ADR-0160-core-workbench-v1.md§"Trace Drawer", §"Replay Theater"docs/decisions/ADR-0162-workbench-design-system.md§"Component map"
Outcome
Four components + one route.
src/trace/TraceDrawer.tsx— slide-in drawer showing the trace for a focused turn / proposal / ratification. Renders trace hash + step list (not graph). Opens viag tkeybind.src/replay/ReplayTheater.tsx— full-width panel hosting replay comparison + diff.src/replay/ReplayComparisonPanel.tsx— side-by-side original-vs-replay hash + equivalent boolean + divergence count.src/replay/ReplayDiffViewer.tsx— divergence list with severity coloring (info / warning / failure per ADR-0162); usesStableJsonDiffViewerfrom new component below.src/json/StableJsonDiffViewer.tsx— canonical JSON diff; structural-aware (key/value/array-index), not line-based.src/routes/trace.tsx—/trace/:idroute.src/routes/replay.tsx—/replay/:idroute.
Hard requirements
- Read-only. No replay-mutation UI. No "re-run replay" button in v1 (the backend route is read; replay is run server-side by separate mechanisms).
- Diff is structural.
StableJsonDiffVieweroperates on parsed JSON, not text lines. Byte-equal payloads produce zero divergences regardless of formatting. - Severity rendering. info / warning / failure render distinctly without color-only encoding (icon + text + color).
Tests
src/trace/__tests__/trace_drawer.test.tsxsrc/replay/__tests__/replay_theater.test.tsxsrc/replay/__tests__/replay_diff.test.tsxsrc/json/__tests__/stable_json_diff.test.tsx— round-trip; byte-equal payloads → zero divergencese2e/replay_theater.spec.ts
Forbidden
- Adding replay-mutation routes or buttons
- Line-based diff (must be structural)
- Color-only severity encoding
Brief W4b — Eval Surfaces
Operator profile: Sonnet (tight-scope frontend over existing routes)
Branch: feat/workbench-ui-w4b-eval-surfaces
Base: origin/main (post-W1; parallel-safe with W2/W3/W4a)
Style: Frontend only. Drives existing read-only eval routes.
Dispatch
git fetch origin main && \
git worktree add /tmp/wt-wb-ui-w4b origin/main && \
cd /tmp/wt-wb-ui-w4b && \
git checkout -b feat/workbench-ui-w4b-eval-surfaces
Reads required FIRST
workbench/api.py—/evals,/evals/{id},POST /evals/runroutesworkbench/schemas.py—EvalLaneSummary,EvalRunResultdocs/decisions/ADR-0160-core-workbench-v1.md§"Eval Center"docs/decisions/ADR-0166-measurement-capability-sequencing.md(do not introduce new eval lanes)
Outcome
Three components + one route.
src/evals/EvalCenter.tsx— top-level eval page.src/evals/EvalLaneList.tsx— list of available lanes fromGET /evals. Each row: lane name, latest run status, version, read-only badge, description. Clicking a row opens detail.src/evals/EvalFailureViewer.tsx— for a failed eval run, list failing cases with theirMathReaderRefusalEvidence(or cognition-equivalent) +ArtifactLinkto the run JSON.src/routes/evals.tsx—/evalsroute.
Hard requirements
- No new eval lanes. ADR-0166 holds. The UI runs existing lanes only.
POST /evals/runis the only mutation surface. It is read-only in effect (runs an eval; does not mutate corpus/packs). Other than that, every interaction is a GET.- Failure surfacing tied to evidence. A failing case must link
to the audit row that produced it (via
ArtifactLink). - No vanity dashboard splash. ADR-0162 no-go list: no "total turns" / "active sessions" / pulsing-orb metrics on the eval landing page.
Tests
src/evals/__tests__/eval_center.test.tsxsrc/evals/__tests__/lane_list.test.tsxsrc/evals/__tests__/failure_viewer.test.tsxe2e/eval_center.spec.ts— list lanes, open one, run via the read-only route, view results
Forbidden
- Introducing new eval lanes (ADR-0166)
- Adding splash metrics / "AI cognition theater" panels
- Mutating corpus/packs through the eval surface
Anti-regression invariants (all four briefs)
wrong == 0oncore eval gsm8k_math— preserved- ADR-0166 — no new eval lanes
- ADR-0057 replay-equivalence — inherited
- ADR-0167 partition — math/cognition isolation in UI as on server
- Pinned-lane SHAs — should not require updates
engine_state/*— never committed- Case 0050 hazard — pinned in W3's test suite (mandatory)
- Backend bound to 127.0.0.1 — never relaxed
- No remote runtime dependencies — fonts, icons, telemetry all local
Memory pointers
- feedback-batch-during-research — implementation mode; one PR per wave
- feedback-no-self-dispatch-of-subagents — Shay dispatches operators
- feedback-production-line-pattern — this brief pack pattern
- feedback-parallel-dispatch-pattern — operator profile mapping
- feedback-parallel-agent-worktrees — fresh worktree per brief
- feedback-wrong-zero-hazard-case-0050 — W3 mandatory pin
- feedback-cleanup-as-you-find — apply if W1 finds dead workbench backend stubs
- user-circumstances — operators on library wifi; local-first matters
- milestone-adr-0172-tier1-2026-05-27 — context for throughput-vs-capability framing
- adr-0167-audit-as-evidence-wave — parent corridor
What ships when all PRs land
- W1. Operator can run
core workbench serveand open a bootstrapped UI shell with command palette, keyboard help, and state badges. Nothing actionable yet. - W2. Operator can read every pending math + cognition proposal, inspect evidence/replay/reasoning trace, and copy the suggested ratify CLI for shell fallback.
- W3. Operator can ratify a Lexical / Frame / Composition
proposal via
rkeypress in under 10 seconds per row for a row whose evidence they understand. Case 0050 remains refused. The compounding loop runs at human-realistic throughput. - W4a. Operator can inspect any trace or replay artifact inline; replay divergences are structurally diffed.
- W4b. Operator can run and inspect eval lanes inline; failing cases link back to their audit-row evidence.
Together: the math teaching corridor becomes operable by one operator without context-switching to the shell. The five operator pain points from the scoping brief are individually retired:
| Pain point | Retired by |
|---|---|
| HITL burden high | W3 keyboard corridor |
| No keyboard corridor for ratification | W3 r/x/d/y bindings |
| Evidence is text-block JSON | W2 StableJsonViewer + structured detail |
| Reasoning traces are flat | W2 step-list rendering of MathReasoningStep |
| Replay equivalence is opaque hex | W2 three-state replay visualization; W4a ReplayComparisonPanel |
| No "stuck queue" diagnosis | W2 filter chips + replay-state visualization |
Copy-paste dispatch lines (when ready)
After ADR-0173 (PR #394) lands, dispatch in order:
# W1 — Codex (mechanical scaffold)
Read docs/handoff/WORKBENCH-UI-WAVE-BRIEF-PACK.md §"Brief W1".
git fetch origin main && git worktree add /tmp/wt-wb-ui-w1 origin/main && cd /tmp/wt-wb-ui-w1 && git checkout -b feat/workbench-ui-w1-scaffold
# (wait for W1 merge)
# W2 — Sonnet (read surfaces)
Read docs/handoff/WORKBENCH-UI-WAVE-BRIEF-PACK.md §"Brief W2".
git fetch origin main && git worktree add /tmp/wt-wb-ui-w2 origin/main && cd /tmp/wt-wb-ui-w2 && git checkout -b feat/workbench-ui-w2-proposal-queue
# W4a — Sonnet (parallel with W2)
Read docs/handoff/WORKBENCH-UI-WAVE-BRIEF-PACK.md §"Brief W4a".
git fetch origin main && git worktree add /tmp/wt-wb-ui-w4a origin/main && cd /tmp/wt-wb-ui-w4a && git checkout -b feat/workbench-ui-w4a-replay-surfaces
# W4b — Sonnet (parallel with W2/W4a)
Read docs/handoff/WORKBENCH-UI-WAVE-BRIEF-PACK.md §"Brief W4b".
git fetch origin main && git worktree add /tmp/wt-wb-ui-w4b origin/main && cd /tmp/wt-wb-ui-w4b && git checkout -b feat/workbench-ui-w4b-eval-surfaces
# (wait for W2 merge AND #393 CompositionClaim merge)
# W3 — Opus (ratification corridor; load-bearing wrong=0)
Read docs/handoff/WORKBENCH-UI-WAVE-BRIEF-PACK.md §"Brief W3".
git fetch origin main && git worktree add /tmp/wt-wb-ui-w3 origin/main && cd /tmp/wt-wb-ui-w3 && git checkout -b feat/workbench-ui-w3-ratification-corridor
Status zsh trap (operator convenience)
For local dispatch monitoring (mirror the pattern used during the ADR-0172 Tier 1 wave):
status() {
for pr in 394 393; do
gh pr view "$pr" --json number,title,state,mergeStateStatus,statusCheckRollup \
| python3 -c "
import json,sys
d=json.load(sys.stdin)
checks=d.get('statusCheckRollup') or []
from collections import Counter
states=Counter(c.get('conclusion') or c.get('status') for c in checks)
print(f\"#{d['number']} {d['state']:<6} {d['mergeStateStatus']:<16} {dict(states)} — {d['title'][:60]}\")
"
done
}
Run status to get a single-line summary per gating PR.