core/docs/handoff/WORKBENCH-W3-RATIFICATION-CORRIDOR-BRIEF.md
Shay a67b475e59
docs(adr-0173): accept workbench ratification trust boundary (W0); queue reconciled W3 corridor brief (#469)
Flip ADR-0173 Proposed -> Accepted. This is the W0 gate for the
workbench-UI ratification wave: it amends ADR-0160's read-only stance
narrowly to admit operator-driven invocation of the three existing,
replay-gated Tier 1.5 handlers (apply_lexical_claim, apply_frame_claim,
apply_composition_claim) as a local keyboard accelerator over ADR-0161
Surface C. No UI code; implementation remains gated to the W1..W4
acceptance gates in the ADR.

Also lands the reconciled W3 ratification-corridor brief. It supersedes
the stale W3 section of WORKBENCH-UI-WAVE-BRIEF-PACK.md, which assumed a
'frontend zero' state (W1/W2 already merged via #295/#299/#329/#327/
#328/#303) and prescribed a Zustand store this repo never adopted. The
reconciled brief names the load-bearing facts the old one glossed:
- /ratify is advisory-only today (routes, never applies); W3's core
  backend task is the advisory->in-process flip.
- live proposals carry shape_category='uncategorized', so the safe
  category/polarity must be operator-supplied and allowlist-gated, not
  hard-coded -- the exact silent-wrong-category path case 0050 guards.

Verification this session: read-only workbench v1 confirmed working
(API serves real deterministic data, honest 404s, traversal rejected,
ratify advisory/no-mutation; 31 python + 104 UI tests green; pnpm build
green).

No runtime/algebra/eval paths touched. Docs only.
2026-05-29 11:14:25 -07:00

358 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Brief W3 — Workbench Ratification Corridor (the throughput multiplier)
**Status:** Ready to dispatch once W0 (ADR-0173 acceptance) lands.
**Date:** 2026-05-29
**Author:** Shay
**Supersedes:** the W3 section of
`docs/handoff/WORKBENCH-UI-WAVE-BRIEF-PACK.md` (lines 324487), which
was written against a stale "frontend zero" state and a Zustand layout
this codebase never adopted. Read the *Reconciliation* section below
before treating the older brief as authoritative.
**Parent doctrine:** [ADR-0173](../decisions/ADR-0173-workbench-ratification-trust-boundary.md)
(trust boundary), [ADR-0160](../decisions/ADR-0160-core-workbench-v1.md)
(workbench v1), [ADR-0162](../decisions/ADR-0162-workbench-design-system.md)
(design system + keyboard contract)
**Operator profile:** Opus — load-bearing `wrong == 0` surface; must
mirror the case-0050 hazard-pin rigor of
`tests/test_math_composition_ratification.py`.
---
## Reconciliation — what actually shipped vs. what the old brief assumed
The earlier wave pack (`WORKBENCH-UI-WAVE-BRIEF-PACK.md`) was authored
2026-05-27 against the scoping brief's "backend ready, frontend zero"
framing. That framing was already wrong on arrival. Verified state of
`origin/main` as of 2026-05-29:
| Old brief's wave | Old assumption | Actual state on `main` |
|---|---|---|
| W1 scaffold | "not landed" | **Merged**`workbench-ui/` (#295, #299), Vite/TS-strict/Tailwind, tokens, shell, LeftNav, TopBar, StatusFooter, CommandPalette |
| W2 read surfaces | "not landed" | **Merged**`src/app/proposals/*` (#329), `ProposalsRoute`, `ProposalTable`, `ProposalDetailPanel` analog, `SuggestedCLIBox`, `StableJsonViewer`, state badges |
| W4 verify | "not landed" | **Merged** — replay (#328), eval center (#327), chat+trace (#303) |
| W3 ratify | the genuinely-new piece | **Not started** — this brief |
**Consequence:** W3 no longer depends on a "W2 merge." Its only hard
upstream dependencies are:
1. **W0 — ADR-0173 status flips `Proposed → Accepted`.** It amends
ADR-0160's read-only stance; W3 must not land while it is still a
proposal. (One-line PR; no code.)
2. **The backend advisory→in-process flip** (below) — currently
`/ratify` *routes but does not apply*. This is the load-bearing
change the old brief glossed.
All three handlers now exist on `main`
(`apply_lexical_claim`, `apply_frame_claim`,
`apply_composition_claim` — the last via PR #393), so there is no
"pending CC-2" caveat anymore.
---
## The load-bearing backend fact (read this twice)
`workbench/readers.py::ratify_math_proposal` **does not ratify
anything today.** Verified live:
```
POST /math-proposals/{id}/ratify
→ {"routing_status": "routed", "handler_name": "CompositionClaim",
"suggested_cli": "from teaching.math_composition_ratification import ...",
...} # no JSONL append; git status clean afterward
```
It validates the `proposed_change_kind`, picks a handler name, and
returns a *suggested CLI string with hard-coded placeholder arguments*
(`category='drain_token'`, `composition_category='multiplicative_composition'`).
It never constructs a `MathReaderRefusalEvidence`, never calls a
handler, never writes.
**W3's central backend task is to add a real apply path** that invokes
the handler in-process per ADR-0173 §Q1 — while keeping the advisory
dry-run path reachable for the `y` (copy-CLI) audit fallback.
### Why this is the wrong=0 hazard, not a plumbing change
The live `/math-proposals` payload carries `shape_category:
"uncategorized"` on real proposals. The handlers refuse to write
unless given a category in their safe allowlist
(`SAFE_CATEGORIES` / `SAFE_FRAME_CATEGORIES` /
`SAFE_COMPOSITION_CATEGORIES`) and a polarity in
`{"affirms", "falsifies"}`. The proposal record **does not contain a
ratifiable category** — it contains a *structural commonality* and an
*uncategorized* shape.
Therefore the category and polarity **must be supplied by the
operator at ratification time**, constrained to the per-handler safe
allowlist, and passed in the request body. The old brief hard-coded
`composition_category='multiplicative_composition'` in a CLI comment —
that is exactly the silent-wrong-category path that case 0050 exists to
catch. **Do not auto-derive the category. Make the operator choose it
from the allowlist; refuse if it is absent or off-allowlist.**
---
## Dispatch
```bash
git fetch origin main && \
git worktree add /tmp/wt-wb-ui-w3 origin/main && \
cd /tmp/wt-wb-ui-w3 && \
git checkout -b feat/workbench-ui-w3-ratification-corridor
```
(Fresh worktree per [[feedback-parallel-agent-worktrees]]. Do not
share a working dir with other agents.)
## Reads required FIRST
- `docs/decisions/ADR-0173-workbench-ratification-trust-boundary.md`
— entire ADR. The §"Keyboard contract", §"What the workbench MAY
do / MUST NOT do", §"Ratification record extension", and
§"Acceptance gates → W3" are verbatim requirements.
- `workbench/readers.py::ratify_math_proposal` — the advisory impl
you are extending (not replacing — see §Q1 dry-run retention).
- `workbench/api.py` — the `POST /math-proposals/{id}/ratify` route
and `_math_ratify`; `MathRatifyResult` in `workbench/schemas.py`.
- `teaching/math_lexical_ratification.py``apply_lexical_claim()`
signature, `SAFE_CATEGORIES`, exception types
(`WrongClaimSubType`, `WrongZeroViolationCandidate`,
`AlreadyRatified`, `EvidenceTampered`).
- `teaching/math_frame_ratification.py``apply_frame_claim()`,
`SAFE_FRAME_CATEGORIES`, `EvidenceLaundering`.
- `teaching/math_composition_ratification.py`
`apply_composition_claim()`, `SAFE_COMPOSITION_CATEGORIES`.
- `tests/test_math_composition_ratification.py::test_case_0050_hazard_pin`
— the hazard-pin template W3 mirrors.
- `workbench-ui/src/app/proposals/` — actual component layout
(`ProposalsRoute`, `ProposalTable`, `SuggestedCLIBox`, badges).
- `workbench-ui/src/api/queries.ts` — established `useMutation`
pattern (`useEvalRun`, `useChatTurn`); this is how the ratify
mutation is wired. **There is no Zustand in this repo** — auto
advance is local React state + `queryClient.invalidateQueries`.
- `workbench-ui/src/api/client.ts` — existing POST support.
---
## Outcome
One backend apply path, one component, one keyboard surface, one e2e
per handler.
### Backend
1. **`workbench/readers.py` — add an apply path.**
`ratify_math_proposal(proposal_id, *, category, polarity,
reviewer, dry_run=False, jsonl_path=None)`:
- `dry_run=True` (or category omitted) → existing advisory behavior
unchanged (`routing_status="routed"` + `suggested_cli`). This is
what the `y` copy-CLI affordance and audit fallback rely on.
- `dry_run=False` with a category → reconstruct the
`MathReaderRefusalEvidence` from the proposal record's evidence,
then dispatch to the matching handler:
- `LexicalClaim``apply_lexical_claim(claim=…, category=category, reviewer=reviewer)`
- `FrameClaim``apply_frame_claim(claim=…, frame_category=category, polarity=polarity, reviewer=reviewer)`
- `CompositionClaim``apply_composition_claim(claim=…, composition_category=category, polarity=polarity, reviewer=reviewer)`
- Return a `MathRatifyResult` carrying the handler receipt fields
(target JSONL path, evidence hash, `applied=True`) on success.
- **Do not catch handler exceptions to smooth them.** Let
`WrongClaimSubType` / `WrongCompositionCategory` /
`EvidenceLaundering` / `AlreadyRatified` / `EvidenceTampered`
propagate; the API layer translates them to structured 4xx with
the message verbatim.
2. **`workbench/api.py` — accept a request body** on the existing
route: `{ "category": str, "polarity": "affirms"|"falsifies",
"dry_run": bool }`. `reviewer` is **server-derived** (OS username
of the workbench process, same as Surface C — never operator-typed,
per ADR-0173 §"Ratification record extension"). Reject a non-local
`Origin`/`Host` (CORS stays closed). Map handler exceptions →
`400`/`409` with `{"error": {"code", "message"}}` carrying the raw
handler message.
3. **`ratifier_kind: "workbench"`** written into the JSONL artifact's
provenance record. **Audit-forensic field only, never a permission
gate** (ADR-0173). `proposal_id` and `to` remain the only
replay-load-bearing fields.
4. **Operator-action telemetry** to the existing `chat/telemetry.py`
sink (ADR-0173 §Q3) — `operator_ratify` / `operator_reject` /
`operator_defer`. Outcome-only (no intent-to-ratify events),
redact-by-default (no evidence/surface text):
```json
{"event": "operator_ratify", "proposal_id": "...", "handler": "...",
"outcome": "applied|rejected_precondition", "ratifier_kind": "workbench"}
```
No parallel log file.
### Frontend
5. **`workbench-ui/src/app/proposals/RatificationCommandPanel.tsx`** —
panel attached to the focused proposal in the detail view.
- Renders only when the focused proposal is `pending` AND
`replay_equivalent == true` AND `handler_name ∈ {LexicalClaim,
FrameClaim, CompositionClaim}`. Otherwise renders disabled with a
status-footer message naming the failing precondition (no silent
failure — ADR-0173 §"Keyboard contract").
- **Category selector** populated from the handler's safe allowlist
(fetched/typed from the API; do not hard-code in TSX). **Polarity
selector** (`affirms`/`falsifies`) for Frame/Composition;
Lexical takes no polarity. `r` is disabled until a category is
selected.
- Shows: handler name, claim evidence-hash digest, target JSONL
artifact path, and the four affordances
(`ratify (r)` / `reject (x)` / `defer (d)` / `copy CLI (y)`).
6. **`useMathRatify` mutation in `src/api/queries.ts`** — follow the
`useEvalRun`/`useChatTurn` `useMutation` pattern. On 2xx invalidate
the proposals query and advance focus to the next
`pending`+`replay_equivalent` row. On 4xx, surface
`WorkbenchApiError.message` **verbatim** into the status footer.
7. **Auto-advance** via local React state + `queryClient
.invalidateQueries`. No new global store.
---
## Keyboard wires (per ADR-0173, pinned — do not invent new bindings)
- `r` → ratify focused proposal. No-op + named status-footer message
unless `pending``replay_equivalent == true` ∧ admitted handler ∧
a category is selected.
- `x` → reject with note (single-line input; `Enter` commits, `Esc`
cancels). Emits `operator_reject`. (Reject is **not** durable
proposal-state mutation in v1 — it is an operator annotation +
telemetry event; the proposal remains in the log.)
- `d` → defer (proposal stays `pending`; `operator_defer` emits).
- `y` → copy `suggested_ratify_cli` to clipboard (audit fallback;
uses the dry-run path).
- No `Cmd`/`Ctrl` chords (collides with browser shortcuts).
---
## Hard requirements (mirror ADR-0173 §"Acceptance gates → W3" verbatim)
- **Same Python entrypoint.** Every non-dry-run ratify executes the
same `apply_*_claim()` the CLI would. Proven by a parametrized test
(one per handler) that ratifies a fixture via the API and an
equivalent fixture via the CLI, then asserts the JSONL artifact rows
are **byte-equal except for `ratifier_kind`**.
- **Operator-supplied category, allowlist-gated.** A ratify request
with no category, or a category outside the handler's `SAFE_*`
allowlist, is refused (`400`) and writes nothing. Proven by test.
- **Case 0050 hazard pin holds end-to-end.** Ratify a synthetic claim
under each safe category via the UI path, then run
`core eval gsm8k_math` (via `/evals/run`) and assert case 0050 stays
**refused**. This is the mandatory pin — mirror
`test_math_composition_ratification.py::test_case_0050_hazard_pin`.
- **Exception surface verbatim.** `AlreadyRatified`,
`WrongClaimSubType`, `WrongCompositionCategory`,
`EvidenceLaundering`, `EvidenceTampered`, replay-regression — all
surface raw into the status footer. No translation, no smoothing.
- **`r` is a no-op** on any proposal failing a precondition; the no-op
names the failing precondition in the footer.
- **Partition.** A math ratify cannot mutate cognition artifacts and
vice versa; a cross-domain attempt returns `400` with a
partition-violation reason. Proven by test.
- **No auto-ratify.** Replay-passed proposals stay `pending` until an
explicit `r`. Proven by polling a queue of replay-passed fixtures
and asserting zero transitions until a keypress.
- **No batch / multi-select ratification.** One `r` = one proposal.
- **CORS stays closed.** Backend binds `127.0.0.1`; cross-origin
refused. Proven by test.
- **Telemetry redact-by-default.** Events carry no evidence/surface
text. Proven by emit→parse→assert-no-leak test.
- **`ratifier_kind` is not load-bearing for replay.** A test strips
`ratifier_kind` from the record and confirms replay still passes.
---
## Tests (mirror `tests/test_math_composition_ratification.py` rigor)
- `tests/test_workbench_ratify_lexical.py` — UI ratify → JSONL
byte-diff vs CLI ratify (except `ratifier_kind`)
- `tests/test_workbench_ratify_frame.py` — same shape, FrameClaim
- `tests/test_workbench_ratify_composition.py` — same shape, CompositionClaim
- `tests/test_workbench_ratify_category_allowlist.py` — missing /
off-allowlist category refused, nothing written
- `tests/test_workbench_ratify_case_0050_hazard_pin.py`**mandatory**
- `tests/test_workbench_ratify_partition.py` — math/cognition isolation
- `tests/test_workbench_ratify_no_auto.py` — auto-ratify forbidden
- `tests/test_workbench_ratify_exception_surface.py` — verbatim 4xx
- `tests/test_workbench_ratify_idempotent.py` — duplicate → `AlreadyRatified`
- `tests/test_workbench_operator_telemetry.py` — events emit; no leak
- `workbench-ui/src/app/proposals/RatificationCommandPanel.test.tsx`
visibility gating, category-required disable, keyboard, no-op paths
- `workbench-ui/e2e/ratification_corridor.spec.ts` — end-to-end for all
three handlers (if Playwright is wired; otherwise a vitest+MSW
integration test against the real API shape)
## Deliverables
- `workbench/readers.py` apply path; `workbench/api.py` body parsing +
exception mapping + telemetry; `workbench/schemas.py` `MathRatifyResult`
extension (`applied`, `target_path`, `evidence_hash`)
- `RatificationCommandPanel.tsx`; `useMathRatify` in `queries.ts`;
category/polarity selectors typed from the safe allowlists
- All tests above, green
- `core test --suite teaching -q` green
- `core test --suite runtime -q` green
- `core eval gsm8k_math` green; **case 0050 remains refused**
- `cd workbench-ui && pnpm build && pnpm test` green
## Forbidden
- Auto-deriving the ratification category from `shape_category` or
`proposed_change_kind` (operator must choose; off-allowlist refused)
- Admitting any handler outside {Lexical, Frame, Composition}
- Auto-ratify of any flavor; batch / multi-select ratification
- Bypassing handler preconditions in any code path
- Catching/smoothing handler exceptions before the API boundary
- Remote operator auth, login, token-bearer; relaxing CORS
- `Cmd`/`Ctrl` keyboard chords
- Writing `engine_state/*` outside the existing checkpoint path
- Treating `ratifier_kind` as a permission gate
- A parallel telemetry log file
- A new Zustand/Jotai global store (use TanStack Query + local state)
---
## Anti-regression invariants
- `wrong == 0` on `core eval gsm8k_math` preserved
- Case 0050 refused after end-to-end UI ratification of any handler
- ADR-0167 partition (math vs cognition) preserved in API and UI
- ADR-0161 §5 replay invariants unchanged (`proposal_id` + `to` only)
- No new eval lane (ADR-0166)
- `engine_state/*` never committed
- Pinned-lane SHAs unchanged by UI work
## Acceptance
W3 is done when an operator can, against a locally-running
`core workbench api`:
1. open the proposal queue, focus a `pending`+`replay_equivalent`
math proposal;
2. choose a safe category (+ polarity for Frame/Composition);
3. press `r` and have it invoke the same `apply_*_claim()` the CLI
would, producing a JSONL artifact append byte-equal to the CLI's
except for `ratifier_kind`;
4. see the handler's exception verbatim when a precondition fails;
5. confirm case 0050 stays refused afterward — under each safe
category, end-to-end.
Each of the five operator pain points in
`WORKBENCH-UI-WAVE-SCOPING.md` is retired by a named affordance here:
queue navigation (j/k), category-confirmed ratify (r), evidence digest
+ target path in the panel, replay-state gating of `r`, and the
no-silent-failure status footer.
## Memory pointers
- [[feedback-no-self-dispatch-of-subagents]] — Shay dispatches; this
brief is written and stops. No Agent calls.
- [[feedback-production-line-pattern]] — brief-pack discipline
- [[feedback-parallel-agent-worktrees]] — fresh worktree per dispatch
- [[feedback-wrong-zero-hazard-case-0050]] — mandatory W3 pin
- [[feedback-address-critiques-dont-waive]] — guardrail violations are
reworked, not waived
- [[adr-0175-calibrated-learning-architecture]] — keep ratification
honest: move intelligence into the solver, not per-shape matchers
- [[milestone-adr-0172-tier1-2026-05-27]] — why throughput, not
capability, is the bottleneck