From 404e69482476ec9f9d4c9c0fe8e799f6a2c9c2bc Mon Sep 17 00:00:00 2001 From: Shay Date: Tue, 26 May 2026 08:22:38 -0700 Subject: [PATCH] docs(workbench): CORE Workbench v1 planning architecture (ADR-0160) Docs-only planning branch. Smoke + verify pinned lane SHAs CI checks pass. --- docs/decisions/ADR-0160-core-workbench-v1.md | 265 ++++++++++++ docs/workbench/README.md | 160 +++++++ docs/workbench/acceptance-gates.md | 153 +++++++ docs/workbench/api-contract-v1.md | 363 ++++++++++++++++ docs/workbench/core-workbench-v1-blueprint.md | 397 ++++++++++++++++++ docs/workbench/data-shapes-v1.md | 254 +++++++++++ docs/workbench/implementation-plan.md | 205 +++++++++ docs/workbench/ui-component-map.md | 292 +++++++++++++ 8 files changed, 2089 insertions(+) create mode 100644 docs/decisions/ADR-0160-core-workbench-v1.md create mode 100644 docs/workbench/README.md create mode 100644 docs/workbench/acceptance-gates.md create mode 100644 docs/workbench/api-contract-v1.md create mode 100644 docs/workbench/core-workbench-v1-blueprint.md create mode 100644 docs/workbench/data-shapes-v1.md create mode 100644 docs/workbench/implementation-plan.md create mode 100644 docs/workbench/ui-component-map.md diff --git a/docs/decisions/ADR-0160-core-workbench-v1.md b/docs/decisions/ADR-0160-core-workbench-v1.md new file mode 100644 index 00000000..65a01b7b --- /dev/null +++ b/docs/decisions/ADR-0160-core-workbench-v1.md @@ -0,0 +1,265 @@ +# ADR-0160 — CORE Workbench v1: operator/auditor UI before public chat + +Status: proposed +Date: 2026-05-26 + +## Context + +CORE now has the first complete contemplation/review corridor: + +- ADR-0152 — learning-arc proof corridor +- ADR-0155 — CI contemplation reports +- ADR-0156 — atomic checkpoint writes +- ADR-0157 — revision-mismatch warning +- ADR-0158 — reboot-event audit trail +- ADR-0159 — contemplation-quality eval lane + +The next surface should not be a generic chatbot clone. CORE's distinction is +not merely that it can answer prompts; it is that its cognition can be inspected, +replayed, audited, reviewed, and ratified under explicit trust boundaries. + +A UI that hides this machinery would erase the strongest architectural evidence. +A UI that exposes everything by default would become cognitive noise. + +The correct first UI is therefore an operator/auditor workbench with a small +chat surface and deep progressive disclosure. + +## Decision + +Build **CORE Workbench v1** as the first UI/UX surface. + +The v1 product is an admin/engineer/dev/auditor interface with five primary +modules: + +1. **Chat Surface** — minimal prompt/response interface for live turns. +2. **Trace Drawer** — per-turn inspection of surfaces, trace hashes, grounding, + replay indicators, proposal/candidate metadata, and telemetry. +3. **Proposal Review Queue** — pending proposals, contemplation provenance, + replay evidence, downstream effect, and accept/reject readiness. +4. **Eval Center** — run/read deterministic eval lanes including cognition, + learning-arc, and contemplation-quality. +5. **Replay Theater** — select an artifact or turn and compare deterministic + replay evidence against the original. + +The first implementation phase is planning + read-only observability. Mutating +actions (proposal acceptance, corpus mutation, workflow dispatch) require a later +ADR and explicit operator confirmation gates. + +## Product doctrine + +### Calm default, infinite depth + +The UI must be quiet by default. The first screen should show a response and +small trust badges, not a wall of internal machinery. Deep evidence is revealed +through drawers, split panes, and artifact inspectors. + +### Audit-native, not analytics theater + +The workbench is not a generic dashboard. Every panel must answer one of these +questions: + +- What happened? +- Why was it allowed? +- What evidence supports it? +- Can it be replayed? +- Did it mutate anything? +- Who/what has authority to ratify the next step? + +If a panel cannot answer one of those questions, it does not belong in v1. + +### Proposal before mutation + +The workbench may visualize proposals and review readiness. It must not add a +parallel learning path. Proposal acceptance remains governed by teaching review +and existing ADR-0057/0151 doctrine. + +### Replay before persuasion + +The UI should never ask the operator to trust an impressive answer. It should +show replay status, trace identity, provenance, and mutation boundaries first. + +## Architecture choice + +### Frontend + +Adopt a thin React/Vite/TypeScript workbench: + +- React + Vite + TypeScript +- TanStack Query for API state +- Zustand or Jotai for local UI state +- Tailwind + shadcn-style primitives for consistent low-noise UI +- Monaco only where structured JSON/editor inspection is necessary +- xterm-like console only where raw CLI/replay output genuinely helps + +No Electron in v1. No heavy design system. No plugin marketplace. No agent +workflow builder. + +### Backend + +Expose a narrow local API over the existing runtime surfaces: + +- chat turn endpoint / stream endpoint +- trace/artifact read endpoints +- proposal-log read endpoint +- eval lane run/read endpoint +- replay artifact read/compare endpoint +- telemetry read endpoint + +The backend must remain local-first and deterministic. Any endpoint that can +mutate corpus, packs, proposals, engine_state, or workflows is out of scope for +v1 unless a later ADR explicitly admits it. + +## Trust boundary + +V1 is **read-only by default**. + +Allowed: + +- read telemetry JSONL +- read proposal logs +- read contemplation run artifacts +- read eval results +- run read-only eval lanes +- execute chat turns against existing runtime APIs +- display trace/replay metadata + +Forbidden in v1: + +- accepting proposals +- rejecting proposals as durable state +- mutating teaching corpus +- mutating packs +- writing engine_state except through normal runtime checkpoint path already + governed by ADR-0146/0150 +- workflow dispatch that commits to main +- hidden background jobs +- remote network dependencies for cognition + +## Module contracts + +### 1. Chat Surface + +Purpose: live interaction with visible trust badges. + +Required visible signals: + +- grounding source +- replay/trace id when available +- refusal/review state when applicable +- mutation status: none / transient / proposal-only / ratified + +### 2. Trace Drawer + +Purpose: expose one turn's evidence stack. + +Should display: + +- final `surface` +- `articulation_surface` when present +- `walk_surface` as telemetry/evidence only +- trace hash / replay digest +- grounding source +- proposal/candidate references +- rejected/admissibility evidence when available + +Must preserve the runtime surface contract: user surface and telemetry evidence +remain distinct. + +### 3. Proposal Review Queue + +Purpose: make proposal lifecycle legible. + +V1 displays only: + +- pending proposal id +- source.kind / source_id +- replay-equivalence evidence +- proposed chain +- candidate evidence +- review state +- downstream quality signal if present + +V1 does not accept/reject. It can show exact CLI command suggestions for local +operator review, but must not execute them. + +### 4. Eval Center + +Purpose: make deterministic quality lanes discoverable. + +Required initial lanes: + +- cognition +- learning-arc demo report view +- contemplation-quality + +Eval Center must distinguish: + +- lane execution +- stored result inspection +- failure details +- mutation boundary + +### 5. Replay Theater + +Purpose: demonstrate deterministic replay as an experience, not a footnote. + +V1 should support: + +- selecting a run/turn/report +- showing original artifact hash +- showing replay result hash +- comparing metrics and surfaces +- highlighting divergence, if any + +## Non-goals + +- public marketing site +- multi-tenant SaaS +- hosted cloud service +- auth system beyond local operator mode +- background daemon requirement +- arbitrary plugin execution +- agent marketplace +- broad chat polish before audit polish +- replacing CLI lanes + +## Acceptance criteria + +The first implementation PR after this planning ADR should include only the +minimal skeleton necessary to prove the workbench shape: + +- local API skeleton or documented endpoint contract +- frontend shell with navigation +- static/mock-free read-only panels wired to real local files or real local API + where practical +- no corpus/packs/proposals mutation path +- no hidden background jobs +- no remote dependencies +- README/runbook for local launch + +A future implementation PR may add proposal mutation only after a separate ADR +ratifies the operator confirmation flow. + +## Consequences + +Positive: + +- Gives CORE a surface that matches its actual strengths. +- Makes replay/audit/proposal boundaries visible. +- Creates an investor/engineer demo that is not reducible to chatbot fluency. +- Preserves existing ADR trust boundaries. + +Negative: + +- More work than a simple chat UI. +- Requires careful API design to avoid accidental mutation paths. +- Forces discipline in telemetry and artifact schemas. + +## Follow-up work + +- W-026: Workbench endpoint contract and local read-only API. +- W-027: Workbench frontend shell and navigation. +- W-028: Chat Surface + Trace Drawer read-only wiring. +- W-029: Proposal Review Queue read-only wiring. +- W-030: Eval Center + contemplation-quality display. +- W-031: Replay Theater artifact comparison. diff --git a/docs/workbench/README.md b/docs/workbench/README.md new file mode 100644 index 00000000..04b1b98c --- /dev/null +++ b/docs/workbench/README.md @@ -0,0 +1,160 @@ +# CORE Workbench v1 + +CORE Workbench is the first dedicated operator/auditor UI for CORE. + +It is intentionally designed as: + +- a cognition observatory, +- replay debugger, +- proposal review workstation, +- deterministic eval console, +- and audit surface. + +It is NOT intended to be a generic chatbot shell. + +--- + +# Document Index + +## Doctrine + +- `../decisions/ADR-0160-core-workbench-v1.md` + +Defines: + +- product doctrine +- trust boundaries +- architecture direction +- module scope +- acceptance criteria + +--- + +## UX Blueprint + +- `core-workbench-v1-blueprint.md` + +Defines: + +- visual philosophy +- interaction model +- module behavior +- navigation +- replay/proposal/eval UX + +--- + +## Implementation Plan + +- `implementation-plan.md` + +Defines: + +- W-026 through W-031 +- phase sequencing +- backend/frontend constraints +- release criteria + +--- + +## API Contract + +- `api-contract-v1.md` + +Defines: + +- read-only endpoint surface +- request/response envelopes +- mutation boundaries +- runtime/trace/proposal/eval/replay routes + +--- + +## Data Shapes + +- `data-shapes-v1.md` + +Defines: + +- typed UI-facing shapes +- proposal structures +- replay structures +- artifact references +- trust badges + +--- + +## UI Component Map + +- `ui-component-map.md` + +Defines: + +- page layout +- navigation structure +- shared components +- trace drawer structure +- proposal/eval/replay modules + +--- + +## Acceptance Gates + +- `acceptance-gates.md` + +Defines: + +- implementation gates +- red flags +- release conditions +- mutation restrictions + +--- + +# Core Principles + +## Replay before persuasion + +The UI should make replay evidence more important than aesthetic persuasion. + +## Calm by default + +The interface should remain quiet and precise. + +## Progressive disclosure + +The system should become more transparent the deeper an operator inspects. + +## Proposal is not ratification + +The UI must preserve ADR-0057 review doctrine. + +## Read-only first + +The first workbench phases prioritize visibility and auditability before any +mutation capability. + +--- + +# Initial Work Queue + +| Work Item | Goal | +|---|---| +| W-026 | Read-only API | +| W-027 | Frontend shell | +| W-028 | Chat + trace drawer | +| W-029 | Proposal queue | +| W-030 | Eval center | +| W-031 | Replay theater | + +--- + +# Guiding Question + +The correct operator reaction to the workbench should be: + +> “Wait… I can actually inspect and replay the cognition.” + +not: + +> “Cool chatbot.” diff --git a/docs/workbench/acceptance-gates.md b/docs/workbench/acceptance-gates.md new file mode 100644 index 00000000..1c7938ac --- /dev/null +++ b/docs/workbench/acceptance-gates.md @@ -0,0 +1,153 @@ +# CORE Workbench v1 — Acceptance Gates + +These gates define when the Workbench v1 planning and implementation phases are +acceptable. They intentionally privilege trust-boundary preservation over visual +polish. + +## Planning branch gates + +The planning branch is acceptable when it includes: + +- ADR defining doctrine and scope +- product/UX blueprint +- implementation plan +- API contract +- data-shape contract +- UI component map +- explicit non-goals +- explicit mutation boundaries +- work queue for W-026 through W-031 + +## W-026 — Read-only API gates + +Required: + +- local-only API defaults to `127.0.0.1` +- typed response schemas +- route tests +- path traversal tests for artifact readers +- no proposal accept/reject route +- no corpus mutation route +- no pack mutation route +- no workflow dispatch route +- no hidden background worker + +Acceptance command candidates: + +```bash +uv run python -m pytest tests/test_workbench_api.py -q +uv run python -m pytest tests/test_workbench_readers.py -q +``` + +## W-027 — Frontend shell gates + +Required: + +- TypeScript strict mode +- left navigation +- top runtime bar +- empty states for all modules +- API client shape aligned to `data-shapes-v1.md` +- no mutation buttons +- no fake/mock runtime as permanent architecture + +Acceptance command candidates: + +```bash +npm --prefix workbench/ui run typecheck +npm --prefix workbench/ui run build +``` + +## W-028 — Chat + Trace Drawer gates + +Required: + +- prompt/response flow +- trust badge row +- trace drawer collapsed by default +- grounding source visible +- trace hash visible when present +- mutation state visible +- raw trace behind explicit expansion + +Forbidden: + +- replacing user surface with telemetry surface +- hidden checkpointing beyond existing runtime behavior +- decorative thinking animation + +## W-029 — Proposal Queue gates + +Required: + +- list proposals +- filter by state +- inspect proposal detail +- display source provenance +- display replay evidence +- display proposed chain +- display suggested CLI copy command + +Forbidden: + +- accept button +- reject button +- workflow dispatch +- direct proposal mutation + +## W-030 — Eval Center gates + +Required: + +- list lanes +- display lane metadata +- run safe/read-only lanes +- show metrics +- show failures prominently +- show contemplation-quality details + +Forbidden: + +- sealed holdout execution without explicit sealed-eval configuration +- hidden saving of results +- mutation-capable workflow execution + +## W-031 — Replay Theater gates + +Required: + +- artifact selection +- original/replay hash comparison +- equivalence badge +- divergence display +- stable JSON viewer + +Forbidden: + +- non-deterministic replay display +- aesthetic animation presented as evidence + +## V1 release gates + +The v1 workbench is releasable when an operator can: + +1. start the local API, +2. open the UI, +3. see runtime status, +4. inspect a proposal without mutating it, +5. run/view contemplation-quality, +6. inspect trace/replay metadata, +7. perform a basic chat turn, +8. verify from UI copy and docs that no hidden ratification/mutation path exists. + +## Red flags + +Any of these should block merge: + +- UI can accept/reject proposals in v1 +- API exposes arbitrary path reads +- API invokes shell from user input +- frontend hardcodes fake eval/proposal data as product path +- mutation status is hidden +- replay status is shown without digest/evidence +- telemetry surfaces are confused with user-facing surfaces diff --git a/docs/workbench/api-contract-v1.md b/docs/workbench/api-contract-v1.md new file mode 100644 index 00000000..3cf458fe --- /dev/null +++ b/docs/workbench/api-contract-v1.md @@ -0,0 +1,363 @@ +# CORE Workbench API Contract v1 + +Status: draft +Scope: read-only local operator API + +This contract defines the initial API surface for CORE Workbench v1. + +The API exists to expose: + +- runtime observability, +- replay evidence, +- proposal visibility, +- eval visibility, +- and trace inspection + +without widening the mutation surface. + +## Transport + +- HTTP JSON +- optional websocket stream later +- local-first (`127.0.0.1`) in v1 + +## Response envelope + +All responses SHOULD follow: + +```json +{ + "ok": true, + "generated_at": "2026-05-26T00:00:00Z", + "data": {} +} +``` + +Errors: + +```json +{ + "ok": false, + "error": { + "code": "not_found", + "message": "artifact not found" + } +} +``` + +--- + +# Runtime + +## GET /health + +Purpose: + +Basic liveness/readiness. + +Response: + +```json +{ + "ok": true, + "data": { + "status": "ok" + } +} +``` + +--- + +## GET /runtime/status + +Purpose: + +Expose runtime metadata. + +Response: + +```json +{ + "ok": true, + "data": { + "backend": "rust", + "git_revision": "abc123", + "engine_state_present": true, + "checkpoint_revision": "abc123", + "revision_warning": false, + "active_session_id": "session-001" + } +} +``` + +--- + +# Chat + +## POST /chat/turn + +Purpose: + +Execute a normal runtime turn. + +Request: + +```json +{ + "prompt": "What does alpha cause?" +} +``` + +Response: + +```json +{ + "ok": true, + "data": { + "turn_id": "turn-001", + "surface": "alpha causes beta", + "grounding_source": "teaching", + "trace_hash": "sha256:...", + "proposal_state": null, + "replay_available": true + } +} +``` + +Important: + +This endpoint must use the existing runtime path. It must not introduce a +parallel persistence layer. + +--- + +# Trace + +## GET /trace/{turn_id} + +Purpose: + +Inspect trace evidence for a turn. + +Response: + +```json +{ + "ok": true, + "data": { + "turn_id": "turn-001", + "surface": "...", + "articulation_surface": "...", + "walk_surface": "...", + "trace_hash": "sha256:...", + "grounding_source": "teaching", + "replay_digest": "sha256:...", + "proposal_refs": [], + "admissibility": { + "rejected_attempts": 0 + } + } +} +``` + +--- + +# Replay + +## GET /replay/{artifact_id} + +Purpose: + +Return replay comparison information. + +Response: + +```json +{ + "ok": true, + "data": { + "artifact_id": "artifact-001", + "original_hash": "sha256:...", + "replay_hash": "sha256:...", + "equivalent": true, + "divergences": [] + } +} +``` + +--- + +# Proposals + +## GET /proposals + +Purpose: + +List proposal metadata. + +Response: + +```json +{ + "ok": true, + "data": { + "items": [ + { + "proposal_id": "proposal-001", + "state": "pending", + "source_kind": "contemplation", + "replay_equivalent": true, + "created_at": "2026-05-26T00:00:00Z" + } + ] + } +} +``` + +--- + +## GET /proposals/{proposal_id} + +Purpose: + +Detailed proposal inspection. + +Response: + +```json +{ + "ok": true, + "data": { + "proposal_id": "proposal-001", + "state": "pending", + "source_kind": "contemplation", + "replay_equivalent": true, + "proposed_chain": {}, + "replay_evidence": {}, + "downstream_effect": {}, + "artifact_refs": [] + } +} +``` + +Forbidden: + +- accept endpoint +- reject endpoint +- mutation endpoint + +--- + +# Evals + +## GET /evals + +Purpose: + +List eval lanes. + +Response: + +```json +{ + "ok": true, + "data": { + "lanes": [ + "cognition", + "learning-arc", + "contemplation-quality" + ] + } +} +``` + +--- + +## POST /evals/run + +Purpose: + +Run a read-only eval lane. + +Request: + +```json +{ + "lane": "contemplation-quality" +} +``` + +Response: + +```json +{ + "ok": true, + "data": { + "lane": "contemplation-quality", + "passed": true, + "metrics": {} + } +} +``` + +Constraints: + +- only lanes explicitly marked safe/read-only may run through the API +- mutation-capable workflows are forbidden in v1 + +--- + +# Artifacts + +## GET /artifacts + +Purpose: + +Enumerate readable runtime artifacts. + +Response: + +```json +{ + "ok": true, + "data": { + "items": [] + } +} +``` + +--- + +## GET /artifacts/{artifact_id} + +Purpose: + +Read structured artifact content. + +Constraints: + +- artifacts must be repo-root constrained +- no arbitrary path traversal +- no arbitrary shell execution + +--- + +# Security Constraints + +## Forbidden in v1 + +- proposal mutation +- corpus mutation +- workflow dispatch +- arbitrary file reads +- remote execution +- arbitrary command execution +- hidden background jobs + +## Required + +- deterministic serialization +- stable artifact ids +- digest visibility +- explicit provenance where available +- read-only route tests diff --git a/docs/workbench/core-workbench-v1-blueprint.md b/docs/workbench/core-workbench-v1-blueprint.md new file mode 100644 index 00000000..c8f0ab76 --- /dev/null +++ b/docs/workbench/core-workbench-v1-blueprint.md @@ -0,0 +1,397 @@ +# CORE Workbench v1 — Product + UX Blueprint + +## Vision + +CORE Workbench is not a chatbot shell. + +It is: + +- a cognition observatory, +- a replay theater, +- a proposal review station, +- a deterministic eval console, +- and a trust-boundary visualizer. + +The system should feel: + +- calm, +- precise, +- inspectable, +- authoritative, +- and progressively deep. + +The design target is not “AI toy.” + +The design target is: + +> a cognition operating system for engineers and auditors. + +--- + +# UX Principles + +## 1. Calm by default + +Most AI products overload the first screen. + +CORE should not. + +The first screen should show: + +- prompt, +- response, +- grounding, +- replay status, +- proposal/refusal state, +- and almost nothing else. + +Everything deeper should open progressively. + +--- + +## 2. Evidence before persuasion + +Never ask the operator to trust the answer emotionally. + +Always allow: + +- replay, +- provenance inspection, +- trace inspection, +- and mutation-boundary verification. + +--- + +## 3. The deeper you go, the more transparent it becomes + +Most frontier systems become more opaque under inspection. + +CORE should become more legible. + +--- + +## 4. No dashboard soup + +No giant analytics walls. + +Every component must answer: + +- what happened? +- why? +- can it replay? +- what mutated? +- who has authority? + +If not, remove it. + +--- + +# Information Architecture + +## Left navigation + +Minimal: + +- Chat +- Replay +- Proposals +- Evals +- Artifacts +- Runtime + +No nested maze. + +--- + +# Module Design + +# 1. Chat Surface + +## Goal + +Minimal interaction layer. + +## Layout + +### Top Bar + +- runtime mode +- backend +- replay status +- session id +- checkpoint status + +### Main Conversation Area + +Responses show: + +- surface +- grounding source +- trace/replay badge +- proposal state +- refusal state + +### Right-side Expandable Drawer + +Collapsed by default. + +Contains: + +- articulation surface +- walk surface +- trace metadata +- replay digest +- admissibility evidence +- grounding provenance + +--- + +# 2. Replay Theater + +## Goal + +Demonstrate deterministic replay visually. + +## Core interaction + +Select: + +- run +- turn +- artifact +- eval result + +Then compare: + +| Original | Replay | +|---|---| +| trace hash | trace hash | +| surface | surface | +| grounding | grounding | +| replay status | replay status | +| divergences | divergences | + +## Visual style + +Extremely restrained. + +This should feel like: + +- debugger, +- theorem proof, +- mission control. + +Not: + +- animated AI theater. + +--- + +# 3. Proposal Review Queue + +## Goal + +Make proposal lifecycle understandable. + +## Queue item fields + +- proposal_id +- state +- source_kind +- replay_equivalent +- downstream effect +- contemplation evidence +- proposal chain +- proposal digest +- created_at + +## Detail panel + +Should support: + +- replay evidence +- downstream comparison +- linked eval artifacts +- cognition chain diff +- provenance graph + +## Important + +V1 is read-only. + +No accept/reject buttons yet. + +Only: + +- copy CLI command +- open artifact +- inspect replay + +--- + +# 4. Eval Center + +## Goal + +Turn deterministic evals into visible operational quality. + +## Initial lanes + +- cognition +- contemplation-quality +- learning-arc + +## Per-result display + +- pass/fail +- metric table +- drift +- replay status +- source digest +- linked artifacts + +## Critical UX detail + +Failures must be more visible than successes. + +The system should encourage investigation, not vanity metrics. + +--- + +# 5. Runtime / Artifact Explorer + +## Goal + +Expose runtime state without mutating it. + +## Displays + +- engine_state manifests +- reboot events +- revision mismatch warnings +- proposal logs +- eval results +- contemplation reports +- telemetry artifacts + +Should feel: + +- filesystem-like, +- deterministic, +- structured, +- low-noise. + +--- + +# Design Language + +## Visual direction + +Closest inspirations: + +- Linear +- Raycast +- Apple developer tooling +- GitHub PR review +- Mission control + +Not: + +- neon AI dashboards +- glassmorphism overload +- streaming gimmicks +- fake thinking animations + +--- + +# Motion + +Use motion only for: + +- panel expansion +- replay comparison transitions +- trace drilldown +- timeline navigation + +No decorative movement. + +--- + +# Typography + +Prioritize: + +- legibility, +- hierarchy, +- audit readability, +- JSON readability, +- terminal readability. + +--- + +# Color Philosophy + +Mostly monochrome. + +Use color only for: + +- replay success/failure +- mutation state +- refusal state +- proposal state +- eval severity + +Color should mean something operational. + +--- + +# Backend API Shape + +## Read-only endpoints + +Initial workbench API should expose: + +- GET /runtime/status +- GET /runtime/checkpoints +- POST /chat/turn +- GET /chat/turn/{id} +- GET /trace/{id} +- GET /replay/{id} +- GET /proposals +- GET /proposals/{id} +- GET /evals +- POST /evals/run +- GET /artifacts/{id} + +No mutation endpoints in v1. + +--- + +# Security / Doctrine + +## Must preserve + +- ADR-0057 proposal boundaries +- ADR-0151 contemplation review doctrine +- ADR-0159 read-only eval semantics +- ADR-0146 persistence guarantees + +## Must never do silently + +- mutate corpus +- accept proposals +- mutate packs +- trigger remote workflow commits +- mutate engine_state outside runtime + +--- + +# Success Condition + +The correct reaction from an engineer seeing CORE Workbench is: + +> “Wait… I can actually inspect and replay the cognition?” + +not: + +> “Cool chatbot.” diff --git a/docs/workbench/data-shapes-v1.md b/docs/workbench/data-shapes-v1.md new file mode 100644 index 00000000..cf4add27 --- /dev/null +++ b/docs/workbench/data-shapes-v1.md @@ -0,0 +1,254 @@ +# CORE Workbench v1 — Data Shape Contracts + +Status: draft + +This document defines the canonical UI-facing shapes for CORE Workbench v1. +They are intentionally narrower than internal runtime structures. + +The workbench should expose enough information to audit and replay behavior +without leaking unnecessary internals into the UI contract. + +## Naming conventions + +- IDs are stable strings. +- Hashes/digests should include an algorithm prefix when possible. +- Timestamps are ISO-8601 UTC strings. +- Unknown values should be explicit (`"unknown"`) rather than omitted when they + materially affect audit interpretation. + +--- + +# Envelope + +```ts +export type WorkbenchResponse = + | { + ok: true; + generated_at: string; + data: T; + } + | { + ok: false; + generated_at: string; + error: WorkbenchError; + }; + +export type WorkbenchError = { + code: + | "bad_request" + | "not_found" + | "unsupported" + | "read_error" + | "eval_failed" + | "runtime_unavailable"; + message: string; + detail?: unknown; +}; +``` + +--- + +# Runtime + +```ts +export type RuntimeStatus = { + backend: "numpy" | "mlx" | "rust" | "unknown"; + git_revision: string; + engine_state_present: boolean; + checkpoint_revision: string | "unknown"; + revision_warning: boolean; + active_session_id: string | null; + mutation_mode: "read_only" | "runtime_turn"; +}; +``` + +`mutation_mode` is UI-facing. It does not redefine runtime doctrine. It tells +an operator whether the current surface is purely reading artifacts or executing +normal runtime turns. + +--- + +# Chat + +```ts +export type ChatTurnRequest = { + prompt: string; +}; + +export type ChatTurnSummary = { + turn_id: string; + surface: string; + grounding_source: string | null; + trace_hash: string | null; + replay_available: boolean; + proposal_state: "none" | "pending" | "accepted" | "rejected" | null; + mutation_state: "none" | "transient" | "proposal_only" | "ratified"; +}; +``` + +`surface` is the user-visible response. The UI must not substitute telemetry +surfaces into the user surface. + +--- + +# Trace + +```ts +export type TraceDetail = { + turn_id: string; + surface: string; + articulation_surface: string | null; + walk_surface: string | null; + trace_hash: string | null; + replay_digest: string | null; + grounding_source: string | null; + proposal_refs: string[]; + candidate_refs: string[]; + admissibility: { + rejected_attempts: number | null; + exhausted: boolean | null; + }; + raw?: unknown; +}; +``` + +The trace drawer may show `raw`, but only behind an explicit expand action. + +--- + +# Proposal + +```ts +export type ProposalSummary = { + proposal_id: string; + state: "pending" | "accepted" | "rejected" | "unknown"; + source_kind: string; + replay_equivalent: boolean | null; + created_at: string | null; + downstream_effect: "unknown" | "none" | "observed"; +}; + +export type ProposalDetail = ProposalSummary & { + proposed_chain: unknown; + replay_evidence: unknown; + source: unknown; + evidence: unknown[]; + artifact_refs: ArtifactRef[]; + suggested_cli?: string; +}; +``` + +V1 may include `suggested_cli` for copy-only operator review. It must not +execute it. + +--- + +# Eval + +```ts +export type EvalLaneSummary = { + lane: string; + versions: string[]; + read_only: boolean; + description: string | null; +}; + +export type EvalRunRequest = { + lane: string; + version?: string; + split?: "dev" | "public" | "holdout"; +}; + +export type EvalRunResult = { + lane: string; + version: string; + split: string; + passed: boolean | null; + metrics: Record; + cases: unknown[]; + source_digest?: string; +}; +``` + +The API should initially allow only explicitly safe lanes. `holdout` should be +disabled unless the backend proves the sealed-eval path is configured. + +--- + +# Replay + +```ts +export type ReplayComparison = { + artifact_id: string; + original_hash: string | null; + replay_hash: string | null; + equivalent: boolean; + divergences: ReplayDivergence[]; +}; + +export type ReplayDivergence = { + path: string; + original: unknown; + replay: unknown; + severity: "info" | "warning" | "failure"; +}; +``` + +--- + +# Artifact + +```ts +export type ArtifactRef = { + artifact_id: string; + kind: + | "trace" + | "eval_result" + | "proposal" + | "contemplation_report" + | "telemetry" + | "engine_state_manifest" + | "unknown"; + path: string; + digest: string | null; + created_at: string | null; +}; + +export type ArtifactDetail = ArtifactRef & { + content_type: "json" | "jsonl" | "text" | "unknown"; + content: unknown; +}; +``` + +Artifact paths must be repo-root constrained. The backend must never honor an +arbitrary user-supplied filesystem path. + +--- + +# UI state tags + +These are display-only semantic tags. + +```ts +export type TrustBadge = + | "replay_passed" + | "replay_failed" + | "grounded" + | "ungrounded" + | "pending_review" + | "mutation_none" + | "mutation_transient" + | "mutation_ratified" + | "revision_warning" + | "refusal"; +``` + +Colors should be mapped only to these operational meanings. + +--- + +# Backward compatibility rule + +Internal runtime objects may evolve faster than the workbench UI. The API layer +must normalize into these v1 shapes so the frontend does not depend on private +runtime structure. diff --git a/docs/workbench/implementation-plan.md b/docs/workbench/implementation-plan.md new file mode 100644 index 00000000..a20916c1 --- /dev/null +++ b/docs/workbench/implementation-plan.md @@ -0,0 +1,205 @@ +# CORE Workbench v1 — Implementation Plan + +This plan intentionally starts with planning and read-only observability before +any mutating workflow. The purpose is to preserve CORE's existing ADR trust +boundaries while making the system legible to operators, engineers, and auditors. + +## Work queue + +| Work item | Title | Scope | Mutation allowed? | +|---|---|---|---| +| W-026 | Read-only API contract | endpoint schemas + local server surface | No | +| W-027 | Frontend shell | navigation, layout, design tokens, empty states | No | +| W-028 | Chat + trace drawer | basic chat, turn metadata, trace drawer | Runtime turn only | +| W-029 | Proposal queue | proposal-log read model, detail view, CLI-copy affordance | No | +| W-030 | Eval center | lane discovery, run/read evals, result viewer | Eval result writes only if CLI already supports it | +| W-031 | Replay theater | artifact selection, digest compare, replay status UI | No | + +## Phase sequencing + +### Phase 0 — Planning package + +Deliverables: + +- ADR-0160 +- Product/UX blueprint +- Implementation plan +- API contract +- data-shape contract +- acceptance gates + +No code beyond docs. + +### Phase 1 — Read-only API skeleton + +Goal: expose a narrow local workbench API without introducing new mutation paths. + +Deliverables: + +- `workbench/api.py` +- `workbench/schemas.py` +- `workbench/readers.py` +- route tests with temp fixtures +- CLI command: `core workbench api --host 127.0.0.1 --port 8765` + +Initial endpoints: + +- `GET /health` +- `GET /runtime/status` +- `GET /artifacts` +- `GET /proposals` +- `GET /evals` +- `GET /evals/{lane}` + +Strict rule: no proposal accept/reject endpoints. + +### Phase 2 — Frontend shell + +Goal: create a minimal, elegant workbench UI shell. + +Proposed layout: + +```text ++--------------------------------------------------------------+ +| CORE Workbench runtime status replay status | ++----------------------+---------------------------------------+ +| Chat | Main Panel | +| Replay | | +| Proposals | | +| Evals | | +| Artifacts | | +| Runtime | | ++----------------------+---------------------------------------+ +``` + +Deliverables: + +- `workbench/ui/` or `apps/workbench/` depending repo convention +- React/Vite/TypeScript shell +- read-only API client +- navigation +- empty states +- no mocks as long-lived architecture; fixtures only in tests/story views + +### Phase 3 — Chat + Trace Drawer + +Goal: prove a small chat surface can expose trace metadata without becoming a +chatbot clone. + +Deliverables: + +- prompt box +- response surface +- trust badges +- expandable trace drawer +- grounding/provenance summary +- telemetry line link when available + +Mutation boundary: + +A live chat turn may use the existing runtime path. The workbench must label +any runtime checkpointing clearly and must not add hidden persistence beyond +runtime behavior already governed by ADR-0146/0150. + +### Phase 4 — Proposal Queue + +Goal: make proposal lifecycle visible. + +Deliverables: + +- pending/accepted/rejected state filters +- proposal detail panel +- replay-equivalence evidence +- source provenance +- proposed chain display +- copyable local CLI command for review + +Forbidden in v1: + +- accept button +- reject button +- workflow dispatch +- direct corpus mutation + +### Phase 5 — Eval Center + +Goal: turn deterministic evals into a UI surface. + +Deliverables: + +- lane list +- run lane action for read-only lanes +- result JSON viewer +- metric cards +- case failures +- contemplation-quality detail view + +Mutation boundary: + +If `--save` equivalent behavior is exposed, it must be explicit and documented; +otherwise first pass should run and display without writing. + +### Phase 6 — Replay Theater + +Goal: make deterministic replay obvious. + +Deliverables: + +- artifact selector +- original vs replay comparison +- trace hash / digest comparison +- divergence panel +- pass/fail badge + +## API design constraints + +- All schemas must be explicit dataclasses or pydantic models. +- No raw arbitrary filesystem reads from user-provided paths. +- Artifact readers must be rooted under known repo directories. +- Every response should contain enough metadata to audit source path, digest, + and timestamp when available. +- Mutation endpoints are forbidden in v1. + +## Frontend constraints + +- TypeScript strict mode. +- API client generated or manually typed from schema contract. +- No global mutable singleton for runtime state. +- No decorative animation as status. +- Color only communicates operational meaning. +- JSON viewers must preserve stable key ordering when possible. + +## Test expectations + +Each implementation PR must include: + +- route-level tests for API endpoints +- schema serialization tests +- permission/path traversal tests for artifact readers +- frontend smoke/build check once UI exists +- no-mutation regression checks for read-only routes + +## Release criteria for v1 + +The v1 workbench is accepted when an operator can: + +1. start the local workbench server, +2. open the UI, +3. inspect runtime status, +4. run or view contemplation-quality, +5. inspect a proposal without accepting it, +6. inspect a trace/replay artifact, +7. perform a basic chat turn, +8. verify no hidden mutation path exists. + +## Explicit deferrals + +- auth +- multi-user collaboration +- cloud deployment +- mobile UI +- proposal accept/reject buttons +- workflow dispatch +- packaged desktop app +- public marketing site +- arbitrary plugin/tool execution diff --git a/docs/workbench/ui-component-map.md b/docs/workbench/ui-component-map.md new file mode 100644 index 00000000..14006c03 --- /dev/null +++ b/docs/workbench/ui-component-map.md @@ -0,0 +1,292 @@ +# CORE Workbench v1 — UI Component Map + +This document maps conceptual modules to concrete UI components. + +The purpose is to keep the implementation coherent and prevent uncontrolled UI +sprawl. + +--- + +# Global Layout + +```text ++---------------------------------------------------------------+ +| TopBar | ++-------------+-------------------------------------------------+ +| SideNav | MainContent | +| | | +| | | ++-------------+-------------------------------------------------+ +| StatusBar | ++---------------------------------------------------------------+ +``` + +--- + +# TopBar + +## Responsibilities + +- runtime status +- backend indicator +- replay health +- revision warning badge +- active session id +- current branch/revision + +## Components + +- `RuntimeBackendBadge` +- `ReplayHealthBadge` +- `RevisionWarningBadge` +- `SessionIndicator` +- `GitRevisionPill` + +## Anti-goals + +- giant metrics wall +- animated status spam +- streaming logs + +--- + +# Side Navigation + +## Sections + +- Chat +- Replay +- Proposals +- Evals +- Artifacts +- Runtime + +## Components + +- `NavItem` +- `NavSection` +- `UnreadIndicator` (optional future) + +Navigation should feel closer to Linear/Raycast than a cloud admin portal. + +--- + +# Chat Screen + +## Layout + +```text ++------------------------------------------------+ +| Conversation | +| | +| Prompt | +| Response | +| Trust badges | +| | ++----------------------+-------------------------+ +| Composer | TraceDrawer | ++----------------------+-------------------------+ +``` + +## Components + +### Conversation + +- `ConversationTimeline` +- `PromptBubble` +- `ResponseSurface` +- `TrustBadgeRow` + +### Composer + +- `PromptInput` +- `SubmitButton` +- `RuntimeModeIndicator` + +### Trace Drawer + +Collapsed by default. + +Components: + +- `TraceHashCard` +- `GroundingCard` +- `ReplayDigestCard` +- `ProposalReferenceCard` +- `AdmissibilityCard` +- `RawTraceViewer` + +--- + +# Replay Theater + +## Layout + +```text ++--------------------+---------------------------+ +| Artifact Selector | Replay Comparison | +| | | +| | Original vs Replay | +| | | ++--------------------+---------------------------+ +``` + +## Components + +- `ArtifactList` +- `ReplayComparisonPanel` +- `ReplayStatusBadge` +- `ReplayDiffViewer` +- `ReplayMetadataTable` + +## Important + +No fake “thinking” animations. + +Replay evidence should feel: + +- precise, +- quiet, +- inspectable. + +--- + +# Proposal Queue + +## Layout + +```text ++----------------------+--------------------------+ +| Proposal List | Proposal Detail | +| | | +| pending | replay evidence | +| accepted | proposed chain | +| rejected | downstream effect | +| | provenance | ++----------------------+--------------------------+ +``` + +## Components + +### List + +- `ProposalTable` +- `ProposalStateBadge` +- `ProposalReplayBadge` + +### Detail + +- `ProposalSummaryCard` +- `ReplayEvidenceCard` +- `ProposalChainViewer` +- `ProposalProvenanceViewer` +- `SuggestedCLIBox` + +## Forbidden + +- accept button +- reject button +- workflow trigger + +--- + +# Eval Center + +## Layout + +```text ++--------------------+----------------------------+ +| Eval Lane List | Eval Result Viewer | +| | | +| cognition | metrics | +| contemplation | failures | +| learning-arc | artifacts | ++--------------------+----------------------------+ +``` + +## Components + +- `EvalLaneCard` +- `EvalRunButton` +- `EvalMetricGrid` +- `EvalFailureViewer` +- `EvalArtifactLink` + +## UX Rule + +Failures should be easier to inspect than successes. + +--- + +# Artifact Explorer + +## Layout + +```text ++--------------------+----------------------------+ +| Artifact Tree | Artifact Viewer | ++--------------------+----------------------------+ +``` + +## Components + +- `ArtifactTree` +- `ArtifactMetadataPanel` +- `ArtifactJSONViewer` +- `ArtifactTextViewer` +- `DigestBadge` + +--- + +# Runtime Screen + +## Components + +- `EngineStateCard` +- `CheckpointCard` +- `RevisionMismatchAlert` +- `RebootEventTimeline` +- `BackendStatusCard` + +--- + +# Shared Components + +## Badges + +- replay passed/failed +- grounded +- pending review +- refusal +- mutation state +- revision warning + +## Data viewers + +- `StableJsonViewer` +- `DigestValue` +- `MetadataTable` +- `Timestamp` + +## UX Principle + +All shared components should prefer: + +- stable layout, +- low motion, +- readable density, +- deterministic ordering. + +--- + +# Components intentionally deferred + +Not allowed in v1: + +- AI avatar system +- agent marketplace +- plugin panels +- node graph builders +- workflow automation canvas +- hidden background orchestration +- animated cognition theater