Commit graph

7 commits

Author SHA1 Message Date
Shay
4cba6f488c feat(workbench): Wave M Phase C legibility — pipeline record, contemplation, identity continuity
Lands the Phase C "make cognition legible" slice plus Phase A residue, all
backend-reader-first over real engine data (no theater, read-only doctrine
intact, zero serving-path imports).

C1-a — Cognitive pipeline record (persistence-first, per #729 worthiness edit):
  - workbench/pipeline_record.py: curated CognitivePipelineRecord over the real
    CognitiveTurnResult (input → intent → proposition_graph → articulation_target
    → realizer → walk_telemetry → trace_hash). Raw field multivectors are
    DELIBERATELY excluded; _assert_no_raw_field_payload recursively rejects raw
    field keys, and validate_pipeline_record fails closed on missing/duplicate
    stages, non-recorded status, or dangling edges — the UI can never receive a
    partial record that claims to be complete.
  - test_workbench_pipeline_record.py: non-vacuous guards — missing stage,
    monkeypatched new required stage, and injected raw {"F": [...]} each raise.

C2-a — Contemplation as a process: /contemplation route over real persisted
  contemplation/runs/*.json (glob reader; honest-empty when absent).

C4-a — Identity continuity (L10/L11): RunDetail.identity_continuity + Runs
  Identity tab, sourced from the real core.engine_identity (engine_identity /
  parent_engine_identity lineage relation, re-derived to verify).

Demo Theater: renders backend-owned proof-promotion + entailment DAGs.

Phase A residue: density preference wired end-to-end (settings → shell → tokens);
  cross-route consistency touch-ups.

Infra: local API CORS now echoes only validated 127.0.0.1/localhost origins
  (hostname-checked, not arbitrary reflection) so Vite fallback ports work.
  Route chunk-split keeps the build warning-free.

Cleanup: corrected the stale ADR-0175 practice-lane assertions (build_report is
  6 correct / 0 wrong / 44 refused after the current serving lane; wrong=0 held)
  and the two registry-derived count tests (LeftNav + CommandPalette 12 → 13 for
  the new Contemplation route).

Docs: runtime_contracts.md (pipeline-record contract), UI-UX-GUIDE,
  api-contract-v1, data-shapes-v1, wave-M-worthiness, phase-a-residue-ledger.

Validation: 106 workbench/practice Python tests green (incl. wrong=0 lane +
  pipeline-record fail-closed guards); 459/459 frontend; pnpm build clean;
  git diff --check clean. No generate.derivation / reliability_gate / stream /
  field.propagate / vault.store imports.
2026-06-13 15:44:31 -07:00
Shay
bdb294eac3 feat(workbench): land B3.5-b/c/d/e — calibration evidence subject, B4a leeway gate, docs; runner-reproducible practice artifact
Completes the Wave M B3.5 consolidation slice (b–e), built on #728.

B3.5-b — calibration as a first-class evidence subject (`calibration_class`,
address `calibration:<class_name>`): RightInspector projection + Evidence
Chain Rail semantics (serving-discipline evidence, not runtime truth).

B3.5-c / B4a — nullable `LeewayEvidence` read model threaded through turn,
replay, cognition-proposal, and math-proposal surfaces, with a shared
absence-honest card. B4 is gated correctly: the tuple exists in typed data but
no producer populates it, so the card renders absence (verified: no non-null
producer in workbench/core/chat).

B3.5-d/e — UI-UX-GUIDE.md, b4-leeway-feasibility-gate.md, phase-a-residue-ledger.md.

Practice artifact — earn-it-for-real (runner-reproducible). The committed
`report.json` (additive earns PROPOSE @0.861, 95/5/50) is now emitted by a
deterministic runner rather than copied from the queue. `propose_runner`
gains `regenerate_practice_artifacts()`, which runs ONE sealed `resolve_pooled`
practice pass and writes BOTH report.json (the per-class ledger the calibration
reader consumes) and ratification_queue.json — two projections of one ledger,
coherent by construction and byte-reproducible. `runner.main()` delegates to
it (lazy import, no cycle), so both entry points produce the identical pair.
This closes the gap where a hand-copied report.json agreed with the queue but
no runner produced it. `resolve_pooled` is the aggressive sealed PROPOSE-regime
scorer (proposal-only/HITL, unsafe for serving, legitimate for
attempt-and-eliminate); wrong=5 is the sealed-practice learning signal, NOT the
serving wrong=0. No serving/derivation/reliability_gate source touched; the
practice lane is not in the serving-frozen SHA gate.

Validated:
- python -m pytest tests/test_workbench_{calibration,journal,replay,schemas}.py -> 31 passed
- python -m pytest tests/ -k "workbench or propose or learning_arena or practice"
  -> 190 passed (3 failing tests in test_adr_0175_phase2_practice_lane.py are
  PRE-EXISTING reds on clean origin/main: stale 4/0/46 assertions on build_report,
  which this change does not touch)
- report.json + ratification_queue.json: deterministic (run1==run2) and
  reproduced byte-identically by both `python -m ...runner` and `...propose_runner`
- pnpm build green; 144 UI tests across calibration/leeway/evidence/replay/
  doctrine-gates/routes-docs-drift all pass
2026-06-13 07:36:44 -07:00
Shay
1fe56e9b6f feat(workbench): calibration + serving-metrics readers — the gold-tether loop, visible (Wave M B1)
First Wave M / Phase B piece (GATING): read-only backend that makes the
calibrated-learning / serving-discipline loop inspectable — 'the engine
earns the right to guess', ADR-0175.

The workbench computes NONE of these numbers:
- GET /calibration/classes — per-class gold-tether view from the persisted
  practice arena ledger (evals/gsm8k_math/practice/v1/report.json per_class).
  Each class's reliability_floor is the engine's own one-sided Wilson
  conservative_floor (via ClassTally.reliability); PROPOSE (θ=0.85) / SERVE
  (θ=0.99) license verdicts come from core.reliability_gate.license_for.
  Failures-first ordering. A test proves the reader's floor equals a direct
  conservative_floor() call — no re-implementation.
- GET /serving/metrics — the live correct/refused/wrong counts read unchanged
  from the committed train_sample + holdout_dev report.json (currently
  4/46/0 and 5/495/0 — wrong=0). Never re-runs a lane.

Honest current state: the committed practice ledger's three classes
(additive/divisive/multiplicative) are all below N_MIN=10, so none has
earned a license yet — the reader shows exactly that, no fake green light.

- workbench/calibration.py: pure readers; imports core.reliability_gate;
  EvidenceUnavailableError -> 501 (fail-closed) when the artifact is absent.
- schemas + TS mirrors (CalibrationClass, ServingMetrics); both snapshots
  regenerated (deterministic); both drift gates pass.
- trust boundary: read-only over committed artifacts + engine-owned
  derivation; no execution, no mutation, no license ever changed.

Verified: 30 Python tests (incl. the no-reimplementation proof + fail-closed),
390 vitest, both schema drift gates, snapshots deterministic.
2026-06-13 00:38:16 -07:00
Shay
fd0d7f1f6a finish wave r3 ui and trace integrity 2026-06-12 23:25:24 -07:00
Shay
6d1f6bba0c feat: implement CORE Workbench W3 Ratification Corridor 2026-06-03 20:13:36 -07:00
Shay
a612038d41
feat(W-028): chat surface + trace drawer (#303) 2026-05-26 13:22:11 -07:00
Shay
cdead696ed
feat(W-027): Workbench frontend shell — five-region grid + ten empty routes + live StatusFooter (ADR-0160 / ADR-0162) (#299)
- Five-region CSS grid shell (TopBar, LeftNav, Main, StatusFooter; Inspector collapsed)
- Ten placeholder routes with EmptyState cli/string variants and ApiErrorBoundary
- Extended EmptyState API: nextAction accepts string (button) | { kind: "cli", command }
- Updated CommandPalette: real fuzzy search over three commands (Chat, Proposals, Evals)
  with ↑/↓/Enter keyboard nav and useInRouterContext degradation for Branch 1 preview
- TanStack Query hooks for all W-026 endpoints; 30s polling on runtime/status
- TypeScript mirror of workbench/schemas.py at src/types/api.ts
- StatusFooter: mutation_mode badge, git_revision (copy on click), checkpoint_revision
  with amber warning + ADR-0157/ADR-0158 expansion note
- TopBar: CORE Workbench wordmark, ⌘K palette trigger, connection pill
- ApiErrorBoundary (class component) catches WorkbenchApiError → ErrorState
- scripts/dump-api-schemas.py: AST-based Python dataclass field extractor
- workbench-ui/api-schema-snapshot.json: checked-in drift sentinel
- 51 tests across 10 files (0 failures); enum-coverage 4/4; clean vite build
2026-05-26 12:09:12 -07:00