docs(session): 2026-05-26 corridor closure — first GSM8K lift + workbench operational (#305)
Captures today's end-to-end closure of the math architecture corridor (ADR-0163 Phase A → B → C → D + operator ratification, 15 PRs, first non-zero GSM8K correct count: 0 → 3 with wrong = 0 preserved) and the workbench surface (W-026 API + ADR-0162 design system + W-027 shell + W-028 chat surface) becoming operational end-to-end. Added: - docs/sessions/SESSION-2026-05-26-corridor-closure.md — full session ledger, per-fork accomplishments, three lifted GSM8K cases, unexpected-positive observation about skip-only wiring, deferred work, architectural state at close. Updated: - docs/master-plan-post-substrate-audit.md — 2026-05-26 amendment banner pointing to the session recap; historical 2026-05-24 plan preserved below. - docs/PROGRESS.md — appended a new section capturing the day's 15 PRs by fork (math, workbench, HITL), the first-lift counts, and what stays open. - docs/decisions/ADR-0163-gsm8k-path-to-mastery.md — Round 1 amendment with the actual lift evidence, the three lifted cases, the capability-axis preservation, and the unexpected-positive note about skip-only wiring doing more than projected. Scope: docs-only. No runtime, no tests, no code changes.
This commit is contained in:
Shay
GitHub
parent
062d53f151
commit
65ccb8eeee
4 changed files with 332 additions and 0 deletions
|
|
@ -905,3 +905,73 @@ ADR-0119.5, ADR-0119.6, ADR-0119.7, ADR-0119.8.
|
|||
| Tool use (typed deterministic operators) | **Resolved 2026-05-16 — ADR-0018** (typed deterministic operators, no external IO) | Before Phase 3 ✓ |
|
||||
| Code generation (first-class target) | Open | Before Phase 5 |
|
||||
| Embodiment (sensorium gates) | Open | Phase 5 |
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-26 — Math Architecture Corridor Closure + Workbench Operational
|
||||
|
||||
**Status:** Math corridor end-to-end demonstrated ✓ — first non-zero GSM8K lift produced; Workbench operational end-to-end ✓.
|
||||
**Session log:** [SESSION-2026-05-26-corridor-closure.md](sessions/SESSION-2026-05-26-corridor-closure.md)
|
||||
|
||||
### First measurable lift on GSM8K train_sample (50 cases)
|
||||
|
||||
```
|
||||
correct: 3 (up from 0 — first non-zero correct count ever)
|
||||
refused: 47 (down from 50)
|
||||
wrong: 0 (unchanged — invariant holds)
|
||||
```
|
||||
|
||||
Three cases lifted: `gsm8k-train-sample-v1-0014` (Bob's oysters, ans=240), `0018` (Xavier's football, ans=16), `0042` (Ella's apples, ans=30). Capability-axis floor (G1..G5 + S1) preserved at wrong=0 post-ratification.
|
||||
|
||||
### Math architecture (ADR-0163) — corridor closed end-to-end
|
||||
|
||||
| Phase | PR | Deliverable |
|
||||
|---|---|---|
|
||||
| A | #297 | `evals/refusal_taxonomy/` — 9-category shape taxonomy, 72% of 50 refused statements categorized |
|
||||
| B | #298 | `teaching/admissibility_exemplars/` — 60 hand-authored seeds across 3 categories |
|
||||
| C | #301 | `teaching/recognizer_synthesis.py` + admissibility replay gate; deterministic, narrowness-preserving, rules-only synthesis |
|
||||
| D | #302 | `generate/recognizer_registry.py` + `generate/recognizer_match.py` + one-line skip-only guard in `math_candidate_graph.py:455-470`; wrong=0 safe by construction |
|
||||
| Ratify | #304 | Operator accepted all three Phase C recognizers via `core teaching review --accept` |
|
||||
|
||||
The thesis ("decodes, not generates") is now demonstrated as actionable: operator-authored exemplars → engine-derived recognizers → operator ratification → measurable admissibility lift, with no algebra changes, no pack mutations, no hidden normalization.
|
||||
|
||||
### Workbench (ADR-0160 / 0162) — operational end-to-end
|
||||
|
||||
```bash
|
||||
uv run core workbench api # http://127.0.0.1:8765
|
||||
cd workbench-ui && pnpm dev # http://127.0.0.1:5173
|
||||
```
|
||||
|
||||
| Branch | PR | Deliverable |
|
||||
|---|---|---|
|
||||
| W-026 | #292 | read-only HTTP API: 7 GET endpoints + 1 whitelisted POST, path-traversal protection, MAX_ARTIFACT_BYTES guard, single-operator concurrency lock |
|
||||
| ADR-0162 | #293 | design system doctrine: semantic tokens, dark theme, motion rules, `StableJsonViewer` six invariants, empty/error/loading contract |
|
||||
| Branch 1 | #295 | substrate scaffold: tokens, `StableJsonViewer`, enum-bound badges (4 enums, build-time AST coverage check), `/preview` page |
|
||||
| W-027 | #299 | five-region shell, ten empty routes, live `StatusFooter`, `CommandPalette`, `ApiErrorBoundary`, schema-drift sentinel |
|
||||
| W-028 | #303 | first route with real content: `/chat` with PromptComposer + ResponseCard + EvidenceStrip + TraceDrawer (three layers); `POST /chat/turn` with `threading.Lock` + body/prompt caps |
|
||||
|
||||
### HITL queue (ADR-0161) — read-only CLI shipped
|
||||
|
||||
| Step | PR | Deliverable |
|
||||
|---|---|---|
|
||||
| Scope | #291 | ADR-0161 ratified; queue is derived view over proposal log + contemplation runs; pending cap 256; only repo owner ratifies |
|
||||
| Step 1 | #296 | `core teaching hitl-queue list\|show` read-only projection; pure function over proposal log |
|
||||
|
||||
### Total session output
|
||||
|
||||
- **15 PRs merged** (#288, #290–#299, #301–#304)
|
||||
- **1 issue filed** (#300 — `ingest/gate.py` versor_condition margin)
|
||||
- **4 new ADRs** ratified (#0160, #0161, #0162, #0163)
|
||||
- **3 recognizers** ratified into the math admission surface
|
||||
- **First non-zero GSM8K correct count** in project history
|
||||
|
||||
### What stays open
|
||||
|
||||
- **Phase E** (re-baseline harness) — briefed, not dispatched
|
||||
- **Phase B round 2** (3 more shape categories from the uncategorized 14)
|
||||
- **W-029/030/031** (proposal queue / eval center / replay theater)
|
||||
- **ADR-0161 Steps 2–5** (backpressure / submission invariants / workflow extension / contemplation PR body)
|
||||
- **Issue #300** fix
|
||||
|
||||
The substrate is now complete enough to run subsequent rounds end-to-end automatically: Phase B round 2 → Phase C synthesis → operator ratification → measurable lift, with the corridor's wrong=0 invariant enforced by the existing replay gate at each step.
|
||||
|
||||
|
|
|
|||
|
|
@ -346,6 +346,69 @@ proposals; the operator ratifies; Phase E re-baseline confirms `correct
|
|||
|
||||
---
|
||||
|
||||
## Round 1 — what actually shipped (2026-05-26 amendment)
|
||||
|
||||
The corridor closed end-to-end in a single session, faster than the
|
||||
implementation plan above projected. Five PRs landed in order:
|
||||
|
||||
| Phase | PR | Title |
|
||||
|---|---|---|
|
||||
| A | #297 | refusal taxonomy lane — 9 shape categories, 72% of 50-case sample categorized |
|
||||
| B | #298 | exemplar corpora — descriptive_setup_no_quantity / temporal_aggregation / rate_with_currency (20 each) |
|
||||
| C | #301 | recognizer synthesis + admissibility replay gate; three pending proposals in live log |
|
||||
| D | #302 | candidate-graph wiring via single-edit skip-only `continue` |
|
||||
| D-ratify | #304 | operator accepted all three Phase C recognizers |
|
||||
|
||||
### First measured lift on GSM8K train_sample
|
||||
|
||||
```
|
||||
correct: 3 (up from 0 — first non-zero correct count ever)
|
||||
refused: 47 (down from 50)
|
||||
wrong: 0 (unchanged — the invariant holds)
|
||||
|
||||
exit_criterion: { correct_min: 10, wrong_max: 0, passed: false }
|
||||
```
|
||||
|
||||
Lifted cases:
|
||||
|
||||
- `gsm8k-train-sample-v1-0014` — "Bob can shuck 10 oysters in 5 minutes.
|
||||
How many oysters can he shuck in 2 hours?" → 240
|
||||
- `gsm8k-train-sample-v1-0018` — "Xavier plays football. During 15
|
||||
minutes Xavier can score 2 goals on average..." → 16
|
||||
- `gsm8k-train-sample-v1-0042` — "Ella has 4 bags with 20 apples in each
|
||||
bag and six bags with 25 apples in each bag. If Ella sells 200
|
||||
apples, how many apples does she have left?" → 30
|
||||
|
||||
**Capability-axis floor preserved** — G1..G5 + S1 all report `wrong = 0`
|
||||
post-ratification, byte-identical to the pre-Phase-D baseline.
|
||||
|
||||
### Unexpected positive observation
|
||||
|
||||
None of the three lifts are pure `descriptive_setup_no_quantity` cases —
|
||||
they all involve temporal or aggregation framings. Phase D's skip-only
|
||||
wiring is doing more useful work than the projection suggested: when a
|
||||
previously-refusing statement is skipped, the *question* + *remaining
|
||||
statements* together carry enough math for the existing solver to
|
||||
produce the right answer. **A Phase B round 2 (more shape categories
|
||||
from the uncategorized 14) may be a more direct path to clearing Round
|
||||
1 exit (`correct ≥ 10`) than the originally-planned Phase D.2
|
||||
(parsed_anchors solver plumbing).** Worth measuring before scoping
|
||||
Phase D.2.
|
||||
|
||||
### Phase E status
|
||||
|
||||
The Phase E re-baseline harness (versioned baselines under
|
||||
`evals/gsm8k_math/train_sample/v1/baselines/`, workflow_dispatch +
|
||||
nightly schedule, `LiftReport` schema, append-only history) was briefed
|
||||
but not dispatched in this session. The re-baseline above was produced
|
||||
by running `evals.gsm8k_math.train_sample.v1.runner` against
|
||||
`origin/main` post-#304. Phase E will automate this.
|
||||
|
||||
See [SESSION-2026-05-26-corridor-closure.md](../sessions/SESSION-2026-05-26-corridor-closure.md)
|
||||
for the full session ledger.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
This ADR is ratifiable when:
|
||||
|
|
|
|||
|
|
@ -4,6 +4,17 @@
|
|||
**Author intent:** carry the strategic plan + active-PR snapshot across the
|
||||
session boundary so morning-self picks up cleanly.
|
||||
|
||||
> **2026-05-26 amendment** — Phases 1–5 substantially complete. Math
|
||||
> architecture corridor closed end-to-end (Phase A → B → C → D + operator
|
||||
> ratification, ADR-0163); first measurable GSM8K lift produced
|
||||
> (`correct: 0 → 3, wrong: 0 unchanged`). Workbench surface operational
|
||||
> end-to-end (W-026 API + W-027 shell + W-028 chat surface). HITL queue
|
||||
> read-only CLI shipped (ADR-0161 Step 1). See
|
||||
> [SESSION-2026-05-26-corridor-closure.md](sessions/SESSION-2026-05-26-corridor-closure.md)
|
||||
> for the day's full ledger (15 merges + 1 issue). Sections below remain
|
||||
> as the historical 2026-05-24 plan; **next-moves are in the session
|
||||
> recap, not here**.
|
||||
|
||||
---
|
||||
|
||||
## Three guiding rules
|
||||
|
|
|
|||
188
docs/sessions/SESSION-2026-05-26-corridor-closure.md
Normal file
188
docs/sessions/SESSION-2026-05-26-corridor-closure.md
Normal file
|
|
@ -0,0 +1,188 @@
|
|||
# Session 2026-05-26 — Corridor Closure: First Measurable Lift on GSM8K
|
||||
|
||||
**Status:** Complete ✓
|
||||
**Closed:** 2026-05-26 (evening PST)
|
||||
**Headline:** Closed the math architecture corridor end-to-end (Phase A → B → C → D + operator ratification) and produced the first non-zero `correct` count on the GSM8K train_sample baseline. Shipped the Workbench UI shell + chat surface concurrently.
|
||||
|
||||
---
|
||||
|
||||
## TL;DR
|
||||
|
||||
**14 PRs merged + 1 issue filed in a single session.** The math architecture went from `0/50/0` (zero correct, fifty refused, zero wrong) to **`3/47/0` — the first lift the engine has ever produced on GSM8K — with the `wrong = 0` invariant preserved by construction at every step.** The Workbench went from "doctrine only" to "fully operational end-to-end" (read-only API + design substrate + frontend shell + live chat surface with trace drawer).
|
||||
|
||||
The day demonstrated that the **"decodes, not generates" thesis is actionable as a corridor**: operator-authored exemplars → engine-derived recognizers → operator ratification → measurable admissibility lift, with no algebra changes, no pack mutations, no hidden normalization.
|
||||
|
||||
---
|
||||
|
||||
## Merged PRs (in chronological order)
|
||||
|
||||
| # | Title | SHA |
|
||||
|---|---|---|
|
||||
| #288 | chore(contemplation): CI run 2026-05-26T103018Z | `ddb94d3` |
|
||||
| #290 | fix(W-025): polish contemplation-quality eval lane follow-ups | `8829529` |
|
||||
| #291 | docs(L11): ADR-0161 — HITL async queue (proposed) | `0909ef2` |
|
||||
| #292 | feat(W-026): read-only workbench API (ADR-0160 Phase 1) | `8a24ebe` |
|
||||
| #293 | docs(workbench): ADR-0162 — Workbench Design System v1 (proposed) | `8a24256` |
|
||||
| #294 | docs(math): ADR-0163 — path to GSM8K mastery (proposed) | `8b3314f` |
|
||||
| #295 | feat(workbench-ui): design system v1 scaffold (ADR-0162 Branch 1) | (squash) |
|
||||
| #296 | feat(ADR-0161.1): core teaching queue list\|show + fix + R1..R5 | `ec5d6f5` |
|
||||
| #297 | feat(ADR-0163.A): refusal taxonomy lane — shape categorization | `5b4dcb1` |
|
||||
| #298 | feat(ADR-0163.B.1): exemplar corpora — three categories | `1bff568` |
|
||||
| #299 | feat(W-027): Workbench frontend shell — five-region grid | `cdead69` |
|
||||
| #301 | feat(ADR-0163.C): contemplation ingests exemplars + DerivedRecognizer proposals | `08c5e0e` |
|
||||
| #302 | feat(ADR-0163.D): wire ratified RecognizerSpecs into math_candidate_graph | `e9b7eb0` |
|
||||
| #303 | feat(W-028): chat surface + trace drawer (ADR-0160 Phase 3) | `a612038` |
|
||||
| #304 | chore(ratify): accept three Phase C exemplar_corpus recognizers (round 1) | `062d53f` |
|
||||
|
||||
Plus **issue #300** — `ingest/gate.py` versor_condition margin bug, discovered during Tier 1 manual GSM8K iteration.
|
||||
|
||||
---
|
||||
|
||||
## What landed by fork
|
||||
|
||||
### Fork A — Math Architecture (ADR-0163 corridor)
|
||||
|
||||
The corridor closed end-to-end in a single session:
|
||||
|
||||
- **Phase A** (PR #297) — `evals/refusal_taxonomy/` lane. Categorized the 50-case GSM8K train_sample refusal histogram into 9 disjoint shape categories + UNCATEGORIZED. 72% categorized (target was 50%). Top three by count: `descriptive_setup_no_quantity` (17), `temporal_aggregation` (4), `rate_with_currency` (3). Doctrine encoded as runtime guards (no LLM imports test, ≥3 evidence-cite rule per category).
|
||||
- **Phase B** (PR #298) — `teaching/admissibility_exemplars/` carrying 60 hand-authored exemplars across three categories (20 each). Each exemplar binds a statement to an `expected_graph` payload following a per-category schema. Seven `author_note` entries surfaced schema gaps for downstream review.
|
||||
- **Phase C** (PR #301) — `teaching/recognizer_synthesis.py` distils each Phase B corpus into a `RecognizerSpec`. Synthesis is deterministic, narrower-not-broader, rules-only (no LLM / no embeddings). Phase C also extended `teaching/replay.py` with `run_admissibility_replay_gate` that runs cognition + G1..G5+S1 + GSM8K train_sample and auto-rejects any candidate that would lift wrong > 0 even by one. Three pending proposals landed in the proposal log.
|
||||
- **Phase D** (PR #302) — `generate/recognizer_registry.py` (pure projection over the proposal log; only `state="accepted"` + `kind="exemplar_corpus"` entries enter) + `generate/recognizer_match.py` (per-category rules-only matchers that honor the Phase C narrowness rule) + a **single-edit guard** at `generate/math_candidate_graph.py:455-470`. Skip-only by construction — recognized statements contribute zero math state, so the Cartesian product is byte-identical to "this statement was never there," preserving `wrong = 0` at the wiring layer.
|
||||
- **Phase D ratification** (PR #304) — operator accepted all three pending proposals via `core teaching review --accept`. The ratified registry returned `len = 3`.
|
||||
|
||||
### First measured lift — GSM8K train_sample (50 cases, post-ratification)
|
||||
|
||||
```
|
||||
correct: 3 (up from 0 — first non-zero correct count ever)
|
||||
refused: 47 (down from 50)
|
||||
wrong: 0 (unchanged — the invariant holds)
|
||||
|
||||
exit_criterion: { correct_min: 10, wrong_max: 0, passed: false }
|
||||
```
|
||||
|
||||
The three lifted cases:
|
||||
|
||||
| case_id | answer | question excerpt |
|
||||
|---|---|---|
|
||||
| `gsm8k-train-sample-v1-0014` | 240 | Bob can shuck 10 oysters in 5 minutes. How many oysters can he shuck in 2 hours? |
|
||||
| `gsm8k-train-sample-v1-0018` | 16 | Xavier plays football. During 15 minutes Xavier can score 2 goals on average. How many goals on average... |
|
||||
| `gsm8k-train-sample-v1-0042` | 30 | Ella has 4 bags with 20 apples in each bag and six bags with 25 apples in each bag. If Ella sells 200 apples, how many apples does she have left? |
|
||||
|
||||
**Capability-axis floor preserved**: G1..G5 + S1 all report `wrong = 0` post-ratification, byte-identical to the pre-Phase-D baseline.
|
||||
|
||||
**Unexpected positive observation**: none of the three lifts are "pure descriptive_setup_no_quantity" cases. They all involve temporal or aggregation framings where the previously-refusing statement gets skipped and the *question* + *remaining statements* carry enough math for the existing solver to produce the right answer. This suggests Phase D's skip-only wiring is doing more useful work than the projection suggested, and that a **Phase B round 2** (more shape categories from the uncategorized 14) may be a more direct path to clearing Round 1 exit (`correct ≥ 10`) than the originally-planned Phase D.2 (parsed_anchors solver plumbing).
|
||||
|
||||
### Fork B — Workbench UI (ADR-0160 + ADR-0162 + W-026..W-028)
|
||||
|
||||
The Workbench is fully operational end-to-end:
|
||||
|
||||
```bash
|
||||
uv run core workbench api # http://127.0.0.1:8765
|
||||
cd workbench-ui && pnpm dev # http://127.0.0.1:5173
|
||||
```
|
||||
|
||||
What ships:
|
||||
|
||||
- **W-026 (PR #292)** — read-only HTTP API (`workbench/api.py` + `workbench/readers.py` + `workbench/schemas.py` + `workbench/server.py`). Seven GET endpoints + one whitelisted POST (`/evals/run` for the `contemplation_quality` lane only). Path-traversal protection, MAX_ARTIFACT_BYTES guard, single-operator concurrency lock on eval runs, request logging gated by `CORE_WORKBENCH_QUIET` env var.
|
||||
- **ADR-0162 (PR #293)** — design system doctrine: semantic tokens, dark-only theme, motion rules ("reveals structure, not cognition"), `StableJsonViewer` six invariants, empty/error/loading state contracts, keyboard-first contract, five-region shell, v1 component map, explicit no-go list.
|
||||
- **Branch 1 (PR #295)** — `workbench-ui/` scaffold with self-hosted Inter + JetBrains Mono, generated `tokens.ts` mirror from `tokens.css`, `StableJsonViewer` with all six invariants tested, enum-bound badge primitives (`EpistemicStateBadge`, `NormativeClearanceBadge`, `ReviewStateBadge`, `GroundingSourceBadge`), and a `/preview` route as the design baseline. Build-time enum-coverage test parses `core/epistemic_state.py` + `teaching/proposals.py` via Python AST and asserts 1:1 badge coverage; adding an enum value on the Python side without updating the badge fails the test loud.
|
||||
- **W-027 (PR #299)** — five-region grid shell (TopBar / LeftNav / MainSurface / RightInspector / StatusFooter), ten empty routes each with copy-CLI affordances, live `StatusFooter` consuming `useRuntimeStatus()` (color + label encoding of `mutation_mode`, click-to-copy `git_revision`, amber-on-warning `checkpoint_revision` with ADR-0157/0158 inline note), `CommandPalette` upgrade with three real commands + fuzzy search, `ApiErrorBoundary` for `WorkbenchApiError`, schema-drift sentinel via `scripts/dump-api-schemas.py`.
|
||||
- **W-028 (PR #303)** — the first route with real content. `/chat` route with `PromptComposer` (⌘/Ctrl+Enter submit, Esc clear, 4096-char cap), `ResponseCard`, `EvidenceStrip` (5 enum-bound badges visible at a glance — grounding source, epistemic state, normative clearance, refusal indicator, mutation/checkpoint), `TraceDrawer` with three layers (structured panels, `StableJsonViewer`, raw payload behind `<details>` with downloadable .json). Server-side `POST /chat/turn` with module-level `threading.Lock`, 64 KiB body cap, returns full `ChatTurnResult` carrying surface/walk_surface/articulation_surface distinctly per `docs/runtime_contracts.md`.
|
||||
|
||||
### Fork C — HITL Queue (ADR-0161 + L10b followups)
|
||||
|
||||
- **ADR-0161 (PR #291)** — HITL async queue scope ratified. Five sub-questions answered with the narrowest commitment compatible with existing ADR-0057 / 0151 / 0152 / 0155 machinery: queue is a derived view over the proposal log + contemplation runs, no new persistence file, three operator surfaces (PR / workflow_dispatch / CLI), pending cap 256, no wall-clock expiry, only repo owner ratifies.
|
||||
- **Step 1 (PR #296)** — `core teaching hitl-queue list|show` read-only projection (renamed from `core teaching queue list|show` after I caught a namespace collision with the existing Phase 1.2 gap-promotion command). `derive_queue()` is a pure function over the proposal log; `find_queue_item()` supports exact + prefix matching; CLI returns formatted human view or JSON.
|
||||
|
||||
---
|
||||
|
||||
## Things discovered along the way
|
||||
|
||||
### Tier 1 manual iteration falsified an assumption
|
||||
|
||||
Earlier in the session I described manual teaching-chain iteration as a path to lift GSM8K cases. Empirical probing showed otherwise: `core teaching propose` writes to the cognition corpus, but the **math admission path is independent** — `generate/math_candidate_graph.py` refuses on its own admission surface, never consulting the teaching corpus. Teaching chains don't lift math cases; only candidate_graph admissibility expansion does (which is ADR-0163's whole point). The manual iteration was the right exercise even though the path it suggested was wrong: it falsified the assumption fast and refocused the session on the corridor.
|
||||
|
||||
### `ingest/gate.py` versor_condition margin bug (issue #300)
|
||||
|
||||
During Tier 1 probing of `Tom has 5 apples. He gives 2 to Sarah. How many does Tom have?`, the engine raised a `RuntimeError: Injection produced non-versor field: condition=2.12e-06` from `ingest/gate.py:346`. Bisected the trigger: requires (a) a declarative statement with a quantity, (b) a transfer phrase with `to <recipient>`, and (c) a "How many" question. Both observed condition values (`1.02e-06`, `2.12e-06`) cluster within 3× of the 1e-6 threshold — `normalize_to_versor()` is *almost* closing and missing by a small margin on this token mix. Per CLAUDE.md, the threshold doesn't move; the construction boundary does. Filed as a separate bug, not in scope for the corridor work.
|
||||
|
||||
### Phase D's skip-only wiring did more than projected
|
||||
|
||||
The Phase D PR projected 3 cases lifting under skip-only — primarily through `descriptive_setup_no_quantity` admissions where the remaining math is solvable by existing G1..G5+S1 admission. Empirically all three lifts were in cases involving rate/temporal framings: the recognizer skipped the previously-refusing statement, the question and remaining statements together carried enough math for the solver to produce the correct answer. This is a forward note for the math fork's next-move planning — Phase D.2 (parsed_anchors solver plumbing) may not be the highest-leverage next step.
|
||||
|
||||
### Code review discipline transferred across agents
|
||||
|
||||
Every PR opened today got a mastery-level review. The review pattern that emerged:
|
||||
|
||||
1. Verify locally (run tests, build, smoke check)
|
||||
2. List 5–10 items at mastery level (the pattern to keep for subsequent branches)
|
||||
3. List 0–5 non-blocking forward notes (the pattern to track but not block on)
|
||||
4. Verdict (approve / blocker / forward-action)
|
||||
5. Cross-references checked
|
||||
|
||||
This pattern caught real regressions (Gemini's gap-queue namespace collision in #296) and real architectural framings (Codex's "skip-only by construction" naming in #302). When the agent addressed feedback in a follow-up commit, the same pattern verified the fix. The pattern is now durable; future sessions can use the same shape.
|
||||
|
||||
---
|
||||
|
||||
## Operator-side actions taken
|
||||
|
||||
| When | Action |
|
||||
|---|---|
|
||||
| Mid-session | Removed the `required_pull_request_reviews` policy on `main` (kept the `verify pinned lane SHAs` status check). Authorized in-session by the operator. All later merges flowed through this looser gate. |
|
||||
| Late session | Ran `core teaching propose-from-exemplars` against each of the three Phase B corpora (via Opus 4.7's first commit on PR #302) to land the three pending proposals in the live log. |
|
||||
| Late session | Operator ratified all three Phase C recognizers via `core teaching review --accept`. PR #304 captured the resulting log transitions. |
|
||||
| Late session | Filed issue #300 — `ingest/gate.py` versor_condition margin bug. |
|
||||
|
||||
---
|
||||
|
||||
## What's NOT done (deferred to future sessions)
|
||||
|
||||
- **Phase E** as briefed — the GSM8K train_sample re-baseline harness (versioned baselines under `evals/gsm8k_math/train_sample/v1/baselines/`, workflow_dispatch + nightly schedule, `LiftReport` schema, append-only history). Brief ready to paste; not yet dispatched.
|
||||
- **Phase D.2 / parsed_anchors solver plumbing** — would convert `rate_with_currency` and `temporal_aggregation` recognitions into solver state. Scope-uncertain after the empirical observation that skip-only already produces some rate/temporal lifts; worth measuring more carefully before scoping.
|
||||
- **Phase B round 2** — three more shape categories from the Phase A uncategorized tail (14 statements). Likely candidates: comparative_with_unit, fractional_rate_of_change, indefinite_quantity, nested_question_target. Operator picks.
|
||||
- **W-029 proposal queue UI** — make the HITL queue visible in the Workbench (the three Phase D proposals are in the live log; W-029 surfaces them in the UI).
|
||||
- **W-030 eval center UI** — failures-first layout per the vision doc.
|
||||
- **W-031 replay theater UI** — original vs replay side-by-side, trace_hash comparison.
|
||||
- **ADR-0161 Steps 2–5** — backpressure (pending-count cap + queue_full reports), submission invariants (duplicate, dependent_on_pending), workflow_dispatch extension to reject/withdraw, embed queue summary in contemplation PR body.
|
||||
- **Issue #300** — `ingest/gate.py` versor_condition margin fix.
|
||||
|
||||
---
|
||||
|
||||
## Architectural state at session close
|
||||
|
||||
The **substrate is complete enough that the next round can run end-to-end automatically**:
|
||||
|
||||
```
|
||||
Phase B round 2 → 3 new exemplar corpora authored
|
||||
↓
|
||||
Phase C → contemplation runner ingests; emits 3 new RecognizerSpec proposals
|
||||
↓
|
||||
(replay gate auto-rejects anything that would lift wrong > 0)
|
||||
↓
|
||||
Phase D → proposals appear in the HITL queue; operator inspects + ratifies
|
||||
↓
|
||||
(skip-only wiring + ratified registry both already on main)
|
||||
↓
|
||||
next GSM8K eval shows next round of lift, wrong=0 invariant intact
|
||||
```
|
||||
|
||||
That loop, once you decide to run it, is the math architecture working as designed. Today we proved the loop closes. Future rounds widen.
|
||||
|
||||
---
|
||||
|
||||
## Cross-references
|
||||
|
||||
- [ADR-0163 — Path to GSM8K mastery](../decisions/ADR-0163-gsm8k-path-to-mastery.md)
|
||||
- [ADR-0162 — Workbench Design System v1](../decisions/ADR-0162-workbench-design-system.md)
|
||||
- [ADR-0161 — HITL async queue](../decisions/ADR-0161-hitl-async-queue.md)
|
||||
- [ADR-0160 — CORE Workbench v1](../decisions/ADR-0160-core-workbench-v1.md)
|
||||
- [Master plan post-substrate-audit](../master-plan-post-substrate-audit.md) — superseded by this session for the math + workbench forks
|
||||
- [docs/PROGRESS.md](../PROGRESS.md) — capability-roadmap tracker; this session adds the math-corridor closure entry
|
||||
|
||||
### Memory cross-references
|
||||
|
||||
- `thesis-decoding-not-generating` — the load-bearing thesis that this session demonstrated as actionable
|
||||
- `feedback-address-critiques-dont-waive` — applied consistently across mastery-level reviews
|
||||
- `feedback-cleanup-as-you-find` — stale worktrees and merged branches cleaned at session close
|
||||
- `feedback-adr-cross-reference-discipline` — every new ADR cross-references existing substrate
|
||||
- `feedback-no-timelines` — phases sequenced by prerequisite, not date
|
||||
Loading…
Reference in a new issue