core/docs/sessions/2026-05-28-risk-reward-learning-architecture.md

# Session 2026-05-28 — From "Another Matcher" to a Self-Calibrating Problem Solver

**Status:** Discussion captured ✓ — ADR-0175 to follow
**Headline:** A scoping session for GSM8K Phase 5b turned into the discovery that the
whole "build another recognizer per problem shape" strategy is structurally
self-limiting (overfitting by construction), and converged on a different
architecture: **move the intelligence into the solver (attempt + eliminate),
separate a sealed practice regime from the wrong=0 serving regime, and gate
attempts with a deterministic risk/reward ratio grounded in earned per-class
calibration.** This document is the *journey* — the problem, the dead-ends, the
pivots, and how we arrived at the solution. The formal decision is **ADR-0175**.

---

## TL;DR

We started trying to scope "Phase 5b.1 — single-sentence multiplicative
aggregate." Measurement showed the target was **one idiosyncratic case (0021)**,
and that the broader per-shape-matcher approach can never compound: GSM8K
phrasings are unbounded, so each matcher buys a handful of cases and the curve
flattens. That is overfitting wearing an engineering hat.

The turn (driven by Shay): stop asking the *front-end* to recognize every shape;
give the *solver* room to **attempt** a derivation over extracted quantities and
**learn by elimination** from what's wrong. The solver already supports the
operations; the front-end was the bottleneck and the source of brittleness.

That exposed the real contradiction: a system that **refuses whenever uncertain
is safe but frozen** — it can only ever learn what a human hands it. Autonomous
learning *requires* attempting through uncertainty, which is exactly where being
wrong lives. You cannot have autonomous learning and a global "never be wrong" at
once.

Resolution: **two regimes** — *serving* (wrong=0, refuse-unless-certain,
unchanged) and *sealed practice* (attempt-and-eliminate, where wrong is the
learning signal, not a failure) — separated by a **proof-carrying seal**. Which
regime applies is decided per-attempt by a **deterministic risk/reward ratio**:
`measured_reliability(class) / required_reliability(action) ≥ 1`. Reliability is
an **earned, per-class, conservatively-bounded count** from a live gold tether —
not a learned weight, not a probability from nowhere. Checkability is a **tiered
ladder** (gold > convergent self-verification > consistency-only) where the
*privilege a passed check earns* scales with the check's strength, and everything
weak is reversible. Humans raise the per-class ceiling on evidence; the engine
never self-authorizes.

---

## Where we started: scoping Phase 5b.1

Phase 5a had just shipped (PR #430): the inert GSM8K reader retired, single
canonical parse path, `3/47/0` byte-identical, net −1,038 LOC. We turned to
Phase 5b — the *semantic* lift — and began with what the scope called the
cleanest first slice: "single-sentence multiplicative aggregate" (e.g. case 0021,
"He bench presses 15 pounds for 10 reps and does 3 sets" → 15×10×3 = 450).

## Dead-end #1 — the per-shape matcher reflex

Every instinct pulled toward *building another recognizer*: detect this shape,
emit that primitive. Two measurements stopped that cold.

**The brute-force measurement that lied.** A bounded expression search over each
refused case's numbers found "matches" — but several were *coincidences*
(`20/5 = 4` for a case whose answer was 4 for unrelated reasons; `5−3 = 2`). The
measurement itself overfit. Lesson banked: **number-matching without grounding
manufactures spurious answers** — the same hazard a naive solver search would
have.

**The ground-truth measurement.** Parsing GSM8K's own `<<a*b=c>>` calculator
annotations (real operations, not guesses) over the 47 refused cases:

| Profile | Count | Share |
|---|---|---|
| Uses multiplication (`*`) | **37/47** | 79% |
| Uses mul **or** div | 43/47 | 91% |
| Pure add/subtract | 4/47 | 9% |
| **Single-step solvable** | **0/47** | **0%** |

Step-count histogram: `2:13 · 3:12 · 4:8 · 5:8 · 6:3 · 7:3`.

And a scan for *single-sentence* multiplicative aggregates (all operands in one
clause, gold = their product) returned exactly **one** case — 0021 — whose
surface ("for 10 reps and does 3 sets", pronoun subject, three different units)
is the *least* general multiplicative shape in the set, not the simplest.

**The conclusion that reframed everything:** multiplication is needed by 79% of
the corpus (maximally general — a foundational capability, not a niche), but
**no case is solvable in one step**, and the regular in-clause shape *already
works* via existing Wave-A scaffolding. So a "single-sentence multiplicative
matcher" would flip ~1 case via near-bespoke pattern matching — the textbook
definition of overfitting the project has repeatedly ruled out ("lifting a few at
a time → simply overfitting"). The per-shape path is unbounded because phrasings
are unbounded. It cannot compound.

## Pivot #1 (Shay) — breadth-of-impact is the overfitting test

> "If solutions result in generally large chunks of failed results turning green,
> then it's more likely that we are getting GENERALLY smarter... if tweaking the
> injector fixes a big chunk or percentage then it COULD be a solution."

This corrected a sloppy framing ("tweaking the injector = overfitting"). The
overfitting test isn't *where* the fix lives — it's **breadth**: a change that
flips a large *general* chunk (and holds under perturbation) is a real capability;
a per-phrasing patch that flips one case is overfitting. The measurement said the
biggest, most general chunk is *multiplication itself* — so the question became
"how do we get general multiplicative capability," not "how do we match 0021."

## Pivot #2 (Shay) — give the solver room to attempt; move the intelligence

> "Maybe the solution is to give it more room to attempt to solve the problem...
> the better we make the problem solver, the better the model will be at
> anything/everything... the less we will have to put into packs and the less it
> will have to contemplate over time, and the faster it will learn what works and
> it compounds. That's what intelligence does."

This is the architectural turn. Today the front-end (recognizer/injector) must
hand the solver a *fully-typed, correct* graph or the solver does nothing — so
all intelligence is forced into surface pattern-matching, and surface is infinite.
The capable part sits idle:

- The **solver already supports** `{add, subtract, transfer, multiply, divide,
  apply_rate, compare_additive, compare_multiplicative}` with pack lemmas for
  each. It just waits passively.
- The gap is entirely the **reader → injector → Operation** front-end (the
  recognizer matches a shape but extracts *zero* anchors on real sentences).

The redirection: shrink the front-end to **extract quantities + the relations the
text licenses**, and grow the solver to **attempt** — search the bounded space of
operation-chains for a derivation that reaches a *verifiable* answer. This
generalizes across all phrasings because it never looks at phrasing — it looks at
quantities and the goal. The compounding the project wants comes from the solver,
not from a growing library of founds (the [[thesis-decoding-not-generating]]).

**The honest crux we named:** search makes wrong=0 *harder*, not easier — an
active searcher has more chances to land on a spurious-but-valid answer (cf. the
brute-force `20/5`). The whole approach rests on the derivation being **grounded**
(each step licensed by text + unit-consistent), **unique-or-refuse** (the existing
disagreement rule), and **round-trip-checked** (realize back to language, reject
if it doesn't match the problem). Those gates already exist in CORE.

## Insight #3 — two failure modes, and the diagnosis that routes them

> "Either it has enough information... to work the problem with masterfully built
> problem-solving skill, or it needs a wider understanding of what it's trying to
> decode... it needs more pieces to the puzzle."

When the engine can't decode a problem, exactly one of three things is true:

1. **Skill gap** — has the pieces, can't compose the derivation → improve the
   reasoner.
2. **Knowledge gap** — missing a world-fact (what "twice" means, that "each basket
   holds 50" is a per-container rate) → acquire knowledge.
3. **Genuine ambiguity** — under-determined → refuse, correctly, forever.

The mechanism that makes "finding the pieces" efficient is **diagnostic refusal**:
a refusal that *names the missing piece* routes itself to the right axis
("quantities extracted but no grounded derivation" = skill; "unknown relation:
twice" = knowledge; "two grounded derivations disagree" = ambiguity). CORE already
has typed refusals, the OOV honesty gradient, and the math-reader-refusal audit
corridor. Knowing what you don't know is most of knowing how to find it.

**Three compounding stores**, each fed by the matching diagnosis:
- **experience → vault** (exact, deterministic recall),
- **world-knowledge → ratified packs**,
- **skill → solver's elimination-learned pruning**.

**The floor (Shay):** "it cannot conjure world-facts from nothing. not even a
human can. only God himself." Autonomy doesn't mean "no input" — facts about the
world must enter from a data/experience stream. The human role *shifts* from
hand-authoring meaning to **curating what it ingests + ratifying what it has
already self-proven**. The bridge is **self-proving acquisition**: a new fact is
admitted with a *mechanical proof attached* (round-trips, forced-unique, zero
wrong, replay-stable) — the schema-proof-obligation discipline, pointed at
learning. Knowledge acquired with a proof needs less judgment to admit.

## Pivot #4 (Shay) — the contradiction, and the "mode"

> "If we intentionally build around it always 'giving up' whenever it doesn't
> know, it will never learn without humans. Maybe there should be a 'mode' for
> when it's allowed to work on a problem... a type of risk-to-reward RATIO system
> (not... reinforcement learning; not that kind of risk/reward)."

This is the central contradiction stated exactly: **refuse-when-uncertain is safe
and frozen.** The resolution can't be a global setting — it must be contextual.

**Two regimes (the concrete "mode"):**
- **Serving** — committed to someone who will act on it; cost of error ≈ ∞;
  refuse unless certain; **wrong=0, absolute, unchanged.**
- **Practice** — attempting where being wrong is *checkable and not served*; here
  **wrong is the elimination signal**, the most informative event possible. The
  only place autonomous learning can happen.

What keeps wrong=0 intact while practice runs hot: the **proof-carrying seal** —
practice emits *proposals carrying their own proof*; nothing reaches serving
except via the (narrowing) ratification gate. CORE already has this seal
(proposal-only, reviewed teaching).

**"Creative," defined for a deterministic engine:** not stochastic invention — a
**willingness to leap a gap in known structure** (a recombination no stored
pattern directly licenses). The risk is the leap is unwarranted; the reward is
that a *verified* leap becomes new structure. It is always a step from solid
ground, never from the void — the engineering form of "only God conjures from
nothing." The risk/reward gate decides *how far across an unlit gap calibration
permits a step.*

## The checkability question — and the tiered answer

If practice can attempt without a human or a gold label, *what counts as checkable
enough to learn from?* The asymmetry that governs the answer: **a false positive
in the checker is far worse than a missed learning opportunity** — a wrongly
"verified" belief is a persistent contaminant; a refusal is just a missed chance.

So the boundary isn't a line — it's a **confidence-stratified ladder**, with the
rule: *require check-strength proportional to the reversibility/blast-radius of
the action it licenses.*

| Tier | Checker | May change |
|---|---|---|
| **1 — External truth** | gold label / known answer | serving-bound knowledge (via ratification) — anchors |
| **2 — Convergent self-verification** | round-trip **∧** ≥2 *structurally-distinct* derivations agree **∧** unit-consistent **∧** no contradiction with vault/packs | provisional, **retractable** knowledge (still ratified) |
| **3 — Consistency-only** | merely no contradiction | **practice-internal pruning only** — never crosses the seal |

**Tier 2 is the median** — the widest checkability still strong enough to *create*
knowledge, needing no label, so the arena scales toward open-world. Two
safeguards make widening safe:
- **Provenance per belief** `(tier, n_at_admission)` → Tier-2 beliefs are
  *retractable* when a stronger check later contradicts them.
- **A live gold tether** measuring `t2_precision(class)` — catches *correlated
  self-delusion* (a shared wrong premise makes all derivations agree); when it
  drifts, appetite contracts. Independence is *counted* (≥2 structurally-distinct
  paths), not assumed.

## Quantifying it — the part that makes it engineering

The naive ratio needs units for "value" and "learning" — a quagmire. The regime
structure **collapses the reward side**: in serving only reliability matters
(learning is irrelevant to a served answer); in sealed practice the threshold is
just "is it checkable?". So **the entire numeric system reduces to: measure
reliability, compare to a ceiling.** Everything else is counting.

Per **class** (= capability axis G1–G5), a replayable **ledger of counts** —
nothing learned, nothing stochastic:

- `n(C), correct(C), wrong(C), refused(C)` — the eval already counts these.
- `t2_verified(C), t2_agrees_gold(C)` — on the live anchor set.
- `reliability(C) = conservative_floor(correct, n)` — a deterministic **lower
  bound** (pessimistic at small n, so luck can't grant appetite).
- `t2_precision(C) = conservative_floor(t2_agrees_gold, t2_verified)` — how
  trustworthy self-verification is on C; the number that licenses widening past
  gold.

**The gate is a literal ratio:**
```
license(action, C) := reliability_of_relevant_checker(C) / θ_required(action, C) ≥ 1
```
`θ` are **human-set, version-controlled constants** per class and blast-radius
(`θ_practice = 0`; `θ_propose`; `θ_serve` strict, e.g. .99). "Humans ready to let
it" = a human lowering `θ_serve(C)` for a class the ledger has earned.

**Compounding becomes a measurable curve:** `refused(C)` ↓ and `reliability(C)` ↑
with practice volume, `wrong` on serving pinned at 0 throughout. And the
**skill-vs-knowledge diagnosis is quantified**: if reliability still climbs with
practice → skill gap (keep practicing); if it has stalled → knowledge gap (needs a
new world-fact). Learning-rate = reliability gain per unit practice.

Almost none of this is new machinery — it's composition: classes = capability
axes; counts = the eval harness; reliability + lower bound = the calibration
module; replay guarantees reproducibility; `θ` = a small config table; seal +
ratification = existing teaching-safety.

## The converged model (one line)

> attempt-or-refuse is a **per-context risk/reward decision**, not a global rule →
> **two regimes** (serving wrong=0 / sealed practice attempt-and-eliminate)
> separated by a **proof-carrying seal** → the ratio is
> `measured_reliability / required_reliability ≥ 1` → reliability is an **earned,
> conservatively-bounded, per-class count** from a live gold tether → checkability
> is a **tiered ladder** with privilege ∝ reversibility and provenance making weak
> beliefs retractable → humans **raise the ceiling on evidence**; the engine never
> self-authorizes → "creative" = a calibrated leap across a gap, always from given
> ground.

## Open question carried into the ADR

The one genuinely new number to pin: **the shape of the `conservative_floor`
function and `N_min`** — how pessimistic to be at small n. That single choice sets
how cautiously the whole system earns its autonomy.

## What this supersedes

The matcher-oriented Phase 5b sub-phases (5b.1 single-sentence / 5b.2
cross-sentence / 5b.3 deep) in ADR-0174 collapse into *instances* of this
architecture rather than standalone work. The multiplicative cases become the
first **practice arena** where attempt-and-eliminate is proven, not a set of
shapes to hand-match.

## Cross-references

- **Decision:** ADR-0175 (to be written).
- **Builds on:** ADR-0174 (held-hypothesis / `eliminate_violating` / `reevaluate`
  / `contemplate` — the elimination substrate, here repointed from *reading* to
  *solving*); the calibration module; capability axes G1–G5; the round-trip
  filter and multi-branch disagreement rule (the wrong=0 gates that make search
  safe); teaching-safety (proposal-only, reviewed) = the seal.
- **Thesis anchor:** [[thesis-decoding-not-generating]] — find/comprehend/
  rationalize, not a library of founds.
- **Phase 5a (shipped):** PR #430.