# ADR-0114a.6 — Compositional-Depth Curve Auditor (Obligation #6 wired for B3)

**Status:** Accepted (mechanism); coverage gap deferred to B3-owner follow-up
**Date:** 2026-05-23
**Author:** CORE main agent (Opus 4.7)
**Depends on:** ADR-0114a (10 anti-overfitting obligations),
ADR-0119.6 (GSM8K depth-curve substrate — pattern source),
ADR-0131.3 (B3 bounded grammar lane),
ADR-0114a.10 (PR #189 — first obligation auditor; pattern this PR mirrors)
**Parent:** ADR-0114a
**Sequencing:** second of 5 remaining ADR-0114a obligations for
`mathematics_logic`. After this: #2 OOD ratio, #5 perturbation
(in flight, L14 / Opus#2), #8 adversarial.

---

## Context

ADR-0114a Obligation #6:

> `depth_curve.py` produces a per-bucket curve;
> `accuracy(N) ≥ accuracy(depth_1) · (1 − ε)^(N − 1)` for ε = 0.05.

A pattern-matcher's accuracy decays sharply with reasoning depth;
a deterministic reasoning system stays approximately flat. The
obligation pins that decay rate at most 5% per depth-step from the
depth-1 anchor.

ADR-0119.6 shipped this measurement harness for GSM8K (lives at
`evals/gsm8k_math/scoring/depth_curve.py`). This PR ports the
pattern to the math composite gate's B3 lane.

## Decision

`core/capability/depth_curve.py` — pure auditor. For each B3
case marked `expected: solved_correct`:

1. Re-run the candidate-graph pipeline.
2. Bucket by **`len(trace.steps)`** (authoritative depth = the
   number of operations the solver actually executed).
3. Aggregate per-bucket accuracy.
4. Check each populated bucket (≥1 case) against the decay bound
   anchored on depth_1's accuracy.

### Bucket schema (closed, mirrors ADR-0119.6)

`depth_1`, `depth_2-3`, `depth_4-5`, `depth_6-8`. Depth > 8
raises `DepthCurveError` rather than silently extending the
schema. Depth == 0 (initial-only problems) is counted as
"skipped" — zero-depth has no reasoning to decay.

### Representative depth convention

`representative_depth(bucket) = min(bucket)` — most permissive,
gives the bound the best chance of holding even when only the
shallow end of the bucket is populated. Tightening (e.g.,
max-depth-in-bucket) requires an ADR amendment.

### Verdict shape — two orthogonal facts

The report distinguishes **mechanism wiring** from **assertion
meaningfulness**:

| Field | Meaning |
|---|---|
| `obligation_6_mechanism_wired` | The auditor exists and ran end-to-end. |
| `obligation_6_assertion_holds` | Every populated bucket satisfies the decay bound. (Vacuously true when only depth_1 is populated.) |
| `coverage_sufficient` | ≥2 buckets populated AND ≥3 cases per populated bucket. Required for the assertion to be statistically meaningful. |

CLI exits 0 iff `obligation_6_assertion_holds`. `coverage_sufficient`
flows as honest disclosure in the `refusal_reason` when the
assertion holds only vacuously.

## Empirical verdict on current main

```
$ python3 -m core.cli capability depth-curve

lane:                          B3_bounded_grammar
cases_total:                   50
cases_solved:                  22
epsilon:                       0.05
mechanism_wired:               True
assertion_holds:               True
coverage_sufficient:           False
populated_buckets:             ['depth_1', 'depth_2-3']

  bucket       total   correct  accuracy   bound      satisfied
  depth_1      21      21       1.0000     (anchor)   True
  depth_2-3    1       1        1.0000     0.9500     True
  depth_4-5    0       0        0.0000     (anchor)   True
  depth_6-8    0       0        0.0000     (anchor)   True

refusal_reason: assertion holds but coverage insufficient — populated
buckets: ['depth_1', 'depth_2-3'] (need ≥2 buckets, ≥3 cases each)
```

**Both populated buckets satisfy the decay bound** (depth_1 anchors
at 100%; depth_2-3 at 100% > 0.95 bound). The mechanism is wired
correctly. But **coverage is insufficient** — depth_2-3 has only 1
case, depth_4-5 and depth_6-8 have none.

## Honest scope-limit + named follow-up

For obligation #6 to *meaningfully* gate the full ADR-0120 expert
promotion, B3 v1's case set needs more depth coverage:

- ≥3 cases at depth_2-3 (currently 1)
- ≥3 cases at depth_4-5 (currently 0)
- Ideally ≥3 cases at depth_6-8 (currently 0)

This is **B3-owner work**, not auditor work. The case authoring is
governed by ADR-0131.3's grammar contract; depth-4+ problems are
admissible by B3's grammar (sequence of additive/subtractive ops
with consistent units) but the v1 case set leaned heavily on
single-op problems for the lane-gate's `correct_rate ≥ 0.95` target.

**Reserved follow-up**: a small B3 v1.1 PR that authors additional
multi-step cases under B3's existing grammar. After it lands,
`coverage_sufficient` flips to True and obligation #6 becomes
materially gating.

## What this does NOT do

- Does NOT change the solver, the parser, or any B-lane runner.
- Does NOT modify B3's case set. (The case authoring is reserved
  follow-up, scoped to B3's owner.)
- Does NOT promote `mathematics_logic` to `expert`.
- Does NOT wire B1 or B2 equivalents:
  - **B1** (symbolic equivalence): algebra-not-arithmetic, no
    solver trace, no depth notion. Different metric needed.
  - **B2** (teaching corpus): chain-validation surface, not
    problem-solving. Different metric needed.
  - Both deferred to separate sub-ADRs.

## Trust boundary

- **Reads only**:
  - `evals/math_bounded_grammar/v1/cases.jsonl`
  - The pipeline's own pack reads (transitively via
    `parse_and_solve` → `solve`)
- **Writes only**: artifact path (default
  `evals/obligation_6_depth_curve/<lane_id>.json`)
- No dynamic imports, no shell passthrough, no network.
- Pure deterministic function — verified by
  `test_report_is_deterministic` and
  `test_artifact_emission_byte_equal`.

## Tests

`tests/test_adr_0114a_6_depth_curve.py` — 24 tests:

| Group | Count | What it pins |
|---|---|---|
| bucket schema + classification | 12 | closed set; depth 1..8 maps correctly; depth ≥ 9 raises |
| decay bound math | 5 | ε = 0.05 pinned; bound = anchor · 0.95^(N-1); representative_depth = min(bucket); depth_1 has no bound |
| coverage policy | 1 | thresholds pinned (≥2 buckets, ≥3 cases each) |
| lane evaluation | 3 | passes on real B3; populates depth_1; refuses on missing cases |
| coverage-sufficient distinction | 2 | B3 today is coverage-insufficient with assertion-holds; synthetic 5+5 fixture is both |
| determinism | 2 | report identical across calls; artifact byte-equal |

All pass in 0.28s.

## Composition with ADR-0131.4 and obligation #10

Orthogonal. The composite math gate (ADR-0131.4 / PR #188 / merged)
gates the *benchmark* portion of the contract. Obligation #10
(PR #189) audits *pack provenance*. Obligation #6 (this PR) audits
*reasoning-depth decay*. Each is its own evaluator; a future full
ADR-0120 wire-up composes them into a single signed `expert_claims`
artifact for ledger promotion.

## CLAUDE.md PR-checklist

- **Capability added:** depth-curve auditor with honest two-axis
  verdict (mechanism wired + assertion holds vs. coverage
  sufficient) for B3.
- **Invariant proving field validity:** decay bound holds on every
  populated bucket; ε = 0.05 pinned; representative_depth
  convention pinned.
- **CLI/eval proving the lane:** `python3 -m core.cli capability
  depth-curve` + `pytest tests/test_adr_0114a_6_depth_curve.py`.
- **Avoided hidden normalization / stochastic / approximate /
  unreviewed mutation:** Yes. Pure deterministic auditor.
- **Trust boundary:** read-only inputs from documented paths;
  single deterministic write; no dynamic imports.