core/docs/decisions/ADR-0109-lane-shape-aware-thresholds.md
Shay 36053317be
feat(capability): implement ADR-0109 lane-shape-aware thresholds (#116)
Replaces the cognition-shape-uniform threshold dispatch in
core/capability/expert_demo.py with an explicit LANE_SHAPE_REGISTRY
mapping 8 ratified lane ids to 5 shapes:

  cognition           -> cognition_shape
  elementary_math_ood -> accuracy_shape
  foundational_physics_ood -> accuracy_shape
  symbolic_logic      -> symbolic_logic_shape
  hebrew_fluency      -> accuracy_shape
  koine_greek_fluency -> accuracy_shape
  inference_closure   -> inference_shape
  fabrication_control -> refusal_shape

Each shape has a documented threshold checker. Unknown lane ids
fail-closed with a named reason. ADR-0106 \xc2\xa71.1/\xc2\xa71.3/\xc2\xa71.4/\xc2\xa71.5
unchanged; only \xc2\xa71.2 (threshold rules) dispatches by shape.

tests/test_lane_shape_thresholds.py pins all four ADR-0109 invariants
plus dead-shape and threshold-value gates (13 new tests).
tests/test_expert_demo_contract.py fixtures updated to provide
shape-appropriate metrics (no semantic change to those tests; same
12 cases still pin the ADR-0106 contract).

ADR-0109 status: Proposed -> Accepted. README sequencing updated
(ADR-0110 now only blocked by inference_closure, not by metric-shape
amendment).

Ledger: all five domains remain reasoning-capable, expert_demo=false.
2026-05-22 12:11:58 -07:00

220 lines
7.4 KiB
Markdown

# ADR-0109 — Lane-Shape-Aware Thresholds (ADR-0106 Amendment)
**Status:** Accepted
**Date:** 2026-05-22
**Author:** CORE agents + reviewers
**Amends:** ADR-0106
**Depends on:** ADR-0106, ADR-0107
**Reserves:** ADR-0110 (math expert-demo re-attempt under this amendment)
---
## Context
ADR-0106 §1.2 prescribed a single set of threshold metrics — taken from
the cognition pack's eval shape — applied uniformly across every lane
attached to a ratified domain:
```text
surface_groundedness >= 0.95
term_capture_rate >= 0.85
intent_accuracy >= 0.95
versor_closure_rate == 1.0
```
ADR-0107 surfaced that this is wrong in practice. Each lane reports its
own native metric shape:
| Lane | Native metrics |
|---|---|
| cognition eval | `surface_groundedness`, `term_capture_rate`, `intent_accuracy`, `versor_closure_rate` |
| `elementary_mathematics_ood` | `accuracy`, `by_construction`, `passed`, `total` |
| `inference_closure` | `all_pass_rate`, `derived_recall_rate`, `premises_stored_rate`, `replay_determinism`, `overall_pass` |
| `fabrication_control` | `fabricated`, `refused`, by-class refusal counts |
| `foundational_physics_ood` | `accuracy` (shape-equivalent to math OOD) |
| `hebrew_fluency` / `koine_greek_fluency` | `accuracy`, by-construction pass rates |
Enforcing cognition-shape keys uniformly causes every non-cognition
lane to fail the gate by absence-of-key, not by substance. The
ADR-0106 contract is technically correct (no domain passes a contract
written against keys it doesn't produce) but operationally useless.
ADR-0109 amends ADR-0106 to make threshold rules lane-shape-aware,
without weakening the contract's discipline.
---
## Decision
### 1. Lane-shape registry
Introduce an explicit, code-pinned mapping from lane id → lane shape →
threshold rule. The registry lives in `core/capability/expert_demo.py`
(or a sibling module if scope grows). A lane not registered fails
the expert-demo gate fail-closed; introducing a new shape requires an
ADR citing this one.
### 2. Initial shapes
ADR-0109 v1 ships five shapes covering every lane currently attached to
a ratified pack:
#### `cognition_shape`
Threshold (unchanged from ADR-0106 §1.2):
- `surface_groundedness >= 0.95`
- `term_capture_rate >= 0.85`
- `intent_accuracy >= 0.95`
- `versor_closure_rate == 1.0`
Applies to: cognition eval lane only.
#### `accuracy_shape`
Threshold:
- `accuracy >= 0.95` on both public and holdout
- `accuracy` interpreted as `passed / total` when `accuracy` key
absent and `passed` + `total` are present (deterministic fallback)
Applies to: `elementary_mathematics_ood`, `foundational_physics_ood`,
`hebrew_fluency`, `koine_greek_fluency`, and other domain-specific
positive-coverage lanes that report a single accuracy scalar.
#### `inference_shape`
Threshold:
- `all_pass_rate >= 0.95`
- `replay_determinism == 1.0`
- `overall_pass == true`
Applies to: `inference_closure`.
#### `refusal_shape`
Threshold:
- Total `fabricated` count across all by-class buckets == 0
- Every by-class bucket reports `refused == n` (every case refused)
Applies to: `fabrication_control`. Replaces ADR-0106 §1.3's
`passed_rate >= 1.0` check, which presumed a key the lane doesn't
actually emit.
#### `symbolic_logic_shape`
Threshold:
- `accuracy >= 0.95` (re-uses `accuracy_shape` rule)
Applies to: `symbolic_logic` (the v1 closest-fit lane for
`systems_software` per ADR-0101).
### 3. Registry resolution rule
The lane-shape registry is consulted by id, not by metric introspection.
A lane named `inference_closure` always resolves to `inference_shape`;
the registry cannot be tricked by a lane emitting accuracy-shaped keys
under a different lane id. This preserves ADR-0106 §1.1.4's
domain-aware invariant.
### 4. Unknown lanes are fail-closed
If a manifest references a lane id that the registry doesn't recognise,
`evaluate_expert_demo` returns `passed=False` with reason
`"lane <id> has no registered shape — introduce via ADR amendment"`.
This is intentional: adding lanes silently broadens the gate; the ADR
process is the change-control mechanism.
### 5. No effect on ADR-0106 §1.1, §1.3 reviewer-signature
requirements, §1.4 signature scoping, or §1.5 replay byte-equality
ADR-0109 amends §1.2 only. The reviewer-signed claim, the signer's
`eval` scope requirement, the cross-domain bleed refusal, and the
evidence-digest reproducibility invariant all remain unchanged.
The `derive_evidence_digest` canonicalisation (sorted keys, compact
separators, full lane metrics included) is unchanged. A claim signed
under ADR-0106 metric assumptions remains replay-equivalent under
ADR-0109; the gate's *acceptance* changes, the *digest input* does not.
---
## Invariants
### `lane_shape_explicit`
Every lane id referenced by any ratified pack's manifest must resolve
to a registered shape. A pytest gate iterates `DOMAIN_PACKS`, collects
all `eval_lanes[].lane` values, and asserts each resolves.
### `shape_thresholds_are_named`
Each shape carries a documented minimum per metric. No implicit defaults.
A new shape is introduced only by amending this ADR (or a successor).
### `unknown_lane_fails_closed`
A claim citing a lane id absent from the registry produces
`ExpertDemoVerdict(passed=False, reason=...)`. Tested by a fixture
that asserts this exact behaviour.
### `cognition_shape_unchanged_under_amendment`
A claim signed against cognition-shape thresholds before ADR-0109
remains valid under ADR-0109. This is enforced by holding the
`cognition_shape` thresholds bit-identical to ADR-0106 §1.2 and by a
test that verifies `derive_evidence_digest` is stable across the
amendment.
---
## Acceptance evidence
Accepted when:
- `core/capability/expert_demo.py` carries the registered shapes,
resolution function, and the rewritten `_meets_thresholds` that
dispatches by shape.
- `tests/test_lane_shape_thresholds.py` covers the four invariants
above.
- README "Accepted reasoning-capable domains" preface notes the
shape-aware gating.
- No domain row's `expert_demo` field flips by this PR — the math row
remains at `reasoning-capable` (ADR-0107's deferral stands; ADR-0110
is the re-attempt under this amended contract).
- ADR-0106 status remains Accepted (this ADR amends, doesn't supersede).
---
## Consequences
- The expert-demo contract becomes operationally testable for the
first time without forcing every domain into cognition-shape
evidence.
- Adding a new domain or a new lane shape becomes an explicit ADR
step. The registry is the change-control surface.
- ADR-0110 can proceed to attempt `mathematics_logic` promotion under
rules that match the math lanes' actual outputs. Promotion still
requires `inference_closure` to pass (`all_pass_rate >= 0.95` per
the new `inference_shape` rule) — independent of metric-shape work.
- Three of the four ratified domains
(`physics`, `systems_software`, `hebrew_greek_textual_reasoning`)
can also be evaluated for expert-demo without further amendment
once their attached lanes carry results meeting their shape's
thresholds.
---
## Out of scope
- This ADR does not investigate or fix `inference_closure`. ADR-0110
must verify that lane passes before promoting math.
- This ADR does not change which lanes are attached to which domains.
- This ADR does not amend the reviewer registry schema. The
`expert_demo_claims` block from ADR-0106 is sufficient under
ADR-0109; only the threshold-resolution logic changes.
- Multi-reviewer governance (the ADR-0105 candidate frontier item)
remains orthogonal and future work.