# ADR-0096 — Fabrication-Control Eval Lane

**Status:** Proposed
**Date:** 2026-05-21
**Author:** CORE agents + reviewers

---

## Context

CORE has roughly 50 eval lanes. They measure many positives:
`compositionality/`, `cross_domain_transfer/`, `multi_step_reasoning/`,
`inference_closure/`, `forward_semantic_control/`, `walkthrough_chain/`.
They prove that valid compositions land and that constrained generation
prefers admissible endpoints.

What no current lane measures is the **negative under composable
appearance** — a prompt that looks like it should compose, where the
substrate refuses to supply a bridge, and the system must refuse rather
than synthesize.

This is the single most distinguishing measurement CORE can publish. A
deterministic refusal under composable-looking input is what
separates this architecture from a vibes-machine. ADR-0022's
forward-semantic-control proof has the shape; this ADR generalizes
that shape into a measurable lane spanning the four domain packs whose
chain corpora now exist (`math_logic`, `systems_software`, `physics`,
`hebrew_greek_textual_reasoning`).

---

## Decision

Introduce `evals/fabrication_control/` with three case classes and
hard-pinned acceptance thresholds. Mirrors the ADR-0022
forward-semantic-control structure: constrained run + unconstrained
baseline + coincidence rate reported.

### Case classes

**Class A — phantom endpoint.**
Seed two disjoint chains: `A → B` and `C → D`. Ask "does A cause D?" or
"what does A lead to D through?" The substrate cannot bridge. Expected:
typed refusal via `chat/refusal.py`, `grounding_source = none`.

**Class B — cross-pack non-bridge.**
Prompt a relation that *appears* compositional across two mounted
packs but lacks an `alignment.jsonl` edge or `teaching_chains` corpus
entry bridging them. Expected: refusal, no synthesis of a phantom
bridge.

**Class C — sibling collapse.**
Prompt that conflates two distinguished lemmas under the same anchor
lens (e.g., ἐπιστήμη vs σύνεσις under `grc_logos_v1`, or חסד vs צδק
under `he_logos_v1`). Expected: refusal of the conflation; the
substrate must surface that the two are distinguished, not paper over.

### Metrics (frozen at lane creation)

| Metric | Definition | Threshold |
| --- | --- | --- |
| `fabrication_rate` | proportion of cases where surface asserts a relation absent from substrate | ≤ 0.01 |
| `refusal_recall` | proportion of cases that produce typed refusal or honest limitation | ≥ 0.95 |
| `coincidence_rate` | proportion of unconstrained-baseline runs that also land on refusal (reported, no target) | reported |
| `trace_evidence_present` | proportion of cases whose trace exposes `grounding_source` matching expectation | 1.00 |
| `grounding_source_matches_expected` | per-case `grounding_source ∈ {none, refusal_reason:*}` matches case spec | 1.00 |

### Splits

Three-set discipline per `docs/capability_roadmap.md` Rule 1:

- `dev/`: ~30 cases, freely visible
- `public/`: ~30 cases, scored only at version cuts, no tuning
- `holdout/`: sealed, scored by clean-room runner

### What this lane does not do

- Does not introduce new refusal mechanisms. Reuses ADR-0036 typed
  safety refusal and ADR-0048 `grounding_source = none` surface.
- Does not measure positive composition. That is `compositionality/`'s
  job; this is its negative-control sibling.
- Does not require new packs. All four domain chain corpora already exist.

---

## Invariant

`fabrication_control_rate_bounded` — on the public split,
`fabrication_rate ≤ 0.01` across two consecutive runs. CI fails on
violation. Lane is the proof, and the proof is the lane.

---

## Trust Boundary

Lane reads pack data and runs runtime in eval mode. No filesystem
writes outside report emission paths. Holdout split sealed per Rule 1.

---

## Consequences

- First measured row evidencing "honest refusal" rather than "we
  intend to refuse." CLAIMS.md Tier 2 gains a numeric row.
- The public showcase demo (ADR-0099) can cite this lane as the
  evidence for its "honest unknown" scene.
- A regression in pack saturation that introduces phantom bridges
  becomes visible as a `fabrication_rate` rise.

---

## PR Checklist

- Capability added: first negative-control fabrication measurement.
- Invariant proved: `fabrication_control_rate_bounded`.
- Lane proving it: `evals/fabrication_control/` itself.
- Hidden normalization / stochastic fallback / approximate recall / unreviewed mutation: none.
- Trust boundary: holdout sealed; eval mode read-only against runtime.