# Forward Semantic Control — Results Directory

This directory holds the canonical evidence reports for every phase of
the ADR-0024 Forward Semantic Control chain. Every report here is
machine-generated by a runner under `evals/forward_semantic_control/`
and re-runnable via the `core` CLI.

If you are reading this cold and want to know "what does CORE actually
do that an LLM cannot," **start with `phase6_demo_report.json`** — it
holds the three head-to-head conditions vs the in-system baseline.

---

## How to regenerate everything

```bash
core demo all              # runs phase5 + phase6, refreshes reports + index
core demo list-results     # prints the index with headline metrics
```

To run a single phase:

```bash
core demo phase5           # stratified mechanism-isolation (~30 s)
core demo phase6           # 3-condition comparative demo (~10 s)
```

Both write JSON reports here and refresh `index.json`.

---

## Reports — what's in this directory

### `phase2_inner_loop_report.json`

**Lane:** Corpus observation (4-condition matrix on the existing FSC v1 corpus)
**Runner:** `inner_loop_runner.py`
**ADR:** ADR-0024 Phase 2
**Status:** Historical evidence — the case schema is teaching-driven (v1/dev) and is no longer the load-bearing benign corpus (see `phase5_benign_inner_loop_report.json` below).

Reports per condition (boundary_only / null_control / inner_loop_t0 / inner_loop_tpos):
`pass_rate`, `mean_rejection_count_per_turn`, `non_empty_rejected_attempts_rate`,
`exhaustion_rate`, `mean_admissibility_checks_per_turn`, latency stats,
`trace_hash_stability_pass_rate`.

**Headline interpretation.** The Phase 2 corpus is *adversarial-by-accident*: the v1 case schema probes teaching-driven walks, not inner-loop admissibility. Exhaustion rates above 0.05 on this corpus are an *architectural finding* (documented in `docs/adr/ADR-0024-inner-loop-admissibility.md` Phase 1 addendum), not a regression.

### `phase3_v2_report.json`

**Lane:** Mechanism isolation (5 adversarial v2 cases)
**Runner:** `v2_runner.py`
**ADR:** ADR-0024 Phase 3
**Headline metric:** `mechanism_isolated == True` iff
`pass_rate == 1.0 AND boundary_decoy_rate == 1.0 AND rejection_traced_rate == 1.0`.

**Interpretation.** Each case is constructed so the boundary picks a *forbidden* token and the inner-loop must override and trace the rejection. A pass on all three conditions causally attributes the selection difference to the inner-loop, not to any other code path.

### `phase4_characterization_v1_plus_dev.json` / `phase4_characterization_v2.json` / `phase4_characterization_combined.json` / `phase4_summary.json`

**Lane:** Threshold characterization (diagnostic sweep)
**Runner:** `threshold_characterization.py`
**ADR:** ADR-0024 Phase 4 (diagnostic, not a tuning artifact)
**Headline metric:** `best_separation_quality` and `geometry_supports_static_threshold` per threshold value swept over `{-1.0, -0.5, 0.0, 0.1, 0.25, 0.5, 1.0}`.

**Interpretation.** **No static threshold delivers separation_quality ≥ 0.8 on the v1+dev+v2 corpus.** This is the load-bearing finding that motivated ADR-0026's switch from threshold to ranked-with-margin. If you want the geometric justification for the δ-margin gate, this report is it.

### `phase5_report.json` *(load-bearing — read first if you only read one)*

**Lane:** Stratified mechanism-isolation across 5 failure-mode families
**Runner:** `phase5_runner.py`
**ADR:** ADR-0024 Phase 5
**Corpus:** 20 cases under `public/v2_phase5/cases.jsonl`
**Headline metrics:**
- `metrics.pass_rate_threshold` — overall under static threshold
- `metrics.pass_rate_margin` — overall under δ-margin (the operational mode)
- `metrics.mechanism_isolated_threshold` / `mechanism_isolated_margin` — booleans for each mode
- `per_family[<family>].pass_rate_threshold` / `pass_rate_margin` / `refusal_rate_margin` — per-family breakdown

**Interpretation.** All five families should pass at 100% under both modes. Family B (`near_equal_admissible`) should additionally show `refusal_rate_margin = 100%` — those cases are constructed so δ-margin MUST refuse. Family C (`no_admissible_path`) should show `refusal_rate = 100%` in BOTH modes.

**When to look here:** any per-family pass_rate drop. The family that drops is the failure mode that broke.

### `phase5_benign_inner_loop_report.json`

**Lane:** Benign corpus exhaustion-ceiling lane
**Runner:** `inner_loop_runner.py` (same as Phase 2, different corpus)
**Corpus:** `public/inner_loop_benign/cases.jsonl` (10 single-token cases)
**Headline gate:** `per_condition[*].exhaustion_rate ≤ 0.05` (EXHAUSTION_CEILING)

**Interpretation.** This is the corpus the EXHAUSTION_CEILING gate was *actually* designed against — a curated benign corpus where the inner-loop should rarely refuse. Expected: 0.00 exhaustion in all four conditions. If exhaustion rises here, either (a) a benign case's expected token now has negative self-`cga_inner` under the active pack (Cl(4,1) signature artifact, see `docs/evals/phase5_stratified_findings.md`), or (b) the inner-loop has regressed.

### `phase6_demo_report.json` *(headline demo — start here if you're new)*

**Lane:** Comparative demo, 3 conditions vs in-system baseline (ADR-0023 ablation)
**Runner:** `phase6_demo.py`
**ADR:** ADR-0024 Phase 6
**Corpus:** 8 cases under `public/v2_phase6_demo/cases.jsonl`
**Headline metrics:**
- `metrics.c1_pass` — Replay determinism (baseline AND CORE byte-identical across 5 reruns)
- `metrics.c2_pass` — Traced rejection (boundary emits forbidden, CORE corrects + traces)
- `metrics.c3_pass` — Coherent refusal (baseline emits inadmissible, CORE raises typed refusal)
- `metrics.all_three_conditions_pass` — boolean AND

**Interpretation.** This is THE comparative demo. Each `cN_pass` is a *single boolean claim*. If `all_three_conditions_pass` is true, the chain delivers on its three head-to-head claims. If any is false, see the per-case detail and the C-specific breakdown in the same JSON.

### `index.json`

**Auto-generated by `core demo`.** Lists every report file with size and a curated subset of headline metrics. Refreshed by every `core demo` run. Use:

```bash
core demo list-results       # human-readable
core demo list-results --json # machine-readable index
```

---

## How to interpret a per-case detail (general schema)

Each report's `case_details` list carries per-case evidence. Common fields:

| Field | Meaning |
|---|---|
| `id` | Stable case ID (e.g. `FSC-P5-A-001`) |
| `family` | Failure-mode family (Phase 5 only) |
| `condition` | One of `replay_determinism`, `traced_rejection`, `coherent_refusal` (Phase 6 only) |
| `boundary` | Boundary-only leg result: `{selected, admitted, rejected_words, ...}` |
| `threshold_leg` / `margin_leg` / `core` | Per-mode leg results |
| `passed_threshold` / `passed_margin` / `passed_threshold` | Pass predicate verdicts |
| `replay_hashes_*` | List of N trace hashes from N reruns (Phase 6 C1) |
| `c2_*` / `c3_*` | Phase 6 condition-specific predicates |

A `refused: true` leg additionally carries `refusal_reason` (the stable enum value), `refusal_message`, and `rejected_attempts`.

---

## When something looks wrong

1. **Re-run the responsible phase first.**
   `core demo phase5` or `core demo phase6` regenerates the report from scratch.
2. **Compare against the contract tests.**
   `core test --suite phase5` (20 tests) and `core test --suite phase6` (17 tests) pin the headline numbers. If those pass and the JSON looks wrong, the runner has drifted from the test predicate — not the implementation.
3. **Check the central architectural finding doc.**
   `docs/evals/phase5_stratified_findings.md` documents the known geometric quirks of Cl(4,1) that motivate the δ-margin gate. Many "regressions" turn out to be self-`cga_inner < 0` cases that need a different region shape, not a code fix.
4. **Last resort: full chain.**
   `core test --suite adr-0024` (98 tests, ~2 minutes) runs every Phase 2-6 contract test. If THAT passes and a demo report still looks wrong, the corpus has aged out of the failure mode it was designed to surface — see each corpus's own README for guidance on adding new cases.

---

## Related docs

- `docs/specs/runtime_contracts.md` — Refusal / Margin / Rotor admissibility contracts
- `docs/evals/phase5_stratified_findings.md` — Phase 5 architectural findings
- `docs/evals/phase6_comparative_demo.md` — Phase 6 narrative + "what this does NOT claim"
- `docs/adr/ADR-0024-inner-loop-admissibility.md` — the foundational ADR
- `docs/adr/ADR-0025-rotor-frame-admissibility-design-note.md` — rotor admissibility (accepted)
- `docs/adr/ADR-0026-ranked-admissibility-with-margin.md` — δ-margin gate (accepted)
