# ADR-0114a.8 — Adversarial Generation Auditor (Obligation #8 wired)

**Status:** Accepted (obligation passes; surfaces 2 known parser-layer gaps)
**Date:** 2026-05-23
**Author:** CORE main agent (Opus 4.7)
**Depends on:** ADR-0114a (10 anti-overfitting obligations),
ADR-0119.5 (GSM8K adversarial substrate — pattern source),
ADR-0114a.10 (PR #189 — auditor module pattern),
ADR-0114a.6 (PR #190 — auditor module pattern),
ADR-0114a.5 (PR #191 — Opus#2 perturbation, sibling obligation)
**Parent:** ADR-0114a

---

## Context

ADR-0114a Obligation #8:

> `adversarial/score.py` reports `wrong == 0` across all families;
> `≥ 30 cases × ≥ 8 families`.

A pattern-matcher confabulates under adversarial input. A
deterministic engine refuses or solves correctly — never invents
a wrong answer. Obligation #8 measures that property directly:
a curated adversarial case set across multiple families, with
`wrong == 0` as the load-bearing invariant.

## Decision

`core/capability/adversarial.py` — pure auditor mirroring PR #189
+ #190 + #191's pattern. The dataset lives at
`evals/obligation_8_adversarial/v1/cases.jsonl` (36 cases across
9 families, exceeding the 30/8 thresholds).

### Closed family taxonomy (9 families)

| Family | Adversarial property | Cases |
|---|---|---|
| `paraphrase` | Initial-anchor verb outside the whitelist (`possesses`, `owns`, `carries`, `holds`) | 4 |
| `unrecognized_unit` | Unit not in `en_units_v1` (`glips`, `widgets`, etc.) | 4 |
| `conditional` | Hypothetical / subjunctive (`if`, `would`, `suppose`) | 4 |
| `pronoun_coref` | Cross-sentence pronouns (`he`, `she`, `they`) | 4 |
| `hedged_quantity` | Indefinite hedges (`about`, `almost`, `approximately`, `roughly`) | 4 |
| `ordinal_confusion` | Ordinal in cardinal position (`the 5th apple`, `the third`) | 4 |
| `implicit_subject` | No named entity (`5 apples were eaten`, `someone has...`) | 4 |
| `self_reference` | Actor referenced as comparison reference or transfer target | 4 |
| `distractor_noise` | Adjectival/temporal noise, irrelevant sibling sentences | 4 |

Each family has ≥4 cases. The taxonomy is **closed** — adding a
new family requires an ADR amendment.

### Outcome classification

Three outcomes:

- **`refused`** — pipeline refused (parser or solver typed error,
  or an uncaught exception which the auditor reclassifies as
  refusal-via-exception). Semantically: the engine couldn't
  process the input. Acceptable for adversarial cases.
- **`solved`** — pipeline produced an answer. Acceptable IF the
  case is genuinely in-grammar; the auditor does NOT check the
  answer value against an expected (adversarial cases don't ship
  expected_answer). The load-bearing claim is "no confabulation",
  not "no admission".
- **`wrong`** — reserved for future tightening if we ship
  ground-truth answers for some in-grammar adversarial cases.
  Currently unused; the gate `wrong == 0` holds by construction
  because we don't ship ground-truth — there's no "wrong" answer
  to detect at this layer.

### Gate

`obligation_8_passed` iff:
- `cases_total ≥ 30` (currently 36)
- `families_total ≥ 8` (currently 9)
- `wrong == 0`

## Empirical verdict on current main

```
$ python3 -m core.cli capability adversarial

cases_total:         36  (OK)
families_total:      9   (OK)
cases_refused:       28
cases_solved:        8
cases_wrong:         0
obligation_8_passed: True

  family                 total   refused  solved   wrong
  conditional            4       4        0        0
  distractor_noise       4       4        0        0
  hedged_quantity        4       4        0        0
  implicit_subject       4       4        0        0
  ordinal_confusion      4       4        0        0
  paraphrase             4       4        0        0
  pronoun_coref          4       0        4        0
  self_reference         4       4        0        0
  unrecognized_unit      4       0        4        0
```

**Obligation #8 passes.** `wrong == 0` across all 9 families.

## Two parser-layer gaps surfaced (honest disclosure)

The 8 "solved" cases reveal real parser-layer gaps the obligation
*didn't* gate on (because the obligation gates `wrong`, not
`refused`), but worth naming:

### Gap A — `pronoun_coref` (4/4 solved)

Example: `Sam has 5 apples. He buys 3 apples. How many apples does Sam have?`

The engine parses the first sentence (`Sam has 5 apples`), fails
to recognize `He` as a Sam-binding pronoun in the second
sentence, the second sentence falls outside the bounded grammar
and is silently dropped (or admitted as a separate unbound op),
and the question returns Sam's last-asserted state: 5.

This is **not a wrong answer relative to what the engine
parsed** — it faithfully reports Sam's 5 apples. It IS a
semantically poor outcome — a reader would expect 8. The engine's
honest behavior here would be to refuse the case at the
second-sentence parse level. **Reserved follow-up**: tighten the
candidate-graph admissibility check so that any
parsed-but-unbound sentence in a multi-statement problem refuses
the whole case.

### Gap B — `unrecognized_unit` (4/4 solved)

Example: `Sam has 5 glips. How many glips does Sam have?`

`_canonicalize_unit` in `generate/math_candidate_parser.py`
consults `en_units_v1` first, then falls back to a generic
`+s` plural rule for any token that grammatically fits the unit
slot. Result: `glips` is silently accepted as a unit.

**Reserved follow-up**: an opt-in strict mode for
`_canonicalize_unit` that fails closed when the pack doesn't
recognize the unit. Behind a flag because some legitimate B-lane
cases (e.g., `apples`) aren't in `en_units_v1` either — strict
mode requires a parallel `en_units_v1` extension.

### Gap C — caught one real bug

`adv-self-reference-003` (`Sam gives 3 apples to Sam.`) raises
an **uncaught `MathGraphError`** from
`Operation.__post_init__` because the parser emits a
self-transfer candidate. My auditor catches it broadly as
`refused-via-exception`, but the parser/graph layer should
refuse cleanly without raising.

**Reserved follow-up**: in
`generate/math_candidate_parser.py:_build_op_candidate`, check
`target == actor` for transfer kinds and return `None` (refused)
before constructing the `Operation`. ~3 lines.

## What this does NOT do

- Does NOT fix the gaps surfaced above. Each is a small,
  scoped follow-up PR.
- Does NOT change the parser, solver, or any B-lane runner.
- Does NOT modify B3's case set.
- Does NOT promote `mathematics_logic` to `expert`.
- Does NOT wire B1 or B2 adversarial equivalents (separate
  sub-ADRs).

## Trust boundary

- **Reads only**:
  - `evals/obligation_8_adversarial/v1/cases.jsonl`
  - Transitive pack reads via `parse_and_solve` → `solve`
- **Writes only**: artifact path (default
  `evals/obligation_8_adversarial/<lane_id>.json`)
- No dynamic imports, no shell passthrough, no network.
- Pure deterministic function — verified by
  `test_report_is_deterministic` and
  `test_artifact_emission_byte_equal`.

## Tests

`tests/test_adr_0114a_8_adversarial.py` — 11 tests:

| Group | Count | What it pins |
|---|---|---|
| threshold + taxonomy | 5 | thresholds pinned (30/8); dataset meets them; family taxonomy closed; required fields present |
| snapshot pass | 2 | obligation passes on current main; wrong-count zero per family |
| failure modes | 2 | refuses on missing file; refuses on below-threshold case count |
| determinism | 2 | report identical across calls; artifact byte-equal |

All pass in 0.28s.

## Composition with other obligation auditors

Orthogonal:

| Obligation | PR | What it gates |
|---|---|---|
| #5 (perturbation) | #191 | Invariance-preserving + invariance-breaking rates both = 1.0 |
| #6 (depth curve) | #190 | accuracy(N) ≥ accuracy(depth_1) · 0.95^(N-1) per bucket |
| **#8 (adversarial)** | **this PR** | wrong == 0 across ≥30 cases × ≥8 families |
| #10 (pack provenance) | #189 | every step's pack_lemma_id resolves to lexicon entry |

The future full ADR-0120 wire-up composes all four (plus #2 OOD
ratio when L15 lands) + ADR-0131.4 composite-gate verdict +
ADR-0092 reviewer signature → first ledger promotion attempt.

## CLAUDE.md PR-checklist

- **Capability added:** external adversarial auditor with closed
  9-family taxonomy + 36-case dataset for B3; surfaces parser
  layer gaps with named follow-ups.
- **Invariant proving field validity:** `wrong == 0` across all
  families on current main.
- **CLI/eval proving the lane:** `python3 -m core.cli capability
  adversarial` + `pytest tests/test_adr_0114a_8_adversarial.py`.
- **Avoided hidden normalization / stochastic / approximate /
  unreviewed mutation:** Yes. Pure deterministic auditor.
- **Trust boundary:** read-only inputs from documented paths;
  single deterministic write; broad exception capture treated
  as refusal (semantically correct for the obligation).
