core/evals/gsm8k_math/adversarial
Shay 3bda4313c9 feat: ADR-0119.5 — adversarial generation (closes ADR-0114a Obligation #8)
Phase 5.5 of ADR-0119. Adversarial case generator + scoring CLI;
discharges the last remaining ADR-0114a obligation.

Numbers
  adversarial suite: 38 cases × 12 families
  per-family: every family produces wrong == 0
  overall: correct 5, wrong 0, refused 33

Families
  conditional_phrasing       (4)  "If/When/Suppose ..."
  compound_questions         (3)  multiple ?
  undefined_entity_question  (3)  question references unknown entity
  unknown_verb               (5)  "polishes", "admires", etc.
  empty_or_whitespace        (3)  empty input
  no_question                (3)  statement-only
  numbers_spelled_out        (3)  "five", "ten"
  passive_voice              (3)  "X are bought by Y"
  red_herring_numbers        (3)  digits in name positions, mid-quantity
  question_only              (2)  no preceding statements
  mid_sentence_punctuation   (2)  embedded ? or !
  subtle_in_grammar          (4)  IN-grammar; runner must produce correct
                                  (gate-sanity: not trivially "refuse all")

The subtle_in_grammar family is the load-bearing sanity check —
proves the gate isn't trivially satisfied by refusing everything.

ADR-0114a obligation status

  10 of 10 discharged on main:
    #1  fab_control lane (0119.1); GSM8K test pending (0119.7)
    #2  ADR-0118a
    #3  ADR-0117
    #4  ADR-0116 + ADR-0119.3
    #5  ADR-0125
    #6  ADR-0119.6 harness; ε threshold to ADR-0120
    #7  ADR-0119.4
    #8  THIS ADR
    #9  ADR-0116/0117/0118/0119.3
    #10 ADR-0116

Phase 5 remaining: 5.7 (sealed GSM8K test, real corpus) and 5.8
(overall lane gate). After those, ADR-0120 (first expert promotion
contract) can compose all ten obligations.

Tests: 18 new + 25 prior Phase 5 = 43 green; 67/67 smoke.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 18:11:36 -07:00
..
__init__.py feat: ADR-0119.5 — adversarial generation (closes ADR-0114a Obligation #8) 2026-05-22 18:11:36 -07:00
generator.py feat: ADR-0119.5 — adversarial generation (closes ADR-0114a Obligation #8) 2026-05-22 18:11:36 -07:00
score.py feat: ADR-0119.5 — adversarial generation (closes ADR-0114a Obligation #8) 2026-05-22 18:11:36 -07:00