core/evals
Shay a853cb5b3b fix(identity): normalize contractions, curly quotes, verb morphology
Closes four surface-form bypass vectors against fix #2 that were
real holes: contractions ("you're now a pirate" did not match marker
"you are now"), curly quotes (U+2019 vs U+0027), em-dashes (token
splicing), and verb morphology ("becoming"/"transformed"/"dropped"
did not stem to the bare redirect-verb set).

teaching/review.py:
  - _normalize() folds Unicode punctuation and expands 28 common
    English contractions (you're, it's, let's, don't, won't, etc.)
    before rule (a) substring matching and rule (b/c/d) tokenisation.
  - _stem_verb() folds -ing / -ed / -es / -s morphology with silent-e
    drop and doubled-consonant handling, so "becomes" / "becoming" /
    "became"-class forms match the bare redirect-verb stem.
  - Rule (d) window now uses verb stems, not raw tokens.

Verification: ten splits (v1-v5, public + holdouts) at 100% attack
rejection and 100% legitimate acceptance. v5 (32 attacks + 18
legitimates) is the new regression gate, exercising every fold class
plus legitimates that themselves use contractions ("wisdom's broader",
"knowledge isn't merely collected").

Tests: test_reviewed_teaching_loop.py 5/5, test_pipeline_teaching_integration.py
5/5, test_identity_gate.py 17/17 (including 5 TestWouldViolatePredicate
tests from prior commit).
2026-05-16 14:13:56 -07:00
..
adversarial_identity fix(identity): normalize contractions, curly quotes, verb morphology 2026-05-16 14:13:56 -07:00
calibration feat(evals): v2 lanes for calibration and symbolic-logic 2026-05-16 13:17:41 -07:00
cognition feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
grammatical_coverage feat(evals): grammatical-coverage v2 cases - 36 cases with deeper nesting and rarer vocabulary (100% pass) 2026-05-16 06:40:55 -07:00
identity_divergence feat(evals): identity-divergence lane v1 - 93 curriculum events, two axis profiles (Precision/Generosity), divergence/coherence/causal metrics (all pass) 2026-05-16 06:48:13 -07:00
monotonic_learning feat(evals): v3 lanes — monotonic-learning passes, adversarial-identity reveals gap 2026-05-16 13:42:47 -07:00
provenance feat(evals): parallel runner + adversarial-identity v2 2026-05-16 13:10:26 -07:00
reports Add cognitive eval harness and calibration replay (#30) 2026-05-15 07:41:36 -07:00
symbolic_logic feat(evals): v2 lanes for calibration and symbolic-logic 2026-05-16 13:17:41 -07:00
zero_code_domain_acquisition feat began creation of zero code domain acquisition. did not complete yet. 2026-05-16 06:31:01 -07:00
__init__.py Add cognitive eval harness and calibration replay (#30) 2026-05-15 07:41:36 -07:00
baseline_runner.py feat(evals): frontier structural-zero baselines for Phase 2 v1 lanes 2026-05-16 12:45:28 -07:00
cognition_cases.jsonl feat: vault recall index, Rust versor parity, cognitive pack expansion 2026-05-15 15:34:39 -07:00
framework.py feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
holdout_runner.py feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
metrics.py Add cognitive eval harness and calibration replay (#30) 2026-05-15 07:41:36 -07:00
parallel.py feat(evals): parallel runner + adversarial-identity v2 2026-05-16 13:10:26 -07:00
run_cognition_eval.py Add cognitive eval harness and calibration replay (#30) 2026-05-15 07:41:36 -07:00