core/docs
Shay 1e2fce5f4f feat(evals): v3 lanes — monotonic-learning passes, adversarial-identity reveals gap
Closes the Phase 2 roadmap exit gate (v3 for at least two lanes).

monotonic-learning v3:
  public/v3  — 7 domains × 3-4 probes × 30 cycles (805 ops)
                domains: truth, light, wisdom, order, memory, meaning, identity
                max_regression=0.0, floor_score=1.0
  holdouts/v3 — 6 distinct domains × 4 probes × 25 cycles (597 ops)
                domains: creation, knowledge, reason, spirit, principle, judgment
                max_regression=0.0, floor_score=1.0

  Zero regression across 55 cumulative cycles and 13 distinct domains.
  Sequential within a split (single shared longitudinal session) is
  structural; public/v3 ran in 12.5 min, holdouts/v3 in 8.2 min.

adversarial-identity v3:
  public/v3  — 20 paraphrased attacks + 10 legitimate corrections
  holdouts/v3 — 12 paraphrased attacks + 8 legitimate corrections
  Result: attack_rejection_rate=0.0, legitimate_acceptance_rate=1.0
          on both splits.

  THIS IS THE INTENDED OUTCOME OF v3. The attack set was constructed
  to use only paraphrased / synonym-shifted phrasings that avoid every
  literal string in teaching/review.py::_IDENTITY_MARKERS. Every
  attack ("become a different agent", "behave as", "assume the role
  of", "switch into helper mode", "transform into", "drop the
  framework", "let's roleplay that") bypasses the marker list and is
  admitted to the teaching store as a PackMutationProposal.

  evals/adversarial_identity/gaps.md documents the finding in detail
  and proposes three follow-up fixes in increasing order of weight:
    1. Extend _IDENTITY_MARKERS with verb-of-becoming and role-frame
       classes (cheapest, still string-matching).
    2. Semantic syntactic check on
       [redirect-verb] + [self-reference] + [role-frame] structure.
    3. Geometric identity-versor check (architectural; aligns with
       ADR-0010 identity-as-geometry doctrine — synonymous attacks
       produce similar field deltas, so the defense is paraphrase-
       invariant by construction).

  v1 (38 attacks, all blocked) and v2 (32 attacks, all blocked)
  remain valid for their declared coverage (the marker-list smoke
  test and its punctuation/case variants). v3 is recorded as a
  known-failing stress test, not a regression — it is load-bearing
  evidence for the v4 / architectural fix work above.

Phase 2 status: COMPLETE.
  - All five lanes v1+v2 at 100% (provenance, monotonic-learning,
    calibration, symbolic-logic, adversarial-identity)
  - Frontier structural baselines documented for all five
  - v3 exit gate met: monotonic-learning v3 passes, adversarial-
    identity v3 reveals load-bearing architectural finding
  - Test suite: 596 passing (no regression)
2026-05-16 13:42:47 -07:00
..
architecture feat(physics): add mind-physics layer — ADR-0008/0009/0010, blueprint, and operator stubs 2026-05-12 23:20:58 -07:00
decisions feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
sessions feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
capability_roadmap.md feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
eval_methodology.md feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
frontier_baselines.md feat(evals): frontier structural-zero baselines for Phase 2 v1 lanes 2026-05-16 12:45:28 -07:00
PROGRESS.md feat(evals): v3 lanes — monotonic-learning passes, adversarial-identity reveals gap 2026-05-16 13:42:47 -07:00
runtime_contracts.md Clean up runtime contracts before cognitive pipeline 2026-05-14 18:47:59 -07:00
RUST.md Activate and verify Rust backend 2026-05-13 22:23:48 -07:00
Whitepaper.md docs: update Whitepaper and Yellowpaper with ingest governance, sensorium layer, and pipeline diagram 2026-05-13 11:20:38 -07:00
Yellowpaper.md docs: update Whitepaper and Yellowpaper with ingest governance, sensorium layer, and pipeline diagram 2026-05-13 11:20:38 -07:00