core/evals/adversarial_identity
Shay 86ef117f6e docs(identity): empirical finding — fix #3 needs upstream ingest-gate work
Followed up the prior carry-forward (sharpen IdentityManifold axis
vectorisation) with a focused empirical investigation. Probed every
candidate per-case discriminator derivable from the existing
CognitiveTurnResult across v3 and v5:

  Signal                          Attack   Legit   Separable
  identity_score.alignment         1.000   1.000   no - identical
  field-delta L2 norm              ~3.4    ~3.9    no - heavy overlap
  semantic-coord energy ratio      ~0.88   ~0.91   no - overlap
  vault_hits                       ~8.6    ~7.9    no - overlap
  surface length / intent tag      same    same    no

The pipeline encodes identity-override attacks and legitimate
corrections into statistically indistinguishable field-state
geometries. No amount of axis-direction sharpening on the
IdentityManifold can recover a signal that isn't present in the
trajectory data being projected.

Architectural conclusion: fix #3 cannot be made load-bearing in
place. Required upstream work (out of scope for this PR):

  1. ingest/gate.py: encode token semantic categories (redirect-verb,
     role-frame, self-reference, negating-qualifier) into specific
     blade coordinates of the field versor at injection time.
  2. IdentityManifold axes in the 32-dim Cl(4,1) basis with directions
     derived from post-(1) empirical signatures.
  3. Replace _axis_projection with a real inner-product projection of
     trajectory delta onto axis directions.

What stands today: fix #2 (syntactic) + normalization reject 100% of
v1-v5 attacks (n=121) with 0 false positives on 51 legitimates -
this is the load-bearing defense. Fix #3's predicate, unit tests,
and pipeline wiring remain as scaffolding for the upstream work.

Adds:
  - evals/adversarial_identity/calibration/probe_field_signature.py
    The reproducible empirical baseline. Any future ingest-gate
    change must demonstrate per-case attack/legitimate separation
    on this probe before fix #3 can be claimed load-bearing.
  - Architectural finding written into gaps.md and PROGRESS.md.

This unblocks Phase 3 (reasoning depth). Sharpening fix #3 will be
authored separately when the upstream ingest-gate work is scoped.
2026-05-16 14:23:20 -07:00
..
baselines
calibration docs(identity): empirical finding — fix #3 needs upstream ingest-gate work 2026-05-16 14:23:20 -07:00
dev
holdouts
public
results
__init__.py
contract.md
gaps.md docs(identity): empirical finding — fix #3 needs upstream ingest-gate work 2026-05-16 14:23:20 -07:00
runner.py