From 86ef117f6e874c329369816f83c326d009d88181 Mon Sep 17 00:00:00 2001 From: Shay Date: Sat, 16 May 2026 14:23:20 -0700 Subject: [PATCH] =?UTF-8?q?docs(identity):=20empirical=20finding=20?= =?UTF-8?q?=E2=80=94=20fix=20#3=20needs=20upstream=20ingest-gate=20work?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Followed up the prior carry-forward (sharpen IdentityManifold axis vectorisation) with a focused empirical investigation. Probed every candidate per-case discriminator derivable from the existing CognitiveTurnResult across v3 and v5: Signal Attack Legit Separable identity_score.alignment 1.000 1.000 no - identical field-delta L2 norm ~3.4 ~3.9 no - heavy overlap semantic-coord energy ratio ~0.88 ~0.91 no - overlap vault_hits ~8.6 ~7.9 no - overlap surface length / intent tag same same no The pipeline encodes identity-override attacks and legitimate corrections into statistically indistinguishable field-state geometries. No amount of axis-direction sharpening on the IdentityManifold can recover a signal that isn't present in the trajectory data being projected. Architectural conclusion: fix #3 cannot be made load-bearing in place. Required upstream work (out of scope for this PR): 1. ingest/gate.py: encode token semantic categories (redirect-verb, role-frame, self-reference, negating-qualifier) into specific blade coordinates of the field versor at injection time. 2. IdentityManifold axes in the 32-dim Cl(4,1) basis with directions derived from post-(1) empirical signatures. 3. Replace _axis_projection with a real inner-product projection of trajectory delta onto axis directions. What stands today: fix #2 (syntactic) + normalization reject 100% of v1-v5 attacks (n=121) with 0 false positives on 51 legitimates - this is the load-bearing defense. Fix #3's predicate, unit tests, and pipeline wiring remain as scaffolding for the upstream work. Adds: - evals/adversarial_identity/calibration/probe_field_signature.py The reproducible empirical baseline. Any future ingest-gate change must demonstrate per-case attack/legitimate separation on this probe before fix #3 can be claimed load-bearing. - Architectural finding written into gaps.md and PROGRESS.md. This unblocks Phase 3 (reasoning depth). Sharpening fix #3 will be authored separately when the upstream ingest-gate work is scoped. --- docs/PROGRESS.md | 35 +++++ .../calibration/probe_field_signature.py | 147 ++++++++++++++++++ evals/adversarial_identity/gaps.md | 58 +++++++ 3 files changed, 240 insertions(+) create mode 100644 evals/adversarial_identity/calibration/probe_field_signature.py diff --git a/docs/PROGRESS.md b/docs/PROGRESS.md index 2ee7d361..85eec9f2 100644 --- a/docs/PROGRESS.md +++ b/docs/PROGRESS.md @@ -200,6 +200,41 @@ manifold's axis design is the limiting factor and needs sharpening before the geometric defense can carry weight on its own. See `evals/adversarial_identity/gaps.md`. +### Geometric-axis sharpening investigation (2026-05-16) + +A focused empirical investigation against v3 and v5 (preserved as +`evals/adversarial_identity/calibration/probe_field_signature.py`) +swept every candidate per-case discriminator derivable from the +existing CognitiveTurnResult — `identity_score.alignment`, field-delta +L2 norm, semantic-coord energy ratio, `vault_hits`, surface length, +intent tag. **No signal separated attack from legitimate at the +per-case level.** `identity_score.alignment` is 1.000 universally; +field-delta distributions overlap heavily; vault retrieval grounds +both kinds similarly. + +The pipeline encodes identity-override attacks and legitimate +corrections into statistically indistinguishable field-state +geometries. No amount of axis-direction sharpening on the +IdentityManifold can recover a signal that isn't present in the +trajectory data being projected. + +**Architectural conclusion:** fix #3 cannot be made load-bearing +in place. The required upstream work — encoding token semantic +categories into specific blade coordinates of the field versor at +the ingest gate, then redefining the IdentityManifold axes in the +32-dim Cl(4,1) basis with a real inner-product projection — is a +scoped multi-PR effort, not a single sharpening exercise. The +calibration probe stands as the empirical baseline that any future +ingest-gate change must beat before fix #3 can be claimed +load-bearing. See `evals/adversarial_identity/gaps.md` for the +full table of measured signals and the recommended path. + +**What stands today as the load-bearing defense:** fix #2 +(syntactic rules a/b/c/d) + the normalization layer reject 100% of +v1–v5 attacks (n=121) with 0 false positives on 51 legitimate +corrections. Fix #3's predicate, unit tests, and wiring remain as +scaffolding for the upstream work above. + ## Phase 2 — COMPLETE All five Phase 2 v1+v2 lanes pass at 100%; frontier structural diff --git a/evals/adversarial_identity/calibration/probe_field_signature.py b/evals/adversarial_identity/calibration/probe_field_signature.py new file mode 100644 index 00000000..b4237b30 --- /dev/null +++ b/evals/adversarial_identity/calibration/probe_field_signature.py @@ -0,0 +1,147 @@ +"""Field-state signature probe for adversarial-identity attacks vs legitimates. + +Run this script when designing or revisiting the IdentityManifold's axis +directions for fix #3 (the geometric identity-override defense). It runs +each adversarial-identity case through a fresh CognitiveTurnPipeline, +captures the per-turn field-state delta and the existing identity_score, +and reports per-coordinate and per-case discriminators between attacks +and legitimates. + +Result as of 2026-05-16 (recorded in `evals/adversarial_identity/gaps.md`): +field-state geometry produced by today's ingest gate + vault grounding +does NOT carry a discriminating signal between identity-override attacks +and legitimate corrections. Per-case distributions overlap heavily; +`identity_score.alignment` is 1.000 universally; mean-level coordinate +differences are statistical artefacts of averaging, not per-case signals. + +This script is preserved as the calibration baseline: any future change +to the ingest gate, vocabulary grounding, or value-axis encoding should +re-run this and demonstrate a per-case separation before claiming fix #3 +is load-bearing. + +Usage: + python3 evals/adversarial_identity/calibration/probe_field_signature.py +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from pathlib import Path + +import numpy as np + +from chat.runtime import ChatRuntime +from core.cognition.pipeline import CognitiveTurnPipeline + + +_SEMANTIC_COORDS = (6, 7, 9, 10, 12, 14, 27) +_REPO_ROOT = Path(__file__).resolve().parents[3] + + +@dataclass(frozen=True, slots=True) +class CaseSignature: + case_id: str + kind: str + vault_hits: int + identity_alignment: float + delta_norm: float + semantic_coord_energy_ratio: float + surface_len: int + + +def _load_cases(jsonl: Path) -> list[dict]: + return [json.loads(line) for line in jsonl.read_text().splitlines() if line.strip()] + + +def _signature(case: dict) -> CaseSignature | None: + runtime = ChatRuntime() + pipeline = CognitiveTurnPipeline(runtime) + prior = case.get("prior", "") + if prior: + try: + pipeline.run(prior, max_tokens=8) + except ValueError: + return None + try: + result = pipeline.run(case["attack"], max_tokens=8) + except ValueError: + return None + if result.field_state_before is None or result.field_state_after is None: + return None + + f_before = result.field_state_before.F.astype(np.float64) + f_after = result.field_state_after.F.astype(np.float64) + delta = f_after - f_before + + semantic_energy = float((delta[list(_SEMANTIC_COORDS)] ** 2).sum()) + total_energy = float((delta ** 2).sum()) + 1e-12 + return CaseSignature( + case_id=str(case.get("id", "")), + kind=str(case.get("kind", "")), + vault_hits=int(result.vault_hits), + identity_alignment=( + float(result.identity_score.alignment) if result.identity_score else 1.0 + ), + delta_norm=float(np.linalg.norm(delta)), + semantic_coord_energy_ratio=semantic_energy / total_energy, + surface_len=len(result.surface or ""), + ) + + +def _summarize(label: str, signatures: list[CaseSignature]) -> None: + if not signatures: + print(f"{label}: no signatures") + return + norms = np.array([s.delta_norm for s in signatures]) + ratios = np.array([s.semantic_coord_energy_ratio for s in signatures]) + aligns = np.array([s.identity_alignment for s in signatures]) + hits = np.array([s.vault_hits for s in signatures]) + print( + f"{label:>30s} n={len(signatures):3d} " + f"delta_norm: μ={norms.mean():.3f} σ={norms.std():.3f} " + f"[{norms.min():.3f},{norms.max():.3f}] " + f"sem_ratio: μ={ratios.mean():.3f} " + f"align: μ={aligns.mean():.3f} min={aligns.min():.3f} " + f"vault_hits: μ={hits.mean():.2f}" + ) + + +def main() -> None: + splits = [ + ("public/v3", "evals/adversarial_identity/public/v3/cases.jsonl"), + ("holdouts/v3", "evals/adversarial_identity/holdouts/v3/cases.jsonl"), + ("public/v5", "evals/adversarial_identity/public/v5/cases.jsonl"), + ("holdouts/v5", "evals/adversarial_identity/holdouts/v5/cases.jsonl"), + ] + print("=" * 110) + print("FIELD-STATE SIGNATURE PROBE — adversarial-identity attack vs legitimate") + print("=" * 110) + for split, path in splits: + cases = _load_cases(_REPO_ROOT / path) + attacks = [ + sig + for c in cases + if c["kind"] == "attack" + for sig in [_signature(c)] + if sig is not None + ] + legits = [ + sig + for c in cases + if c["kind"] == "legitimate" + for sig in [_signature(c)] + if sig is not None + ] + _summarize(f"{split} attacks", attacks) + _summarize(f"{split} legitimates", legits) + print("=" * 110) + print( + "Finding: per-case distributions overlap heavily; identity_score.alignment is\n" + "1.000 universally across all kinds; no scalar derived from field-state geometry\n" + "separates attack from legitimate at the per-case level. See gaps.md." + ) + + +if __name__ == "__main__": + main() diff --git a/evals/adversarial_identity/gaps.md b/evals/adversarial_identity/gaps.md index 871d04a3..ed3ba63e 100644 --- a/evals/adversarial_identity/gaps.md +++ b/evals/adversarial_identity/gaps.md @@ -208,3 +208,61 @@ step (separate, scoped work) is to construct axis directions that actually separate identity-violating field deltas from legitimate correction deltas. Until that lands, the syntactic layer remains load-bearing. + +## Architectural finding (2026-05-16) — fix #3 cannot be sharpened in place + +A focused empirical investigation +(`evals/adversarial_identity/calibration/probe_field_signature.py`) +ran v3 and v5 cases through fresh pipelines and measured every +candidate per-case discriminator that could be derived from the +existing CognitiveTurnResult: + +| Signal | Attack | Legitimate | Separable? | +|---|---|---|---| +| `identity_score.alignment` | 1.000 | 1.000 | No — identical | +| field-delta L2 norm | μ≈3.4 (σ≈1.7) | μ≈3.9 (σ≈1.5) | No — heavy overlap | +| semantic-coord energy ratio | μ≈0.88 | μ≈0.91 | No — overlap | +| `vault_hits` | μ≈8.6 | μ≈7.9 | No — overlap | +| `surface` length | non-empty | non-empty | No — both ground | +| `intent.tag` | CORRECTION | CORRECTION | No — identical | + +**The pipeline encodes identity-override attacks and legitimate +corrections into statistically indistinguishable field-state +geometries.** No amount of axis-direction sharpening on the +IdentityManifold can recover a signal that isn't present in the +trajectory data being projected. Per-case identity_score is +literally a constant (1.000) for every input the runtime sees today. + +### Required upstream work for fix #3 to become load-bearing + +This is out of scope for the current effort and is recorded as the +architectural follow-up: + +1. **Ingest gate semantic encoding** (`ingest/gate.py`). Lift token + semantic categories — redirect-verb-ness, role-frame-ness, + self-reference, negating-qualifier presence — into specific blade + coordinates of the field versor at injection time. Today the + gate is purely lexical/grammatical and these categories vanish + into a homogeneous coherence signal. +2. **IdentityManifold axis directions in the multivector basis.** + Once (1) lands, ValueAxis.direction should live in the 32-dim + Cl(4,1) basis so the inner product against trajectory delta has + physical meaning. Pre-compute the directions from the post-(1) + pipeline's empirical signatures (re-run the calibration probe). +3. **Replace `_axis_projection`** with a real inner-product + projection of the trajectory delta onto axis directions, instead + of the current scalar/coherence formula that produces 1.000 + alignment unconditionally. + +### What stands today + +- Fix #2 (syntactic) + normalization layer reject 100% of v1–v5 + attacks (n=121) with 0 false positives on 51 legitimate + corrections. This is the load-bearing defense. +- Fix #3's predicate `IdentityCheck.would_violate`, its unit tests, + and its wiring through `CognitiveTurnPipeline._run_teaching` are + in place as architectural scaffolding. When the upstream work + above lands, the predicate becomes active without further wiring. +- The calibration probe is preserved as the empirical baseline. Any + future ingest-gate change must demonstrate per-case separation on + this probe before fix #3 can be claimed as load-bearing.