docs(identity): empirical finding — fix #3 needs upstream ingest-gate work
Followed up the prior carry-forward (sharpen IdentityManifold axis vectorisation) with a focused empirical investigation. Probed every candidate per-case discriminator derivable from the existing CognitiveTurnResult across v3 and v5: Signal Attack Legit Separable identity_score.alignment 1.000 1.000 no - identical field-delta L2 norm ~3.4 ~3.9 no - heavy overlap semantic-coord energy ratio ~0.88 ~0.91 no - overlap vault_hits ~8.6 ~7.9 no - overlap surface length / intent tag same same no The pipeline encodes identity-override attacks and legitimate corrections into statistically indistinguishable field-state geometries. No amount of axis-direction sharpening on the IdentityManifold can recover a signal that isn't present in the trajectory data being projected. Architectural conclusion: fix #3 cannot be made load-bearing in place. Required upstream work (out of scope for this PR): 1. ingest/gate.py: encode token semantic categories (redirect-verb, role-frame, self-reference, negating-qualifier) into specific blade coordinates of the field versor at injection time. 2. IdentityManifold axes in the 32-dim Cl(4,1) basis with directions derived from post-(1) empirical signatures. 3. Replace _axis_projection with a real inner-product projection of trajectory delta onto axis directions. What stands today: fix #2 (syntactic) + normalization reject 100% of v1-v5 attacks (n=121) with 0 false positives on 51 legitimates - this is the load-bearing defense. Fix #3's predicate, unit tests, and pipeline wiring remain as scaffolding for the upstream work. Adds: - evals/adversarial_identity/calibration/probe_field_signature.py The reproducible empirical baseline. Any future ingest-gate change must demonstrate per-case attack/legitimate separation on this probe before fix #3 can be claimed load-bearing. - Architectural finding written into gaps.md and PROGRESS.md. This unblocks Phase 3 (reasoning depth). Sharpening fix #3 will be authored separately when the upstream ingest-gate work is scoped.
This commit is contained in:
Shay
parent
a853cb5b3b
commit
86ef117f6e
3 changed files with 240 additions and 0 deletions
|
|
@ -200,6 +200,41 @@ manifold's axis design is the limiting factor and needs sharpening
|
|||
before the geometric defense can carry weight on its own. See
|
||||
`evals/adversarial_identity/gaps.md`.
|
||||
|
||||
### Geometric-axis sharpening investigation (2026-05-16)
|
||||
|
||||
A focused empirical investigation against v3 and v5 (preserved as
|
||||
`evals/adversarial_identity/calibration/probe_field_signature.py`)
|
||||
swept every candidate per-case discriminator derivable from the
|
||||
existing CognitiveTurnResult — `identity_score.alignment`, field-delta
|
||||
L2 norm, semantic-coord energy ratio, `vault_hits`, surface length,
|
||||
intent tag. **No signal separated attack from legitimate at the
|
||||
per-case level.** `identity_score.alignment` is 1.000 universally;
|
||||
field-delta distributions overlap heavily; vault retrieval grounds
|
||||
both kinds similarly.
|
||||
|
||||
The pipeline encodes identity-override attacks and legitimate
|
||||
corrections into statistically indistinguishable field-state
|
||||
geometries. No amount of axis-direction sharpening on the
|
||||
IdentityManifold can recover a signal that isn't present in the
|
||||
trajectory data being projected.
|
||||
|
||||
**Architectural conclusion:** fix #3 cannot be made load-bearing
|
||||
in place. The required upstream work — encoding token semantic
|
||||
categories into specific blade coordinates of the field versor at
|
||||
the ingest gate, then redefining the IdentityManifold axes in the
|
||||
32-dim Cl(4,1) basis with a real inner-product projection — is a
|
||||
scoped multi-PR effort, not a single sharpening exercise. The
|
||||
calibration probe stands as the empirical baseline that any future
|
||||
ingest-gate change must beat before fix #3 can be claimed
|
||||
load-bearing. See `evals/adversarial_identity/gaps.md` for the
|
||||
full table of measured signals and the recommended path.
|
||||
|
||||
**What stands today as the load-bearing defense:** fix #2
|
||||
(syntactic rules a/b/c/d) + the normalization layer reject 100% of
|
||||
v1–v5 attacks (n=121) with 0 false positives on 51 legitimate
|
||||
corrections. Fix #3's predicate, unit tests, and wiring remain as
|
||||
scaffolding for the upstream work above.
|
||||
|
||||
## Phase 2 — COMPLETE
|
||||
|
||||
All five Phase 2 v1+v2 lanes pass at 100%; frontier structural
|
||||
|
|
|
|||
147
evals/adversarial_identity/calibration/probe_field_signature.py
Normal file
147
evals/adversarial_identity/calibration/probe_field_signature.py
Normal file
|
|
@ -0,0 +1,147 @@
|
|||
"""Field-state signature probe for adversarial-identity attacks vs legitimates.
|
||||
|
||||
Run this script when designing or revisiting the IdentityManifold's axis
|
||||
directions for fix #3 (the geometric identity-override defense). It runs
|
||||
each adversarial-identity case through a fresh CognitiveTurnPipeline,
|
||||
captures the per-turn field-state delta and the existing identity_score,
|
||||
and reports per-coordinate and per-case discriminators between attacks
|
||||
and legitimates.
|
||||
|
||||
Result as of 2026-05-16 (recorded in `evals/adversarial_identity/gaps.md`):
|
||||
field-state geometry produced by today's ingest gate + vault grounding
|
||||
does NOT carry a discriminating signal between identity-override attacks
|
||||
and legitimate corrections. Per-case distributions overlap heavily;
|
||||
`identity_score.alignment` is 1.000 universally; mean-level coordinate
|
||||
differences are statistical artefacts of averaging, not per-case signals.
|
||||
|
||||
This script is preserved as the calibration baseline: any future change
|
||||
to the ingest gate, vocabulary grounding, or value-axis encoding should
|
||||
re-run this and demonstrate a per-case separation before claiming fix #3
|
||||
is load-bearing.
|
||||
|
||||
Usage:
|
||||
python3 evals/adversarial_identity/calibration/probe_field_signature.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
from chat.runtime import ChatRuntime
|
||||
from core.cognition.pipeline import CognitiveTurnPipeline
|
||||
|
||||
|
||||
_SEMANTIC_COORDS = (6, 7, 9, 10, 12, 14, 27)
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CaseSignature:
|
||||
case_id: str
|
||||
kind: str
|
||||
vault_hits: int
|
||||
identity_alignment: float
|
||||
delta_norm: float
|
||||
semantic_coord_energy_ratio: float
|
||||
surface_len: int
|
||||
|
||||
|
||||
def _load_cases(jsonl: Path) -> list[dict]:
|
||||
return [json.loads(line) for line in jsonl.read_text().splitlines() if line.strip()]
|
||||
|
||||
|
||||
def _signature(case: dict) -> CaseSignature | None:
|
||||
runtime = ChatRuntime()
|
||||
pipeline = CognitiveTurnPipeline(runtime)
|
||||
prior = case.get("prior", "")
|
||||
if prior:
|
||||
try:
|
||||
pipeline.run(prior, max_tokens=8)
|
||||
except ValueError:
|
||||
return None
|
||||
try:
|
||||
result = pipeline.run(case["attack"], max_tokens=8)
|
||||
except ValueError:
|
||||
return None
|
||||
if result.field_state_before is None or result.field_state_after is None:
|
||||
return None
|
||||
|
||||
f_before = result.field_state_before.F.astype(np.float64)
|
||||
f_after = result.field_state_after.F.astype(np.float64)
|
||||
delta = f_after - f_before
|
||||
|
||||
semantic_energy = float((delta[list(_SEMANTIC_COORDS)] ** 2).sum())
|
||||
total_energy = float((delta ** 2).sum()) + 1e-12
|
||||
return CaseSignature(
|
||||
case_id=str(case.get("id", "")),
|
||||
kind=str(case.get("kind", "")),
|
||||
vault_hits=int(result.vault_hits),
|
||||
identity_alignment=(
|
||||
float(result.identity_score.alignment) if result.identity_score else 1.0
|
||||
),
|
||||
delta_norm=float(np.linalg.norm(delta)),
|
||||
semantic_coord_energy_ratio=semantic_energy / total_energy,
|
||||
surface_len=len(result.surface or ""),
|
||||
)
|
||||
|
||||
|
||||
def _summarize(label: str, signatures: list[CaseSignature]) -> None:
|
||||
if not signatures:
|
||||
print(f"{label}: no signatures")
|
||||
return
|
||||
norms = np.array([s.delta_norm for s in signatures])
|
||||
ratios = np.array([s.semantic_coord_energy_ratio for s in signatures])
|
||||
aligns = np.array([s.identity_alignment for s in signatures])
|
||||
hits = np.array([s.vault_hits for s in signatures])
|
||||
print(
|
||||
f"{label:>30s} n={len(signatures):3d} "
|
||||
f"delta_norm: μ={norms.mean():.3f} σ={norms.std():.3f} "
|
||||
f"[{norms.min():.3f},{norms.max():.3f}] "
|
||||
f"sem_ratio: μ={ratios.mean():.3f} "
|
||||
f"align: μ={aligns.mean():.3f} min={aligns.min():.3f} "
|
||||
f"vault_hits: μ={hits.mean():.2f}"
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
splits = [
|
||||
("public/v3", "evals/adversarial_identity/public/v3/cases.jsonl"),
|
||||
("holdouts/v3", "evals/adversarial_identity/holdouts/v3/cases.jsonl"),
|
||||
("public/v5", "evals/adversarial_identity/public/v5/cases.jsonl"),
|
||||
("holdouts/v5", "evals/adversarial_identity/holdouts/v5/cases.jsonl"),
|
||||
]
|
||||
print("=" * 110)
|
||||
print("FIELD-STATE SIGNATURE PROBE — adversarial-identity attack vs legitimate")
|
||||
print("=" * 110)
|
||||
for split, path in splits:
|
||||
cases = _load_cases(_REPO_ROOT / path)
|
||||
attacks = [
|
||||
sig
|
||||
for c in cases
|
||||
if c["kind"] == "attack"
|
||||
for sig in [_signature(c)]
|
||||
if sig is not None
|
||||
]
|
||||
legits = [
|
||||
sig
|
||||
for c in cases
|
||||
if c["kind"] == "legitimate"
|
||||
for sig in [_signature(c)]
|
||||
if sig is not None
|
||||
]
|
||||
_summarize(f"{split} attacks", attacks)
|
||||
_summarize(f"{split} legitimates", legits)
|
||||
print("=" * 110)
|
||||
print(
|
||||
"Finding: per-case distributions overlap heavily; identity_score.alignment is\n"
|
||||
"1.000 universally across all kinds; no scalar derived from field-state geometry\n"
|
||||
"separates attack from legitimate at the per-case level. See gaps.md."
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -208,3 +208,61 @@ step (separate, scoped work) is to construct axis directions that
|
|||
actually separate identity-violating field deltas from legitimate
|
||||
correction deltas. Until that lands, the syntactic layer remains
|
||||
load-bearing.
|
||||
|
||||
## Architectural finding (2026-05-16) — fix #3 cannot be sharpened in place
|
||||
|
||||
A focused empirical investigation
|
||||
(`evals/adversarial_identity/calibration/probe_field_signature.py`)
|
||||
ran v3 and v5 cases through fresh pipelines and measured every
|
||||
candidate per-case discriminator that could be derived from the
|
||||
existing CognitiveTurnResult:
|
||||
|
||||
| Signal | Attack | Legitimate | Separable? |
|
||||
|---|---|---|---|
|
||||
| `identity_score.alignment` | 1.000 | 1.000 | No — identical |
|
||||
| field-delta L2 norm | μ≈3.4 (σ≈1.7) | μ≈3.9 (σ≈1.5) | No — heavy overlap |
|
||||
| semantic-coord energy ratio | μ≈0.88 | μ≈0.91 | No — overlap |
|
||||
| `vault_hits` | μ≈8.6 | μ≈7.9 | No — overlap |
|
||||
| `surface` length | non-empty | non-empty | No — both ground |
|
||||
| `intent.tag` | CORRECTION | CORRECTION | No — identical |
|
||||
|
||||
**The pipeline encodes identity-override attacks and legitimate
|
||||
corrections into statistically indistinguishable field-state
|
||||
geometries.** No amount of axis-direction sharpening on the
|
||||
IdentityManifold can recover a signal that isn't present in the
|
||||
trajectory data being projected. Per-case identity_score is
|
||||
literally a constant (1.000) for every input the runtime sees today.
|
||||
|
||||
### Required upstream work for fix #3 to become load-bearing
|
||||
|
||||
This is out of scope for the current effort and is recorded as the
|
||||
architectural follow-up:
|
||||
|
||||
1. **Ingest gate semantic encoding** (`ingest/gate.py`). Lift token
|
||||
semantic categories — redirect-verb-ness, role-frame-ness,
|
||||
self-reference, negating-qualifier presence — into specific blade
|
||||
coordinates of the field versor at injection time. Today the
|
||||
gate is purely lexical/grammatical and these categories vanish
|
||||
into a homogeneous coherence signal.
|
||||
2. **IdentityManifold axis directions in the multivector basis.**
|
||||
Once (1) lands, ValueAxis.direction should live in the 32-dim
|
||||
Cl(4,1) basis so the inner product against trajectory delta has
|
||||
physical meaning. Pre-compute the directions from the post-(1)
|
||||
pipeline's empirical signatures (re-run the calibration probe).
|
||||
3. **Replace `_axis_projection`** with a real inner-product
|
||||
projection of the trajectory delta onto axis directions, instead
|
||||
of the current scalar/coherence formula that produces 1.000
|
||||
alignment unconditionally.
|
||||
|
||||
### What stands today
|
||||
|
||||
- Fix #2 (syntactic) + normalization layer reject 100% of v1–v5
|
||||
attacks (n=121) with 0 false positives on 51 legitimate
|
||||
corrections. This is the load-bearing defense.
|
||||
- Fix #3's predicate `IdentityCheck.would_violate`, its unit tests,
|
||||
and its wiring through `CognitiveTurnPipeline._run_teaching` are
|
||||
in place as architectural scaffolding. When the upstream work
|
||||
above lands, the predicate becomes active without further wiring.
|
||||
- The calibration probe is preserved as the empirical baseline. Any
|
||||
future ingest-gate change must demonstrate per-case separation on
|
||||
this probe before fix #3 can be claimed as load-bearing.
|
||||
|
|
|
|||
Loading…
Reference in a new issue