Opus 4.8 audit of Fable's slice-1 scaffold + authorized hardening (Step 2).
Stacked on feat/adr-0246-slice1-scaffold. Not a PR, not merged, no status flip.
AUDIT (Step 1) — VERDICT PASS. Re-derived A(F)/d_orth/d_stab/typed channels from
§3.1/3.2/3.6 independently: BIT-EXACT vs impl (0.00e+00). Confirmed in code (not
just comments): H_id={I} locked (singleton hardcoded, no enlargement path); path
composes lawful only, refused=break marker, never raw, never soft-I; no §7 scope
creep; serve byte-identity. Finding F1 (doc, not bug): composition uses the raw
CERTIFIED action (required for §6.2 accumulation, else A_path≡I detects nothing) —
now documented for ADR ratification.
HARDENING (Step 2):
§3.7 pure surface (identity_action.py): AdmissionPolicy (calibrated flag),
evaluate_admission (admit-or-abstain, no corrector); CERTIFIED_GAMMA_ID pinned
== identity._WAVE_LEAKAGE_BOUND; all other bounds UNCERTIFIED placeholders.
§3.7 SERVE WIRING (Steps 2.1/2.2): new default-off flag identity_action_surface
(config); threaded chat/runtime -> check -> _wave_field_score; refusal folds
into flagged -> existing would_violate/conjugate_correct abstains; IdentityScore
gains action_surface_active/d_orth/d_stab (legacy defaults). Flag-off is
byte-identical (D4 gate surfaces green unchanged; smoke 176 post-wiring).
§6.3 DISCRIMINATION REPORT (evals/adr_0246_discrimination) — HONEST numbers:
benign pass 0.00, false refusal 1.00, adversarial detect 1.00, control pass
1.00; d_stab AUC 0.375 (95% CI [0.15,0.62]) — BELOW chance. Benign cognition
sits ~18x farther from the frame (mean d_stab 27.8) than the attacks (1.55).
The gate refuses everything and does NOT discriminate; must stay off; usable
separation needs the §11 grounding work, not threshold tuning. Claims language
enforced: 'lawfulness relative to the declared frozen frame', NOT 'semantic
inalienability'.
Ledger raw-sneak hardening test (Step 2.3): mixed lawful/refused sequence must
equal the lawful sub-product, fails if raw sneaks into A_path_lawful.
Handoff to Sonnet: §4.1 per-turn record, path-ledger serve integration, ADR-0246
body + acceptance packet (Proposed, no self-Accept), §11 grounding-feasibility.
[Verification]: uv run core test --suite smoke -q => 176 passed (post-wiring);
ADR-0246 suites 80 passed; egress wiring + D4 gate surfaces 47 passed
(byte-identity); §6.1/6.2 eval 14/14; discrimination report numbers above.
See docs/audit/adr-0246-slice1-opus-audit-and-hardening.md + run log.
124 lines
4.7 KiB
Python
124 lines
4.7 KiB
Python
"""ADR-0246 §3.7 admit-surface + §6.3 discrimination-report pins.
|
|
|
|
Pins the pure admit surface (`evaluate_admission`, locked `H_id={I}`, placeholder
|
|
thresholds) and the honest discrimination verdict: on the declared placeholder
|
|
frame the gate refuses benign and adversarial alike and does NOT separate them —
|
|
a result that must be reported plainly, never framed as a working detector.
|
|
Offline/deterministic: cohorts are injected, so no runtime is spun up here.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import numpy as np
|
|
import pytest
|
|
|
|
from algebra.cl41 import N_COMPONENTS
|
|
from core.physics import identity
|
|
from core.physics.identity_manifold import IdentityManifoldGeometry, MalformedVersorError
|
|
from core.physics.identity_action import (
|
|
AdmissionPolicy,
|
|
CERTIFIED_GAMMA_ID,
|
|
evaluate_admission,
|
|
)
|
|
from evals.adr_0246_discrimination import build_discrimination_report
|
|
|
|
_E12, _E14, _E15 = 6, 8, 9
|
|
|
|
|
|
def _rotor(biv, theta):
|
|
r = np.zeros(N_COMPONENTS, dtype=np.float64)
|
|
r[0] = np.cos(theta / 2.0)
|
|
r[biv] = np.sin(theta / 2.0)
|
|
return r
|
|
|
|
|
|
def _boost(biv, theta):
|
|
r = np.zeros(N_COMPONENTS, dtype=np.float64)
|
|
r[0] = np.cosh(theta / 2.0)
|
|
r[biv] = np.sinh(theta / 2.0)
|
|
return r
|
|
|
|
|
|
def _identity_versor():
|
|
v = np.zeros(N_COMPONENTS, dtype=np.float64)
|
|
v[0] = 1.0
|
|
return v
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def geometry():
|
|
return IdentityManifoldGeometry.from_directions(
|
|
((1.0, 0.0, 0.0), (0.0, 1.0, 0.0), (0.0, 0.0, 1.0))
|
|
)
|
|
|
|
|
|
def test_certified_gamma_id_matches_d4_bound_no_drift():
|
|
# the one certified threshold must equal the D4-pinned serve bound
|
|
assert CERTIFIED_GAMMA_ID == identity._WAVE_LEAKAGE_BOUND
|
|
|
|
|
|
def test_placeholder_policy_is_flagged_uncalibrated():
|
|
assert AdmissionPolicy.placeholder_default().calibrated is False
|
|
|
|
|
|
def test_identity_versor_is_admitted(geometry):
|
|
result = evaluate_admission(geometry, _identity_versor(), AdmissionPolicy.placeholder_default())
|
|
assert result.admitted is True
|
|
assert result.refusal_reasons == ()
|
|
assert result.d_orth < 1e-9 and result.d_stab < 1e-9
|
|
|
|
|
|
@pytest.mark.parametrize("versor", [_rotor(_E14, 1.5), _boost(_E15, 1.2), _rotor(_E12, np.pi)])
|
|
def test_attacks_are_refused_with_reasons(geometry, versor):
|
|
result = evaluate_admission(geometry, versor, AdmissionPolicy.placeholder_default())
|
|
assert result.admitted is False
|
|
assert len(result.refusal_reasons) >= 1
|
|
|
|
|
|
def test_admission_is_admit_or_abstain_never_corrects(geometry):
|
|
# evaluate_admission returns a verdict + measurements; it never returns a
|
|
# modified versor/action (no corrector surface exists)
|
|
result = evaluate_admission(geometry, _rotor(_E14, 1.0), AdmissionPolicy.placeholder_default())
|
|
assert set(result.as_dict()) == {
|
|
"admitted", "refusal_reasons", "d_orth", "d_stab",
|
|
"leakage_rms", "max_leakage", "min_self_alignment", "typed_channels",
|
|
}
|
|
|
|
|
|
def test_malformed_versor_raises_for_failclosed_serve(geometry):
|
|
bad = _identity_versor()
|
|
bad[3] = np.nan
|
|
with pytest.raises(MalformedVersorError):
|
|
evaluate_admission(geometry, bad, AdmissionPolicy.placeholder_default())
|
|
|
|
|
|
def test_discrimination_report_reports_honest_non_separation(geometry):
|
|
# inject a benign cohort that mimics REAL benign traffic (far from the frame,
|
|
# per D4/slice-0) so the honest verdict is pinned without a live runtime.
|
|
benign = [
|
|
("benign_like_boost", _boost(_E15, 1.1)),
|
|
("benign_like_boost2", _boost(9, 1.3)),
|
|
("benign_like_tilt", _rotor(_E14, 1.2)),
|
|
("benign_like_big", _rotor(_E12, 2.5)),
|
|
]
|
|
report = build_discrimination_report(benign, geometry=geometry)
|
|
assert report["policy"]["calibrated"] is False
|
|
# benign mass-refused; a refuse-all "detects" all attacks but does not discriminate
|
|
assert report["rates"]["benign_pass_rate"] == 0.0
|
|
assert report["rates"]["false_refusal_rate"] == 1.0
|
|
assert report["rates"]["adversarial_detection_rate"] == 1.0
|
|
assert report["verdict"]["gate_discriminates_benign_from_adversarial"] is False
|
|
assert report["verdict"]["benign_usable_at_this_policy"] is False
|
|
# the honest claims language must be present and must NOT oversell
|
|
claims = report["verdict"]["claims_language"].lower()
|
|
assert "lawfulness relative to the declared frozen frame" in claims
|
|
assert "inalienab" in claims # explicitly names what it is NOT
|
|
|
|
|
|
def test_discrimination_control_admits_true_near_identity(geometry):
|
|
# the synthetic-near-identity control passing confirms the gate MECHANISM is
|
|
# sound — the benign failure is the frame, not a broken gate.
|
|
report = build_discrimination_report(
|
|
[("benign_like", _boost(_E15, 1.1))], geometry=geometry
|
|
)
|
|
assert report["rates"]["synthetic_near_identity_pass_rate"] == 1.0
|