core/tests/test_frame_verdict.py
Shay ba05ebb0cd
feat(frame-verdict): closed-world FrameVerdict substrate — PR-1..4 + hardening (ADR-0222 B4) (#787)
* wip(b4): lift _basis to generate/epistemic_basis.py (behavior-preserving)

B4 PR-1 foundation checkpoint — the shared epistemic-basis helper extracted
byte-identically from determine.py so the closed-world frame_verdict evaluator
(coming) can compute standing without importing determine (ADR-0222 §8 A3).
determine.py imports it aliased; its 3 Determined sites + INV-30 unchanged.
PAUSED: rest of B4 (FrameVerdict type, evaluator, INV-31, PR-2/3/4) resumes
after the suite-speed arc.

* feat(frame-verdict): closed-world envelope + isolated evaluator + INV-31 (B4 PR-1)

The sealed, off-serving FrameVerdict type + the isolated text-frame evaluator + the
INV-31 firewall (ADR-0222). Composes proof_chain.entail (no second prover); a
closed-world False comes ONLY from an ROBDD refutation; absence is never false.

- generate/frame_verdict/{types,evaluate,__init__}: FrameVerdict / ClosedFrame /
  ClosedWorldProof + PositiveRefutationKind enum. evaluate_frame_verdict(frame, query)
  maps entail ENTAILED/REFUTED/UNKNOWN/REFUSED -> entailed_true/false/undetermined/
  contradiction/scope_boundary; OPEN / undeclared-closure / non-TEXT -> scope_boundary.
  __post_init__ admissibility: entailed_false needs a NAMED positive refutation; a
  generic FALSIFIED raises. trace_hash is order-invariant + replay-stable.
- INV-31 (test_architectural_invariants.py): A1 determine.py clean+visible; A2 exact
  construction allowlist (evals-inclusive, tests-excluded) + non-vacuity anchor; A3
  transitive spine -/-> frame_verdict (reuses INV-27 walker) + resolve anchor; B1
  determine() refuses a ClosedFrame. B2 deferred to the governance slice (no
  non-vacuous surface yet; ADR §8 permits).

No serving wire. determine.py unchanged except the _basis import. No Determined(answer=False).
INV-30 green. Shapes follow the B4 operator master brief (PositiveRefutationKind enum,
closure_declared/source/provenance) — a consistent refinement of ADR-0222 §3.

Verified: 97 (full INV incl. INV-31 + FrameVerdict + OWA floor) + 20 frame_verdict + 35 determine.

* feat(frame-verdict): text closed-world (CWA) evaluation lane (B4 PR-2)

A measure-only lane over the text FrameVerdict evaluator (ADR-0222). Proves the sealed
type evaluates propositional closed frames safely — incl. a sound entailed_false —
without touching determine() or runtime serving.

- evals/frame_verdict_text_cwa/: cases.jsonl (12 cases across all 5 verdict kinds +
  gating + absence safety), oracle.py (an INDEPENDENT truth-table propositional checker —
  own recursive-descent parser + brute-force enumeration, disjoint from the ROBDD; imports
  no engine module), score.py, README.
- tests/test_frame_verdict_text_cwa_lane.py: wrong=0; the disjoint oracle confirms every
  gold (non-vacuity) AND the engine matches the oracle on every case; entailed_false is
  proof-backed (ROBDD_REFUTATION); absence / OPEN / undeclared-closure are never
  entailed_false; SHA-pinned fixtures.

Input contract: propositional-formula strings (no prose lowering). Capability-index
deliberately NOT touched (off-serving lane; documented). No Determined, no answer=False,
no serving wire. INV-30 / INV-31 / ProofWriter-OWA stay green.

* feat(frame-verdict): perception changed-slot falsification adapter (B4 PR-3)

Lift an ADR-0211 FalsificationRun into a FrameVerdict SAFELY (ADR-0222 §5.2). Critical
doctrine: "FALSIFIED" is not enough — only a POSITIVELY observed changed-slot contradiction
produces entailed_false (PERCEPTION_CHANGED_SLOT). Missing observation (absence), unexpected
extra (over-observation), and a whole-missing actual frame NEVER become false — they refuse
(undetermined / scope_boundary). SUPPORTED -> entailed_true (frame-conformance proven).

- generate/frame_verdict/perception_adapter.py: frame_verdict_from_perception_falsification;
  the proof carries the FULL run trace_hash (binds expected+actual+verdict, ADR §5.4).
- generate/frame_verdict/_construct.py: extracted the single FrameVerdict builder; the text
  evaluator + perception adapter both funnel through it, so the literal FrameVerdict(...)
  lives in ONE file -> INV-31 ALLOWED_FRAME_VERDICT_SITES = {_construct.py}.
- tests: changed-slot -> entailed_false (+ empty-hash raises); missing/unexpected/whole-missing
  never false; supported -> entailed_true; non-perception -> scope_boundary.

Off-serving; perception_adapter is in generate.frame_verdict, so INV-31 A3 already proves it
unreachable from the open-world spine. No determine() change, no answer=False. INV-30 / INV-31
/ ProofWriter-OWA / text-CWA lane stay green.

* feat(response-governance): default-dark FrameVerdict surface mapping (B4 PR-4)

The only lawful surface path for a closed-world verdict (ADR-0222 §7/§14). Lowers a
FrameVerdict to a served disposition through the EXISTING epistemic_disclosure tables (no
parallel object): entailed_true/false -> COMMIT at INFERRED + DisclosureClaim.NONE (a
committed "Yes"/"No"; entailed_false is an answer, NOT a contradiction/refusal/LimitationKind);
contradiction -> REPORT; undetermined -> REFUSE; scope_boundary -> EXPLAIN.

DEFAULT-DARK: a NEW module that changes no existing file — the open-world govern_response /
shape_surface STRICT path is byte-identical, and nothing in the live runtime calls it. The
TYPE is the closed-world tag: a forged dict / untagged object cannot widen serving (raises).

INV-31: A3 still green — core/response_governance/frame_verdict.py is NOT imported by the
package __init__ or the open-world spine, so frame_verdict stays unreachable from
chat/runtime/session/vault. The B2 anchor is now live (the forged-object rejection); all six
INV-31 anchors firm. The open-world render_determination cannot render a FrameVerdict;
determine() still has no answer=False.

Verified: 84 (governance + full INV incl. INV-31 A3/B2) green.

* harden(frame-verdict): fold in adversarial-review findings (B4)

A 4-skeptic adversarial read of PR-1..4 against real source surfaced one
soundness gap (major) and four defensive gaps (minor). All folded in; the
4 B4 files + full INV suite stay green (128 passed).

S1 (major) — perception negation was not frame-gated. The text evaluator
refuses OPEN / undeclared-closure frames (-> SCOPE_BOUNDARY) but the
perception adapter only gated frame_kind + the missing-frame sentinel, so a
PERCEPTION + OPEN + changed-slot residual produced entailed_false with
world_assumption=OPEN — a verdict that self-contradicts the type invariant
(OPEN => negation illegal). Fixed two ways:
  * perception_adapter: gate OPEN / not-closure_declared -> SCOPE_BOUNDARY,
    mirroring the text evaluator (graceful upstream refusal);
  * types.__post_init__ §(0): STRUCTURAL backstop — ENTAILED_FALSE + OPEN
    raises, frame-general, so NO producer (text/perception/future) can emit
    an OPEN-world negation even if it forgets the gate.

S4-a — entailed_true was admissibility-asymmetric. Only entailed_false was
proof-gated at construction; a committed "Yes." leaned entirely on the
INV-31-A2 allowlist. Added a symmetric §(2) guard: entailed_true requires a
positive entailment/support proof (proof_chain.entail/ENTAILED or
sensorium.falsification/SUPPORTED), non-empty sha, and NO refutation kind —
a mutation test can now trip a forged positive.

S2 — A2 construction detector extended to flag FrameVerdict.<factory>(...)
classmethod constructions and module-qualified mod.FrameVerdict(...), so a
future alternate constructor cannot evade ALLOWED_FRAME_VERDICT_SITES. The
dataclasses.replace gap is RECORDED (a blanket replace() match would
false-positive tree-wide) for the serving-wiring PR, not faked.

S4-b/c — A3 firewall now bars core.response_governance.frame_verdict
DIRECTLY (not only via its re-import of generate.frame_verdict), plus a new
test proving the response_governance package __init__ stays default-dark
(does not re-export the adapter).

S3 — oracle grammar is a strict SUBSET of proof_chain.entail (it mis-parses
`false` as a free atom). Corrected the docstring and added a lane guard:
every DECIDED cases.jsonl formula must stay inside the subset, so a future
out-of-subset case fails loudly at SHA-add review instead of as a confusing
engine-vs-oracle red. SCOPE_BOUNDARY-gold garbage is exempt (both refuse).

S1-minor (construction-time-only admissibility) documented in __post_init__:
a future codec/deserialization path must re-construct via build_frame_verdict.

* docs(analysis): B4 FrameVerdict implementation lookback (PR-1..4 + hardening)

Mandatory multi-slice lookback (CLAUDE.md triggers 2+3). Audits all six B4
commits across the lookback template: documentation drift, test-coverage /
parity gaps, wrong=0 hazard surface, cross-slice consistency, honest LOC.

Categorized findings: solid (INV-30 untouched; entailed_false positive-only;
INV-31 firewall non-vacuous; default-dark; replay determinism) / hazards (the
5 adversarial-review findings, ALL fixed in the hardening commit) / recorded
gaps (dataclasses.replace A2 hole + construction-time-only admissibility,
both deferred with an explicit guard obligation on the serving-wiring PR) /
drift (type shapes are a consistent refinement of ADR-0222 §3; one non-gating
ADR note filed). Global red-line ledger: all green.
2026-06-16 06:23:03 -07:00

246 lines
9.5 KiB
Python

"""FrameVerdict — the sealed closed-world type + isolated text-frame evaluator (B4 PR-1).
Covers the ADR-0222 §5.1 verdict mapping, the frame gating (OPEN / undeclared / non-TEXT
=> SCOPE_BOUNDARY), the admissibility invariant (entailed_false needs a NAMED positive
refutation), absence safety (no fact => UNDETERMINED, never entailed_false), and replay
determinism (order-invariant + repeat-stable trace_hash). INV-31 import/data-flow
containment lives in ``tests/test_architectural_invariants.py``.
"""
from __future__ import annotations
import dataclasses
import pytest
from generate.frame_verdict import (
ClosedFrame,
ClosedWorldProof,
FrameKind,
FrameVerdict,
FrameVerdictKind,
PositiveRefutationKind,
WorldAssumption,
evaluate_frame_verdict,
)
def _frame(
propositions,
*,
closure: bool = True,
kind: FrameKind = FrameKind.TEXT,
wa: WorldAssumption = WorldAssumption.CLOSED,
) -> ClosedFrame:
return ClosedFrame(
frame_id="f1",
frame_kind=kind,
world_assumption=wa,
propositions=tuple(propositions),
closure_declared=closure,
source="test",
provenance=(),
)
# --------------------------------------------------------------------------- #
# §5.1 verdict mapping
# --------------------------------------------------------------------------- #
def test_entailed_true_from_closed_text_frame() -> None:
v = evaluate_frame_verdict(_frame(["a", "a -> b"]), "b")
assert v.verdict is FrameVerdictKind.ENTAILED_TRUE
assert v.proof.producer == "proof_chain.entail" and v.proof.outcome == "ENTAILED"
assert v.proof.positive_refutation_kind is None
assert v.basis == "as_told"
def test_entailed_false_from_refuted_carries_robdd_refutation() -> None:
v = evaluate_frame_verdict(_frame(["a", "a -> ~b"]), "b")
assert v.verdict is FrameVerdictKind.ENTAILED_FALSE
assert v.proof.producer == "proof_chain.entail" and v.proof.outcome == "REFUTED"
assert v.proof.positive_refutation_kind is PositiveRefutationKind.ROBDD_REFUTATION
assert v.proof.proof_sha256 # non-empty positive proof
def test_unknown_maps_to_undetermined() -> None:
v = evaluate_frame_verdict(_frame(["a"]), "b")
assert v.verdict is FrameVerdictKind.UNDETERMINED
def test_inconsistent_premises_map_to_contradiction_not_false() -> None:
v = evaluate_frame_verdict(_frame(["a", "~a"]), "b")
assert v.verdict is FrameVerdictKind.CONTRADICTION
assert v.verdict is not FrameVerdictKind.ENTAILED_FALSE
def test_malformed_maps_to_scope_boundary() -> None:
v = evaluate_frame_verdict(_frame(["@@ not grammar ???"]), "b")
assert v.verdict is FrameVerdictKind.SCOPE_BOUNDARY
# --------------------------------------------------------------------------- #
# Frame gating — a negation needs an explicit declared-complete TEXT frame
# --------------------------------------------------------------------------- #
def test_open_world_assumption_refuses() -> None:
v = evaluate_frame_verdict(_frame(["a", "a -> ~b"], wa=WorldAssumption.OPEN), "b")
assert v.verdict is FrameVerdictKind.SCOPE_BOUNDARY
def test_undeclared_closure_refuses() -> None:
v = evaluate_frame_verdict(_frame(["a", "a -> ~b"], closure=False), "b")
assert v.verdict is FrameVerdictKind.SCOPE_BOUNDARY
def test_perception_frame_is_scope_boundary_in_pr1() -> None:
v = evaluate_frame_verdict(_frame(["a", "a -> ~b"], kind=FrameKind.PERCEPTION), "b")
assert v.verdict is FrameVerdictKind.SCOPE_BOUNDARY
# --------------------------------------------------------------------------- #
# Admissibility invariant — entailed_false needs a NAMED positive refutation
# --------------------------------------------------------------------------- #
def _proof(producer, outcome, kind, sha="abc123") -> ClosedWorldProof:
return ClosedWorldProof(
producer=producer, outcome=outcome, proof_sha256=sha,
proof_keys=("k",), positive_refutation_kind=kind,
)
def _make(verdict, proof) -> FrameVerdict:
return FrameVerdict(
frame_id="f1", frame_kind=FrameKind.TEXT,
world_assumption=WorldAssumption.CLOSED, query="b", verdict=verdict,
proof=proof, basis="as_told", trace_hash="th", provenance=(),
)
def test_entailed_false_with_generic_falsified_raises() -> None:
# generic FALSIFIED (no positive_refutation_kind) cannot prove false.
with pytest.raises(ValueError):
_make(FrameVerdictKind.ENTAILED_FALSE, _proof("sensorium.falsification", "FALSIFIED", None))
def test_entailed_false_with_missing_proof_sha_raises() -> None:
with pytest.raises(ValueError):
_make(
FrameVerdictKind.ENTAILED_FALSE,
_proof("proof_chain.entail", "REFUTED", PositiveRefutationKind.ROBDD_REFUTATION, sha=""),
)
def test_entailed_false_with_mismatched_kind_raises() -> None:
# REFUTED text outcome but the PERCEPTION kind — mismatch must raise.
with pytest.raises(ValueError):
_make(
FrameVerdictKind.ENTAILED_FALSE,
_proof("proof_chain.entail", "REFUTED", PositiveRefutationKind.PERCEPTION_CHANGED_SLOT),
)
def test_valid_text_entailed_false_constructs() -> None:
v = _make(
FrameVerdictKind.ENTAILED_FALSE,
_proof("proof_chain.entail", "REFUTED", PositiveRefutationKind.ROBDD_REFUTATION),
)
assert v.verdict is FrameVerdictKind.ENTAILED_FALSE
def test_entailed_false_in_open_world_raises() -> None:
# Structural backstop (types.py §(0)): OPEN + entailed_false is illegal even WITH a valid
# ROBDD refutation proof — absence is never false. No producer may emit an OPEN negation.
with pytest.raises(ValueError):
FrameVerdict(
frame_id="f1", frame_kind=FrameKind.TEXT, world_assumption=WorldAssumption.OPEN,
query="b", verdict=FrameVerdictKind.ENTAILED_FALSE,
proof=_proof("proof_chain.entail", "REFUTED", PositiveRefutationKind.ROBDD_REFUTATION),
basis="as_told", trace_hash="th", provenance=(),
)
@pytest.mark.parametrize(
"kind",
[
FrameVerdictKind.UNDETERMINED,
FrameVerdictKind.CONTRADICTION,
FrameVerdictKind.SCOPE_BOUNDARY,
],
)
def test_refusal_verdicts_need_no_positive_proof(kind) -> None:
# a proofless / None-kind proof is fine for every refusal/undetermined/contradiction verdict
# (only the two COMMITTED verdicts — entailed_true / entailed_false — are proof-gated).
v = _make(kind, _proof("proof_chain.entail", "UNKNOWN", None, sha=""))
assert v.verdict is kind
# --------------------------------------------------------------------------- #
# Symmetric admissibility — entailed_true needs a positive entailment/support proof
# --------------------------------------------------------------------------- #
def test_valid_text_entailed_true_constructs() -> None:
v = _make(FrameVerdictKind.ENTAILED_TRUE, _proof("proof_chain.entail", "ENTAILED", None))
assert v.verdict is FrameVerdictKind.ENTAILED_TRUE
def test_valid_perception_entailed_true_constructs() -> None:
v = _make(FrameVerdictKind.ENTAILED_TRUE, _proof("sensorium.falsification", "SUPPORTED", None))
assert v.verdict is FrameVerdictKind.ENTAILED_TRUE
def test_entailed_true_with_wrong_outcome_raises() -> None:
# an UNKNOWN/REFUTED outcome cannot back a committed "Yes." — symmetric with entailed_false.
with pytest.raises(ValueError):
_make(FrameVerdictKind.ENTAILED_TRUE, _proof("proof_chain.entail", "UNKNOWN", None))
def test_entailed_true_with_missing_proof_sha_raises() -> None:
with pytest.raises(ValueError):
_make(FrameVerdictKind.ENTAILED_TRUE, _proof("proof_chain.entail", "ENTAILED", None, sha=""))
def test_entailed_true_with_a_refutation_kind_raises() -> None:
# a "true" carrying a positive_refutation_kind is malformed (refutation kinds prove FALSE).
with pytest.raises(ValueError):
_make(
FrameVerdictKind.ENTAILED_TRUE,
_proof("proof_chain.entail", "ENTAILED", PositiveRefutationKind.ROBDD_REFUTATION),
)
# --------------------------------------------------------------------------- #
# Absence safety + determinism + distinctness
# --------------------------------------------------------------------------- #
def test_absence_never_yields_entailed_false() -> None:
# a closed frame with no fact bearing on b: UNDETERMINED, never entailed_false.
v = evaluate_frame_verdict(_frame(["a", "c -> d"]), "b")
assert v.verdict is not FrameVerdictKind.ENTAILED_FALSE
assert v.verdict is FrameVerdictKind.UNDETERMINED
def test_premise_reordering_keeps_trace_hash_stable() -> None:
a = evaluate_frame_verdict(_frame(["a", "a -> b", "b -> c"]), "c")
b = evaluate_frame_verdict(_frame(["b -> c", "a -> b", "a"]), "c")
assert a.verdict is b.verdict is FrameVerdictKind.ENTAILED_TRUE
assert a.trace_hash == b.trace_hash and a.proof.proof_sha256 == b.proof.proof_sha256
def test_repeated_evaluation_keeps_trace_hash_stable() -> None:
a = evaluate_frame_verdict(_frame(["a", "a -> ~b"]), "b")
b = evaluate_frame_verdict(_frame(["a", "a -> ~b"]), "b")
assert a.trace_hash == b.trace_hash and a.proof.proof_sha256 == b.proof.proof_sha256
def test_frame_verdict_is_distinct_from_determined() -> None:
# no `answer` field, a 5-way verdict enum — cannot be confused with an open-world answer.
field_names = {f.name for f in dataclasses.fields(FrameVerdict)}
assert "answer" not in field_names
assert "verdict" in field_names
assert FrameVerdict.__name__ == "FrameVerdict" # distinct call-name from Determined