* wip(b4): lift _basis to generate/epistemic_basis.py (behavior-preserving)
B4 PR-1 foundation checkpoint — the shared epistemic-basis helper extracted
byte-identically from determine.py so the closed-world frame_verdict evaluator
(coming) can compute standing without importing determine (ADR-0222 §8 A3).
determine.py imports it aliased; its 3 Determined sites + INV-30 unchanged.
PAUSED: rest of B4 (FrameVerdict type, evaluator, INV-31, PR-2/3/4) resumes
after the suite-speed arc.
* feat(frame-verdict): closed-world envelope + isolated evaluator + INV-31 (B4 PR-1)
The sealed, off-serving FrameVerdict type + the isolated text-frame evaluator + the
INV-31 firewall (ADR-0222). Composes proof_chain.entail (no second prover); a
closed-world False comes ONLY from an ROBDD refutation; absence is never false.
- generate/frame_verdict/{types,evaluate,__init__}: FrameVerdict / ClosedFrame /
ClosedWorldProof + PositiveRefutationKind enum. evaluate_frame_verdict(frame, query)
maps entail ENTAILED/REFUTED/UNKNOWN/REFUSED -> entailed_true/false/undetermined/
contradiction/scope_boundary; OPEN / undeclared-closure / non-TEXT -> scope_boundary.
__post_init__ admissibility: entailed_false needs a NAMED positive refutation; a
generic FALSIFIED raises. trace_hash is order-invariant + replay-stable.
- INV-31 (test_architectural_invariants.py): A1 determine.py clean+visible; A2 exact
construction allowlist (evals-inclusive, tests-excluded) + non-vacuity anchor; A3
transitive spine -/-> frame_verdict (reuses INV-27 walker) + resolve anchor; B1
determine() refuses a ClosedFrame. B2 deferred to the governance slice (no
non-vacuous surface yet; ADR §8 permits).
No serving wire. determine.py unchanged except the _basis import. No Determined(answer=False).
INV-30 green. Shapes follow the B4 operator master brief (PositiveRefutationKind enum,
closure_declared/source/provenance) — a consistent refinement of ADR-0222 §3.
Verified: 97 (full INV incl. INV-31 + FrameVerdict + OWA floor) + 20 frame_verdict + 35 determine.
* feat(frame-verdict): text closed-world (CWA) evaluation lane (B4 PR-2)
A measure-only lane over the text FrameVerdict evaluator (ADR-0222). Proves the sealed
type evaluates propositional closed frames safely — incl. a sound entailed_false —
without touching determine() or runtime serving.
- evals/frame_verdict_text_cwa/: cases.jsonl (12 cases across all 5 verdict kinds +
gating + absence safety), oracle.py (an INDEPENDENT truth-table propositional checker —
own recursive-descent parser + brute-force enumeration, disjoint from the ROBDD; imports
no engine module), score.py, README.
- tests/test_frame_verdict_text_cwa_lane.py: wrong=0; the disjoint oracle confirms every
gold (non-vacuity) AND the engine matches the oracle on every case; entailed_false is
proof-backed (ROBDD_REFUTATION); absence / OPEN / undeclared-closure are never
entailed_false; SHA-pinned fixtures.
Input contract: propositional-formula strings (no prose lowering). Capability-index
deliberately NOT touched (off-serving lane; documented). No Determined, no answer=False,
no serving wire. INV-30 / INV-31 / ProofWriter-OWA stay green.
* feat(frame-verdict): perception changed-slot falsification adapter (B4 PR-3)
Lift an ADR-0211 FalsificationRun into a FrameVerdict SAFELY (ADR-0222 §5.2). Critical
doctrine: "FALSIFIED" is not enough — only a POSITIVELY observed changed-slot contradiction
produces entailed_false (PERCEPTION_CHANGED_SLOT). Missing observation (absence), unexpected
extra (over-observation), and a whole-missing actual frame NEVER become false — they refuse
(undetermined / scope_boundary). SUPPORTED -> entailed_true (frame-conformance proven).
- generate/frame_verdict/perception_adapter.py: frame_verdict_from_perception_falsification;
the proof carries the FULL run trace_hash (binds expected+actual+verdict, ADR §5.4).
- generate/frame_verdict/_construct.py: extracted the single FrameVerdict builder; the text
evaluator + perception adapter both funnel through it, so the literal FrameVerdict(...)
lives in ONE file -> INV-31 ALLOWED_FRAME_VERDICT_SITES = {_construct.py}.
- tests: changed-slot -> entailed_false (+ empty-hash raises); missing/unexpected/whole-missing
never false; supported -> entailed_true; non-perception -> scope_boundary.
Off-serving; perception_adapter is in generate.frame_verdict, so INV-31 A3 already proves it
unreachable from the open-world spine. No determine() change, no answer=False. INV-30 / INV-31
/ ProofWriter-OWA / text-CWA lane stay green.
* feat(response-governance): default-dark FrameVerdict surface mapping (B4 PR-4)
The only lawful surface path for a closed-world verdict (ADR-0222 §7/§14). Lowers a
FrameVerdict to a served disposition through the EXISTING epistemic_disclosure tables (no
parallel object): entailed_true/false -> COMMIT at INFERRED + DisclosureClaim.NONE (a
committed "Yes"/"No"; entailed_false is an answer, NOT a contradiction/refusal/LimitationKind);
contradiction -> REPORT; undetermined -> REFUSE; scope_boundary -> EXPLAIN.
DEFAULT-DARK: a NEW module that changes no existing file — the open-world govern_response /
shape_surface STRICT path is byte-identical, and nothing in the live runtime calls it. The
TYPE is the closed-world tag: a forged dict / untagged object cannot widen serving (raises).
INV-31: A3 still green — core/response_governance/frame_verdict.py is NOT imported by the
package __init__ or the open-world spine, so frame_verdict stays unreachable from
chat/runtime/session/vault. The B2 anchor is now live (the forged-object rejection); all six
INV-31 anchors firm. The open-world render_determination cannot render a FrameVerdict;
determine() still has no answer=False.
Verified: 84 (governance + full INV incl. INV-31 A3/B2) green.
* harden(frame-verdict): fold in adversarial-review findings (B4)
A 4-skeptic adversarial read of PR-1..4 against real source surfaced one
soundness gap (major) and four defensive gaps (minor). All folded in; the
4 B4 files + full INV suite stay green (128 passed).
S1 (major) — perception negation was not frame-gated. The text evaluator
refuses OPEN / undeclared-closure frames (-> SCOPE_BOUNDARY) but the
perception adapter only gated frame_kind + the missing-frame sentinel, so a
PERCEPTION + OPEN + changed-slot residual produced entailed_false with
world_assumption=OPEN — a verdict that self-contradicts the type invariant
(OPEN => negation illegal). Fixed two ways:
* perception_adapter: gate OPEN / not-closure_declared -> SCOPE_BOUNDARY,
mirroring the text evaluator (graceful upstream refusal);
* types.__post_init__ §(0): STRUCTURAL backstop — ENTAILED_FALSE + OPEN
raises, frame-general, so NO producer (text/perception/future) can emit
an OPEN-world negation even if it forgets the gate.
S4-a — entailed_true was admissibility-asymmetric. Only entailed_false was
proof-gated at construction; a committed "Yes." leaned entirely on the
INV-31-A2 allowlist. Added a symmetric §(2) guard: entailed_true requires a
positive entailment/support proof (proof_chain.entail/ENTAILED or
sensorium.falsification/SUPPORTED), non-empty sha, and NO refutation kind —
a mutation test can now trip a forged positive.
S2 — A2 construction detector extended to flag FrameVerdict.<factory>(...)
classmethod constructions and module-qualified mod.FrameVerdict(...), so a
future alternate constructor cannot evade ALLOWED_FRAME_VERDICT_SITES. The
dataclasses.replace gap is RECORDED (a blanket replace() match would
false-positive tree-wide) for the serving-wiring PR, not faked.
S4-b/c — A3 firewall now bars core.response_governance.frame_verdict
DIRECTLY (not only via its re-import of generate.frame_verdict), plus a new
test proving the response_governance package __init__ stays default-dark
(does not re-export the adapter).
S3 — oracle grammar is a strict SUBSET of proof_chain.entail (it mis-parses
`false` as a free atom). Corrected the docstring and added a lane guard:
every DECIDED cases.jsonl formula must stay inside the subset, so a future
out-of-subset case fails loudly at SHA-add review instead of as a confusing
engine-vs-oracle red. SCOPE_BOUNDARY-gold garbage is exempt (both refuse).
S1-minor (construction-time-only admissibility) documented in __post_init__:
a future codec/deserialization path must re-construct via build_frame_verdict.
* docs(analysis): B4 FrameVerdict implementation lookback (PR-1..4 + hardening)
Mandatory multi-slice lookback (CLAUDE.md triggers 2+3). Audits all six B4
commits across the lookback template: documentation drift, test-coverage /
parity gaps, wrong=0 hazard surface, cross-slice consistency, honest LOC.
Categorized findings: solid (INV-30 untouched; entailed_false positive-only;
INV-31 firewall non-vacuous; default-dark; replay determinism) / hazards (the
5 adversarial-review findings, ALL fixed in the hardening commit) / recorded
gaps (dataclasses.replace A2 hole + construction-time-only admissibility,
both deferred with an explicit guard obligation on the serving-wiring PR) /
drift (type shapes are a consistent refinement of ADR-0222 §3; one non-gating
ADR note filed). Global red-line ledger: all green.
246 lines
9.5 KiB
Python
246 lines
9.5 KiB
Python
"""FrameVerdict — the sealed closed-world type + isolated text-frame evaluator (B4 PR-1).
|
|
|
|
Covers the ADR-0222 §5.1 verdict mapping, the frame gating (OPEN / undeclared / non-TEXT
|
|
=> SCOPE_BOUNDARY), the admissibility invariant (entailed_false needs a NAMED positive
|
|
refutation), absence safety (no fact => UNDETERMINED, never entailed_false), and replay
|
|
determinism (order-invariant + repeat-stable trace_hash). INV-31 import/data-flow
|
|
containment lives in ``tests/test_architectural_invariants.py``.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import dataclasses
|
|
|
|
import pytest
|
|
|
|
from generate.frame_verdict import (
|
|
ClosedFrame,
|
|
ClosedWorldProof,
|
|
FrameKind,
|
|
FrameVerdict,
|
|
FrameVerdictKind,
|
|
PositiveRefutationKind,
|
|
WorldAssumption,
|
|
evaluate_frame_verdict,
|
|
)
|
|
|
|
|
|
def _frame(
|
|
propositions,
|
|
*,
|
|
closure: bool = True,
|
|
kind: FrameKind = FrameKind.TEXT,
|
|
wa: WorldAssumption = WorldAssumption.CLOSED,
|
|
) -> ClosedFrame:
|
|
return ClosedFrame(
|
|
frame_id="f1",
|
|
frame_kind=kind,
|
|
world_assumption=wa,
|
|
propositions=tuple(propositions),
|
|
closure_declared=closure,
|
|
source="test",
|
|
provenance=(),
|
|
)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# §5.1 verdict mapping
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_entailed_true_from_closed_text_frame() -> None:
|
|
v = evaluate_frame_verdict(_frame(["a", "a -> b"]), "b")
|
|
assert v.verdict is FrameVerdictKind.ENTAILED_TRUE
|
|
assert v.proof.producer == "proof_chain.entail" and v.proof.outcome == "ENTAILED"
|
|
assert v.proof.positive_refutation_kind is None
|
|
assert v.basis == "as_told"
|
|
|
|
|
|
def test_entailed_false_from_refuted_carries_robdd_refutation() -> None:
|
|
v = evaluate_frame_verdict(_frame(["a", "a -> ~b"]), "b")
|
|
assert v.verdict is FrameVerdictKind.ENTAILED_FALSE
|
|
assert v.proof.producer == "proof_chain.entail" and v.proof.outcome == "REFUTED"
|
|
assert v.proof.positive_refutation_kind is PositiveRefutationKind.ROBDD_REFUTATION
|
|
assert v.proof.proof_sha256 # non-empty positive proof
|
|
|
|
|
|
def test_unknown_maps_to_undetermined() -> None:
|
|
v = evaluate_frame_verdict(_frame(["a"]), "b")
|
|
assert v.verdict is FrameVerdictKind.UNDETERMINED
|
|
|
|
|
|
def test_inconsistent_premises_map_to_contradiction_not_false() -> None:
|
|
v = evaluate_frame_verdict(_frame(["a", "~a"]), "b")
|
|
assert v.verdict is FrameVerdictKind.CONTRADICTION
|
|
assert v.verdict is not FrameVerdictKind.ENTAILED_FALSE
|
|
|
|
|
|
def test_malformed_maps_to_scope_boundary() -> None:
|
|
v = evaluate_frame_verdict(_frame(["@@ not grammar ???"]), "b")
|
|
assert v.verdict is FrameVerdictKind.SCOPE_BOUNDARY
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Frame gating — a negation needs an explicit declared-complete TEXT frame
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_open_world_assumption_refuses() -> None:
|
|
v = evaluate_frame_verdict(_frame(["a", "a -> ~b"], wa=WorldAssumption.OPEN), "b")
|
|
assert v.verdict is FrameVerdictKind.SCOPE_BOUNDARY
|
|
|
|
|
|
def test_undeclared_closure_refuses() -> None:
|
|
v = evaluate_frame_verdict(_frame(["a", "a -> ~b"], closure=False), "b")
|
|
assert v.verdict is FrameVerdictKind.SCOPE_BOUNDARY
|
|
|
|
|
|
def test_perception_frame_is_scope_boundary_in_pr1() -> None:
|
|
v = evaluate_frame_verdict(_frame(["a", "a -> ~b"], kind=FrameKind.PERCEPTION), "b")
|
|
assert v.verdict is FrameVerdictKind.SCOPE_BOUNDARY
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Admissibility invariant — entailed_false needs a NAMED positive refutation
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def _proof(producer, outcome, kind, sha="abc123") -> ClosedWorldProof:
|
|
return ClosedWorldProof(
|
|
producer=producer, outcome=outcome, proof_sha256=sha,
|
|
proof_keys=("k",), positive_refutation_kind=kind,
|
|
)
|
|
|
|
|
|
def _make(verdict, proof) -> FrameVerdict:
|
|
return FrameVerdict(
|
|
frame_id="f1", frame_kind=FrameKind.TEXT,
|
|
world_assumption=WorldAssumption.CLOSED, query="b", verdict=verdict,
|
|
proof=proof, basis="as_told", trace_hash="th", provenance=(),
|
|
)
|
|
|
|
|
|
def test_entailed_false_with_generic_falsified_raises() -> None:
|
|
# generic FALSIFIED (no positive_refutation_kind) cannot prove false.
|
|
with pytest.raises(ValueError):
|
|
_make(FrameVerdictKind.ENTAILED_FALSE, _proof("sensorium.falsification", "FALSIFIED", None))
|
|
|
|
|
|
def test_entailed_false_with_missing_proof_sha_raises() -> None:
|
|
with pytest.raises(ValueError):
|
|
_make(
|
|
FrameVerdictKind.ENTAILED_FALSE,
|
|
_proof("proof_chain.entail", "REFUTED", PositiveRefutationKind.ROBDD_REFUTATION, sha=""),
|
|
)
|
|
|
|
|
|
def test_entailed_false_with_mismatched_kind_raises() -> None:
|
|
# REFUTED text outcome but the PERCEPTION kind — mismatch must raise.
|
|
with pytest.raises(ValueError):
|
|
_make(
|
|
FrameVerdictKind.ENTAILED_FALSE,
|
|
_proof("proof_chain.entail", "REFUTED", PositiveRefutationKind.PERCEPTION_CHANGED_SLOT),
|
|
)
|
|
|
|
|
|
def test_valid_text_entailed_false_constructs() -> None:
|
|
v = _make(
|
|
FrameVerdictKind.ENTAILED_FALSE,
|
|
_proof("proof_chain.entail", "REFUTED", PositiveRefutationKind.ROBDD_REFUTATION),
|
|
)
|
|
assert v.verdict is FrameVerdictKind.ENTAILED_FALSE
|
|
|
|
|
|
def test_entailed_false_in_open_world_raises() -> None:
|
|
# Structural backstop (types.py §(0)): OPEN + entailed_false is illegal even WITH a valid
|
|
# ROBDD refutation proof — absence is never false. No producer may emit an OPEN negation.
|
|
with pytest.raises(ValueError):
|
|
FrameVerdict(
|
|
frame_id="f1", frame_kind=FrameKind.TEXT, world_assumption=WorldAssumption.OPEN,
|
|
query="b", verdict=FrameVerdictKind.ENTAILED_FALSE,
|
|
proof=_proof("proof_chain.entail", "REFUTED", PositiveRefutationKind.ROBDD_REFUTATION),
|
|
basis="as_told", trace_hash="th", provenance=(),
|
|
)
|
|
|
|
|
|
@pytest.mark.parametrize(
|
|
"kind",
|
|
[
|
|
FrameVerdictKind.UNDETERMINED,
|
|
FrameVerdictKind.CONTRADICTION,
|
|
FrameVerdictKind.SCOPE_BOUNDARY,
|
|
],
|
|
)
|
|
def test_refusal_verdicts_need_no_positive_proof(kind) -> None:
|
|
# a proofless / None-kind proof is fine for every refusal/undetermined/contradiction verdict
|
|
# (only the two COMMITTED verdicts — entailed_true / entailed_false — are proof-gated).
|
|
v = _make(kind, _proof("proof_chain.entail", "UNKNOWN", None, sha=""))
|
|
assert v.verdict is kind
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Symmetric admissibility — entailed_true needs a positive entailment/support proof
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_valid_text_entailed_true_constructs() -> None:
|
|
v = _make(FrameVerdictKind.ENTAILED_TRUE, _proof("proof_chain.entail", "ENTAILED", None))
|
|
assert v.verdict is FrameVerdictKind.ENTAILED_TRUE
|
|
|
|
|
|
def test_valid_perception_entailed_true_constructs() -> None:
|
|
v = _make(FrameVerdictKind.ENTAILED_TRUE, _proof("sensorium.falsification", "SUPPORTED", None))
|
|
assert v.verdict is FrameVerdictKind.ENTAILED_TRUE
|
|
|
|
|
|
def test_entailed_true_with_wrong_outcome_raises() -> None:
|
|
# an UNKNOWN/REFUTED outcome cannot back a committed "Yes." — symmetric with entailed_false.
|
|
with pytest.raises(ValueError):
|
|
_make(FrameVerdictKind.ENTAILED_TRUE, _proof("proof_chain.entail", "UNKNOWN", None))
|
|
|
|
|
|
def test_entailed_true_with_missing_proof_sha_raises() -> None:
|
|
with pytest.raises(ValueError):
|
|
_make(FrameVerdictKind.ENTAILED_TRUE, _proof("proof_chain.entail", "ENTAILED", None, sha=""))
|
|
|
|
|
|
def test_entailed_true_with_a_refutation_kind_raises() -> None:
|
|
# a "true" carrying a positive_refutation_kind is malformed (refutation kinds prove FALSE).
|
|
with pytest.raises(ValueError):
|
|
_make(
|
|
FrameVerdictKind.ENTAILED_TRUE,
|
|
_proof("proof_chain.entail", "ENTAILED", PositiveRefutationKind.ROBDD_REFUTATION),
|
|
)
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Absence safety + determinism + distinctness
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_absence_never_yields_entailed_false() -> None:
|
|
# a closed frame with no fact bearing on b: UNDETERMINED, never entailed_false.
|
|
v = evaluate_frame_verdict(_frame(["a", "c -> d"]), "b")
|
|
assert v.verdict is not FrameVerdictKind.ENTAILED_FALSE
|
|
assert v.verdict is FrameVerdictKind.UNDETERMINED
|
|
|
|
|
|
def test_premise_reordering_keeps_trace_hash_stable() -> None:
|
|
a = evaluate_frame_verdict(_frame(["a", "a -> b", "b -> c"]), "c")
|
|
b = evaluate_frame_verdict(_frame(["b -> c", "a -> b", "a"]), "c")
|
|
assert a.verdict is b.verdict is FrameVerdictKind.ENTAILED_TRUE
|
|
assert a.trace_hash == b.trace_hash and a.proof.proof_sha256 == b.proof.proof_sha256
|
|
|
|
|
|
def test_repeated_evaluation_keeps_trace_hash_stable() -> None:
|
|
a = evaluate_frame_verdict(_frame(["a", "a -> ~b"]), "b")
|
|
b = evaluate_frame_verdict(_frame(["a", "a -> ~b"]), "b")
|
|
assert a.trace_hash == b.trace_hash and a.proof.proof_sha256 == b.proof.proof_sha256
|
|
|
|
|
|
def test_frame_verdict_is_distinct_from_determined() -> None:
|
|
# no `answer` field, a 5-way verdict enum — cannot be confused with an open-world answer.
|
|
field_names = {f.name for f in dataclasses.fields(FrameVerdict)}
|
|
assert "answer" not in field_names
|
|
assert "verdict" in field_names
|
|
assert FrameVerdict.__name__ == "FrameVerdict" # distinct call-name from Determined
|