Compare commits
1 commit
add-ask-ac
...
feat/holdo
| Author | SHA1 | Date | |
|---|---|---|---|
|
6e0d114726 |
424 changed files with 508 additions and 46748 deletions
4
.gitignore
vendored
4
.gitignore
vendored
|
|
@ -25,10 +25,6 @@ frontier_wave1.json
|
|||
# Claude Code local session artifacts (per-developer, never tracked)
|
||||
.claude/
|
||||
|
||||
# Local macro->micro system map — a per-developer NAVIGATION INDEX (like an IDE
|
||||
# symbol index), NOT a project artifact. Regenerated on demand; never tracked.
|
||||
.system-map/
|
||||
|
||||
# Runnable audit-passed showcases (ADR-0112 + ADR-0113) are generated on
|
||||
# demand from the signed claim + on-disk lane result files. The inputs
|
||||
# are committed; the renders are not.
|
||||
|
|
|
|||
|
|
@ -42,7 +42,6 @@ is a CI failure (`.github/workflows/lane-shas.yml`).
|
|||
| ADR-0099 | `public_demo` | Public showcase runs deterministically under 30s; all claims supported | `evals/public_demo/results/v1_dev.json` | `e323adb35ea17987991395424c603ff93bca08c11bc2713bd9f6338e86bb269f` |
|
||||
| ADR-0104 | `curriculum_loop_closure` | Curriculum-sourced proposals route through single reviewed teaching path | `evals/curriculum_loop_closure/results/v1_dev.json` | `b46d56b2d209172cc3ffaf3776dc8dcfe55093f13587c5cb67372be6dfa23e8d` |
|
||||
| ADR-0131 | `math_teaching_corpus_v1` | Math teaching corpus replays deterministically; all chains pass exit criterion (correct_rate=1.0, wrong=0) | `evals/math_teaching_corpus/v1/report.json` | `eaf160d145da29f9050ede8d58bf111b0f651dd40aeae9201857d0b97e014dd4` |
|
||||
| ADR-0206 | `deductive_logic_v1` | Propositional entailment scored against an independent truth-table oracle; dev+holdout+external 716/716 correct, wrong=0, refused=0 | `evals/deductive_logic/report.json` | `97a230949016e38d5e3f37a69e4245b320575ee70e5af92ff7607f7b05f74b5f` |
|
||||
|
||||
## Verification
|
||||
|
||||
|
|
|
|||
36
CLAUDE.md
36
CLAUDE.md
|
|
@ -58,16 +58,6 @@ Allowed sites:
|
|||
- `ingest/gate.py` for raw input injection.
|
||||
- `language_packs/compiler.py` and vocabulary construction.
|
||||
- `algebra/versor.py` for algebra-owned sandwich closure.
|
||||
- `sensorium/*/canonical.py` and pack-governed modality compiler construction
|
||||
boundaries for pinned signal canonicalization and quantization.
|
||||
- `session/context.py` for session-scoped **semantic anchoring** of the field
|
||||
toward the session concept-attractor (the anchor pull, hemisphere
|
||||
consistency). Allowed ONLY because every such op (1) preserves
|
||||
`versor_condition` BY CONSTRUCTION — composed from `rotor_power` /
|
||||
`word_transition_rotor` / `versor_apply` on the Spin manifold, never a
|
||||
post-hoc `unitize`/grade-projection — AND (2) carries semantic meaning in
|
||||
the cognitive model. An op that needs a post-hoc closure repair (the
|
||||
rejected `_slerp_toward`) fails clause (1) and stays forbidden.
|
||||
|
||||
Forbidden sites:
|
||||
|
||||
|
|
@ -80,15 +70,6 @@ Do not add drift repair, grade projection, watchdogs, timers, hot-path
|
|||
normalizers, or monitoring functions whose only purpose is to repair another
|
||||
function.
|
||||
|
||||
**The bright line — semantic anchoring vs. drift repair.** An op is *semantic
|
||||
anchoring* (allowed at the sites above) iff it preserves `versor_condition` by
|
||||
construction AND expresses a relation in the cognitive model. It is *drift
|
||||
repair* (forbidden) iff its purpose is to restore a numerical invariant a prior
|
||||
function should have preserved. Closure of field transitions is owned solely
|
||||
by `algebra/versor.py` (`_close_applied_versor`); no other site may "fix" it.
|
||||
Naming must not disguise the distinction: an op that anchors semantically must
|
||||
not be named or documented as a "drift fix".
|
||||
|
||||
CGA null vectors are not unit versors. Preserve null vectors as null vectors.
|
||||
|
||||
## Core Primitives
|
||||
|
|
@ -123,23 +104,14 @@ runtime path. Vault recall is exact and deterministic.
|
|||
- `calibration/*` — bounded replay-based calibration.
|
||||
- `docs/runtime_contracts.md` — response, telemetry, memory, identity, and testing contracts.
|
||||
|
||||
### GSM8K math comprehension substrate (sealed; serving `7/43/0`, wrong=0 — moves only via ratified PRs)
|
||||
### GSM8K math comprehension substrate (sealed; serving `6/44/0`, wrong=0 — moves only via ratified PRs)
|
||||
|
||||
- `core/reliability_gate/` — calibrated-learning ledger + gate (ADR-0175): `ClassTally` counts, `conservative_floor` (one-sided Wilson, N_MIN=10), θ ceilings.
|
||||
- `generate/derivation/` — the comprehension composer: `extract.py` (lexeme quantity extraction, EX-1/4/5 + function-word unit filter), `clauses.py` (GB-1 segmentation), `compose.py` (GB-2a list-sum + GB-3a clause-scoped referent guard), `accumulate.py` (GB-3b.1 single-referent gain/loss chaining), `goal_residual.py` (ADR-0207 R4 goal-residual production), `multistep.py`/`search.py` (bounded search), `verify.py` (the wrong=0 self-verification gate: grounding ∧ cue ∧ unit ∧ completeness ∧ uniqueness).
|
||||
- `generate/derivation/` — the comprehension composer: `extract.py` (lexeme quantity extraction, EX-1/4/5 + function-word unit filter), `clauses.py` (GB-1 segmentation), `compose.py` (GB-2a list-sum + GB-3a clause-scoped referent guard), `accumulate.py` (GB-3b.1 single-referent gain/loss chaining), `multistep.py`/`search.py` (bounded search), `verify.py` (the wrong=0 self-verification gate: grounding ∧ cue ∧ unit ∧ completeness ∧ uniqueness).
|
||||
- `generate/cue_precision/` — `(cue, op, unit_shape)` reliability ledger + trainer (ADR-0177 CP-1/CP-2a); inert (consulted by no serving/gate path yet).
|
||||
- `evals/gsm8k_math/` — `train_sample/` (real GSM8K dev sample, currently 7 correct / 43 refused / 0 wrong), `practice/` (sealed attempt-and-eliminate lane + ADR-0163-F additive set), `confusers/` (ADR-0163-F2 discrimination probe — scored by `wrong→0` + pair-consistency, NOT flip-count).
|
||||
- `evals/gsm8k_math/` — `train_sample/` (real GSM8K, the capability metric), `practice/` (sealed attempt-and-eliminate lane + ADR-0163-F additive set), `confusers/` (ADR-0163-F2 discrimination probe — scored by `wrong→0` + pair-consistency, NOT flip-count).
|
||||
- `scripts/verify_lane_shas.py`, `scripts/generate_claims.py --check` — the serving-frozen gate (pinned eval-lane SHAs + `CLAIMS.md`).
|
||||
|
||||
### Sensorium / modality compiler substrate (parallel, afferent gates; no broad capability claim)
|
||||
|
||||
- `sensorium/compiler/` — shared compiler law for content-addressed afferent compilation units, canonical deltas, local arenas, and trace-safe merge hashes.
|
||||
- `sensorium/audio/` + `sensorium/adapters/audio.py` — `audio_core_v1`, deterministic audio compiler substrate, gate closed by default.
|
||||
- `sensorium/vision/` + `sensorium/adapters/vision.py` — `vision_core_v1`, tile-first deterministic visual compiler substrate over synthetic eval fixtures, gate closed by default.
|
||||
- `sensorium/environment/` — ADR-0208 observation-frame contract for bundles of already-compiled afferent units; not late fusion and not a mutable world model.
|
||||
- `sensorium/sensorimotor/` + `sensorium/adapters/sensorimotor.py` — ADR-0209 afferent proprioception/contact/action-result evidence substrate; no decode path.
|
||||
- `sensorium/registry.py::decode*` + `AuthorityToken` / `EfferentGate` — ADR-0198 fail-closed efferent governance path. This is not a ratified motor decoder or actuator interface; no real action emission is claimed.
|
||||
|
||||
## Efficiency and Performance Doctrine
|
||||
|
||||
Performance is part of correctness for this project because slow feedback hides
|
||||
|
|
@ -355,7 +327,7 @@ implicated rather than one.
|
|||
things the ADR says that didn't actually ship.
|
||||
|
||||
2. **Before merging a stacked PR sequence into main.** When 2+ PRs
|
||||
stack (PR #420 stacked on #416, PR #423 stacked on #420), the
|
||||
stack (PR #420 stacked on #416, #423 stacked on #420), the
|
||||
review-each-PR-individually pattern misses cross-PR consistency
|
||||
issues. Audit the whole stack as one unit before any merge.
|
||||
|
||||
|
|
|
|||
|
|
@ -1,19 +1,5 @@
|
|||
from .cl41 import geometric_product, reverse, grade_project, scalar_part, norm_squared, basis_vector
|
||||
from .versor import versor_apply, normalize_to_versor, versor_condition
|
||||
from .cga import (
|
||||
EMBED_EXACT_MAX,
|
||||
cga_inner,
|
||||
outer_product,
|
||||
is_null,
|
||||
null_project,
|
||||
embed_point,
|
||||
read_scalar_e1,
|
||||
blade_grade,
|
||||
blade_norm,
|
||||
graded_wedge,
|
||||
is_incident,
|
||||
dual,
|
||||
meet,
|
||||
)
|
||||
from .cga import cga_inner, outer_product, is_null, null_project, embed_point
|
||||
from .holonomy import holonomy_encode, holonomy_similarity
|
||||
from .rotor import word_transition_rotor
|
||||
|
|
|
|||
158
algebra/cga.py
158
algebra/cga.py
|
|
@ -19,34 +19,13 @@ No cosine similarity. No L2 norm. No approximate indexing.
|
|||
"""
|
||||
|
||||
import numpy as np
|
||||
from .cl41 import (
|
||||
geometric_product,
|
||||
grade_project,
|
||||
reverse,
|
||||
scalar_part,
|
||||
N_COMPONENTS,
|
||||
)
|
||||
|
||||
# The unit pseudoscalar I5 = e1 e2 e3 e4 e5 (the grade-5 blade, component 31).
|
||||
# In Cl(4,1) with signature (+,+,+,+,-), I5^2 = -1, so I5^{-1} = -I5. Used by
|
||||
# ``dual`` / ``meet``. Module-level singleton; never mutated.
|
||||
_PSEUDOSCALAR_INDEX = 31
|
||||
_I5 = np.zeros(N_COMPONENTS, dtype=np.float64)
|
||||
_I5[_PSEUDOSCALAR_INDEX] = 1.0
|
||||
from .cl41 import geometric_product, scalar_part, basis_vector, N_COMPONENTS
|
||||
|
||||
# Basis-vector component indices for e4/e5 inside the grade-1 block.
|
||||
# component 1=e1, 2=e2, 3=e3, 4=e4, 5=e5.
|
||||
_E4_IDX = 4
|
||||
_E5_IDX = 5
|
||||
|
||||
# Pinned magnitude ceiling for f64-exact embedding + read-back (Phase 0A).
|
||||
# Below this bound, ``embed_point(..., dtype=np.float64)`` round-trips integer
|
||||
# coordinates exactly through ``read_scalar_e1`` and the conformal distance metric
|
||||
# stays exact (proven in tests/test_cga_f64_exactness.py). The field-reasoner reader
|
||||
# REFUSES any quantity whose magnitude exceeds this bound; the refusal lives in the
|
||||
# reader — this module only states the bound. Generous vs GSM8K (quantities ~< 1e5).
|
||||
EMBED_EXACT_MAX: int = 1_000_000
|
||||
|
||||
|
||||
def cga_inner(X: np.ndarray, Y: np.ndarray) -> float:
|
||||
"""
|
||||
|
|
@ -59,19 +38,10 @@ def cga_inner(X: np.ndarray, Y: np.ndarray) -> float:
|
|||
|
||||
|
||||
def outer_product(X: np.ndarray, Y: np.ndarray) -> np.ndarray:
|
||||
"""The antisymmetric (commutator) product ``0.5 * (XY - YX)``.
|
||||
|
||||
HONEST CONTRACT: this equals the grade-raising wedge ``X ^ Y`` **only when both
|
||||
operands are grade 1** (vectors). For higher-grade operands it is the *commutator*
|
||||
(Lie bracket), which is NOT the wedge — in particular it does NOT build a k-blade
|
||||
by repeated application (a bivector commuted with a vector collapses the grade-3
|
||||
part to grade 1). Existing callers use the result as an opaque, deterministic
|
||||
relationship feature (folded into a scalar via :func:`cga_inner`), where the
|
||||
commutator is well-defined regardless; none read it by grade.
|
||||
|
||||
For the true grade-raising exterior product (lines/planes/incidence) use
|
||||
:func:`graded_wedge`. (Renamed contract only — behaviour is unchanged, so every
|
||||
current caller is byte-identical.)
|
||||
"""
|
||||
Outer (wedge) product: X ^ Y.
|
||||
For a prompt versor X_p and response versor X_r,
|
||||
X_p ^ X_r is a grade-2 object encoding their geometric relationship.
|
||||
"""
|
||||
XY = geometric_product(X, Y)
|
||||
YX = geometric_product(Y, X)
|
||||
|
|
@ -92,25 +62,18 @@ def null_project(X: np.ndarray) -> np.ndarray:
|
|||
return embed_point(euclidean)
|
||||
|
||||
|
||||
def embed_point(x: np.ndarray, *, dtype: "np.typing.DTypeLike" = np.float32) -> np.ndarray:
|
||||
def embed_point(x: np.ndarray) -> np.ndarray:
|
||||
"""
|
||||
Embed a Euclidean point x in R^3 into the CGA null cone.
|
||||
|
||||
X = x + n_o + 0.5|x|^2 n_inf,
|
||||
where n_o = 0.5(e5-e4), n_inf = e4+e5.
|
||||
|
||||
``dtype`` defaults to ``float32`` so every existing caller is byte-unchanged.
|
||||
The field-reasoner reader passes ``dtype=np.float64`` to get an exact embedding:
|
||||
``geometric_product`` already preserves float64 (``np.result_type``), so the
|
||||
only thing that forced f32 was this construction. f32 silently collapses the
|
||||
``n_o`` weight past ~1e4 (the ``0.5|x|^2`` terms lose the ``±1``); f64 keeps it
|
||||
exact up to :data:`EMBED_EXACT_MAX` (see tests/test_cga_f64_exactness.py).
|
||||
"""
|
||||
x = np.asarray(x, dtype=dtype)
|
||||
x = np.asarray(x, dtype=np.float32)
|
||||
assert len(x) == 3, "embed_point expects a 3D vector"
|
||||
|
||||
x_sq = float(np.dot(x, x))
|
||||
result = np.zeros(N_COMPONENTS, dtype=dtype)
|
||||
result = np.zeros(N_COMPONENTS, dtype=np.float32)
|
||||
result[1:4] = x
|
||||
|
||||
# n_o + 0.5|x|^2 n_inf
|
||||
|
|
@ -119,108 +82,3 @@ def embed_point(x: np.ndarray, *, dtype: "np.typing.DTypeLike" = np.float32) ->
|
|||
result[_E4_IDX] = 0.5 * (x_sq - 1.0)
|
||||
result[_E5_IDX] = 0.5 * (x_sq + 1.0)
|
||||
return result
|
||||
|
||||
|
||||
def read_scalar_e1(X: np.ndarray) -> float:
|
||||
"""Projective dehomogenization on the e1 axis — the exact, weight-invariant
|
||||
read-back of a scalar coordinate from a (possibly dilated) conformal point.
|
||||
|
||||
A point at coordinate ``v`` on the e1 number line embeds as
|
||||
``X = v*e1 + n_o + 0.5 v^2 n_inf``; a uniform conformal dilation by ``k``
|
||||
scales the whole null vector. The coordinate is recovered as
|
||||
``e1_coefficient / n_o_weight`` where the n_o weight is ``X[e5] - X[e4]``
|
||||
(== 1 for an un-dilated point), so any dilation weight divides out. This is
|
||||
the correct read-back for weight-changing operators; a raw distance-from-origin
|
||||
is wrong for them.
|
||||
|
||||
Raises ``ValueError`` on a degenerate (zero) n_o weight — a point at infinity
|
||||
or an f32 weight-collapse — rather than returning a silently wrong value.
|
||||
"""
|
||||
no_weight = float(X[_E5_IDX] - X[_E4_IDX])
|
||||
if no_weight == 0.0:
|
||||
raise ValueError(
|
||||
"read_scalar_e1: degenerate n_o weight (point at infinity or f32 collapse)"
|
||||
)
|
||||
return float(X[1]) / no_weight
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Incidence algebra — the corrected grade-raising wedge, dual, and meet.
|
||||
# These let the inner product operate on RELATIONS among entities (lines, planes,
|
||||
# incidence) rather than only pairwise point distance. Built only from the existing
|
||||
# Cl(4,1) primitives (geometric_product, grade_project) + the pseudoscalar; they add
|
||||
# no normalization, no approximation, and leave the versor_condition path untouched
|
||||
# (flats are null-cone outer products, not unit versors).
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_MAX_GRADE = 5 # Cl(4,1): grades 0..5
|
||||
|
||||
|
||||
def blade_grade(X: np.ndarray) -> int:
|
||||
"""The single grade of a homogeneous blade. Raises if X is zero or grade-mixed.
|
||||
|
||||
Grade is detected by EXACT nonzero (no tolerance): integer-coordinate embeddings
|
||||
produce exact integer blades in float64, so a grade block is exactly 0 or not.
|
||||
"""
|
||||
grades = [k for k in range(_MAX_GRADE + 1) if np.any(grade_project(X, k))]
|
||||
if len(grades) != 1:
|
||||
raise ValueError(f"not a homogeneous blade: nonzero grades {grades}")
|
||||
return grades[0]
|
||||
|
||||
|
||||
def graded_wedge(X: np.ndarray, Y: np.ndarray) -> np.ndarray:
|
||||
"""The true grade-raising exterior product ``X ^ Y`` for homogeneous blades.
|
||||
|
||||
``X ^ Y = <X Y>_{grade(X)+grade(Y)}`` — the top-grade part of the geometric
|
||||
product. Unlike :func:`outer_product` (the commutator) this composes correctly:
|
||||
``graded_wedge(graded_wedge(P, Q), n_inf)`` builds the grade-3 line P^Q^n_inf,
|
||||
and so on. If the grades sum past the pseudoscalar (>5) the wedge is identically
|
||||
zero. For two grade-1 vectors it agrees with :func:`outer_product` exactly.
|
||||
"""
|
||||
gx, gy = blade_grade(X), blade_grade(Y)
|
||||
if gx + gy > _MAX_GRADE:
|
||||
return np.zeros(N_COMPONENTS, dtype=geometric_product(X, Y).dtype)
|
||||
return grade_project(geometric_product(X, Y), gx + gy)
|
||||
|
||||
|
||||
def blade_norm(X: np.ndarray) -> float:
|
||||
"""Reversion norm ``sqrt(|<X reverse(X)>_0|)`` — zero iff X is the zero blade."""
|
||||
return float(np.sqrt(abs(scalar_part(geometric_product(X, reverse(X))))))
|
||||
|
||||
|
||||
def is_incident(point: np.ndarray, flat: np.ndarray) -> bool:
|
||||
"""Exact incidence test: is ``point`` on ``flat`` (a line/plane OPNS blade)?
|
||||
|
||||
True iff ``point ^ flat == 0`` EXACTLY (every component zero) — no float
|
||||
tolerance to admit (the wrong=0 discipline: a near-incident point is REFUSED,
|
||||
not admitted). Exact for integer-coordinate points within ``EMBED_EXACT_MAX``.
|
||||
"""
|
||||
return not bool(np.any(graded_wedge(point, flat)))
|
||||
|
||||
|
||||
def dual(X: np.ndarray) -> np.ndarray:
|
||||
"""Pseudoscalar dual ``X * I5^{-1}`` (``I5^{-1} = -I5`` since ``I5^2 = -1``).
|
||||
|
||||
Maps a grade-k blade to grade ``5-k``. Involutive up to sign:
|
||||
``dual(dual(X)) == -X``.
|
||||
"""
|
||||
return geometric_product(X, -_I5)
|
||||
|
||||
|
||||
def meet(A: np.ndarray, B: np.ndarray) -> np.ndarray:
|
||||
"""The meet (intersection) ``dual(dual(A) ^ dual(B))`` of two homogeneous blades.
|
||||
|
||||
Correct for operands in GENERAL POSITION whose join spans the space — e.g. two
|
||||
non-parallel planes meet in their intersection line. The grade of the result is
|
||||
``grade(A)+grade(B)-5``.
|
||||
|
||||
HONEST ENVELOPE: this full-pseudoscalar meet DEGENERATES for operands that share
|
||||
a proper subspace (e.g. two coplanar lines, two parallel planes): the inner wedge
|
||||
``dual(A) ^ dual(B)`` is then identically zero, so ``meet`` returns the **zero
|
||||
multivector** — a detectable signal of "no transversal meet", never a silently
|
||||
wrong value. The general intersection of such operands (e.g. the point where two
|
||||
coplanar lines cross) requires the *join-relative* meet, which is deliberately
|
||||
NOT implemented here; the caller MUST check ``blade_norm(result) == 0`` and treat
|
||||
zero as degenerate/refuse rather than as a geometric object.
|
||||
"""
|
||||
return dual(graded_wedge(dual(A), dual(B)))
|
||||
|
|
|
|||
393
chat/runtime.py
393
chat/runtime.py
|
|
@ -4,7 +4,6 @@ from dataclasses import dataclass, replace
|
|||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import warnings
|
||||
from collections.abc import Sequence
|
||||
from typing import Any, List
|
||||
|
||||
|
|
@ -38,7 +37,6 @@ from core.epistemic_state import (
|
|||
epistemic_state_for_grounding_source,
|
||||
normative_detail_from_verdicts,
|
||||
)
|
||||
from core.proposal_review.summary import ProposalReviewIdleSummary, idle_summary
|
||||
from core.response_governance import govern_response, shape_surface
|
||||
from chat.telemetry import (
|
||||
TurnEventSink,
|
||||
|
|
@ -54,8 +52,7 @@ from teaching.discovery import (
|
|||
format_candidate_jsonl,
|
||||
)
|
||||
from teaching.discovery_sink import DiscoveryCandidateSink
|
||||
from engine_state import EngineStateStore, get_git_revision
|
||||
from core.engine_identity import engine_identity_for_config
|
||||
from engine_state import EngineStateStore
|
||||
from recognition.anti_unifier import derive_recognizer
|
||||
from recognition.outcome import FeatureBundle
|
||||
from recognition.registry import RecognizerRegistry
|
||||
|
|
@ -389,28 +386,6 @@ def _make_trajectory_from_result(result, turn: int):
|
|||
return operator.build(frames, trajectory_id=f"turn_{turn}")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class TurnAccrual:
|
||||
"""The inline-realization outcome of one turn (Step B / E).
|
||||
|
||||
``kind`` is ``"realized"`` (a declarative fact was accrued into the held self),
|
||||
``"determined"`` (a question was answered over realized knowledge), ``"estimated"``
|
||||
(Step E — a converse query DETERMINE refused, for which a calibrated converse-guess
|
||||
exists), or ``"none"`` (nothing comprehensible to accrue/determine). The payload
|
||||
carries the typed result for introspection. B-1 records, does not surface; B-2 and E
|
||||
surface only when their flag is on.
|
||||
"""
|
||||
|
||||
kind: str
|
||||
realized: Any = None # generate.realize.Realized | NotRealized | None
|
||||
determination: Any = None # generate.determine.Determined | Undetermined | None
|
||||
# Step E — the converse-guess candidate and its SERVE license, when ``kind`` is
|
||||
# ``"estimated"``. ``estimate`` is a ``ConverseEstimate``; ``license`` a
|
||||
# ``LicenseDecision`` (None if the predicate-class is absent from the ratified ledger).
|
||||
estimate: Any = None
|
||||
license: Any = None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ChatResponse:
|
||||
surface: str
|
||||
|
|
@ -528,29 +503,6 @@ class ChatResponse:
|
|||
dispatch_trace: DispatchTrace | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class IdleTickResult:
|
||||
"""Outcome of one ``idle_tick``.
|
||||
|
||||
The proposal pass is PROPOSAL-ONLY learning, never ratification (HITL ratifies). The
|
||||
Step D consolidation pass is SESSION-memory learning: ``facts_consolidated`` derived
|
||||
facts were written back into the held self so the next ``determine`` reaches them
|
||||
directly — still never corpus mutation, never a proposal. The proposal-review sub-pass
|
||||
(``proposal_review``) is READ-ONLY: it surfaces pending comprehension-failure proposals for
|
||||
review and mutates nothing.
|
||||
"""
|
||||
|
||||
candidates_contemplated: int
|
||||
proposals_created: int
|
||||
pending_proposals: int
|
||||
#: Step D — derived facts consolidated into the held self this tick (0 unless
|
||||
#: ``config.consolidate_determinations`` and the closure had a new layer to add).
|
||||
facts_consolidated: int = 0
|
||||
#: IT — read-only proposal-review summary (None unless ``config.review_pending_proposals``).
|
||||
#: Surfaces pending comprehension-failure proposals for review; mutates nothing.
|
||||
proposal_review: ProposalReviewIdleSummary | None = None
|
||||
|
||||
|
||||
class ChatRuntime:
|
||||
def __init__(
|
||||
self,
|
||||
|
|
@ -709,11 +661,6 @@ class ChatRuntime:
|
|||
# W-013 — last classified intent, updated each turn for /explain REPL use.
|
||||
self._last_intent: Any | None = None
|
||||
self._last_input_text: str = ""
|
||||
# Step B (inline realization) — the last turn's accrual outcome (what the turn
|
||||
# realized into the held self, or determined over it). Introspectable; the
|
||||
# surface contract is unchanged (slice B-1 records, does not surface).
|
||||
self._last_turn_accrual: TurnAccrual | None = None
|
||||
self._relational_pack_lemmas: frozenset[str] | None = None
|
||||
self._engine_state_store: EngineStateStore | None = (
|
||||
None if no_load_state else EngineStateStore(engine_state_path)
|
||||
)
|
||||
|
|
@ -727,27 +674,6 @@ class ChatRuntime:
|
|||
# checkpoint is loaded, flushed to the sink on attach_telemetry_sink.
|
||||
# None means no reboot was detected this session.
|
||||
self._pending_reboot_payload: str | None = None
|
||||
# L11 — the engine's content-derived identity (who am I), and the
|
||||
# identity stamped in the loaded checkpoint (the lineage parent for the
|
||||
# next checkpoint). ``_loaded_engine_identity`` stays "" at genesis.
|
||||
self._engine_identity: str = engine_identity_for_config(
|
||||
self.config, get_git_revision()
|
||||
)
|
||||
self._loaded_engine_identity: str = ""
|
||||
# CL — the persistent reviewed-learning proposal log. ``idle_tick()``
|
||||
# advances it during idle (proposal-only); it lives alongside the engine
|
||||
# state so the learning backlog survives reboot. None for no_load_state
|
||||
# (ephemeral runtimes don't accumulate a learning lineage).
|
||||
self._proposal_log: Any | None = None
|
||||
if self._engine_state_store is not None:
|
||||
from teaching.proposals import ProposalLog
|
||||
|
||||
self._proposal_log = ProposalLog(
|
||||
path=self._engine_state_store.path / "proposals.jsonl"
|
||||
)
|
||||
# L11 — set True on reboot when the stamped checkpoint identity differs
|
||||
# from the recomputed identity (the ratified substrate changed while down).
|
||||
self.identity_continuity_break: bool = False
|
||||
if self._engine_state_store is not None and self._engine_state_store.exists():
|
||||
self._load_engine_state()
|
||||
|
||||
|
|
@ -755,46 +681,11 @@ class ChatRuntime:
|
|||
store = self._engine_state_store
|
||||
if store is None:
|
||||
return
|
||||
# Schema-version compatibility gates the whole load: load_manifest()
|
||||
# refuses (raises) a checkpoint written by newer code BEFORE we read any
|
||||
# recognizers/candidates (L10 step-2 migration discipline).
|
||||
manifest = store.load_manifest() or {}
|
||||
recognizers = store.load_recognizers()
|
||||
self._recognizer_registry = RecognizerRegistry.from_recognizers(recognizers)
|
||||
self._pending_candidates = store.load_discovery_candidates()
|
||||
manifest = store.load_manifest() or {}
|
||||
self._turn_count = int(manifest.get("turn_count", 0))
|
||||
# L11 — the identity this checkpoint was written under becomes the lineage
|
||||
# parent of the next checkpoint we write. If it differs from the identity
|
||||
# we recomputed at boot, the ratified substrate changed during downtime:
|
||||
# we would resume the lived state under a DIFFERENT identity. Surface it
|
||||
# (warn + flag); refuse only under strict_identity_continuity.
|
||||
self._loaded_engine_identity = str(manifest.get("engine_identity", ""))
|
||||
if (
|
||||
self._loaded_engine_identity
|
||||
and self._loaded_engine_identity != self._engine_identity
|
||||
):
|
||||
self.identity_continuity_break = True
|
||||
message = (
|
||||
"engine identity continuity break: checkpoint was written under "
|
||||
f"{self._loaded_engine_identity[:12]}… but this build computes "
|
||||
f"{self._engine_identity[:12]}… — the ratified identity substrate "
|
||||
"changed while the engine was down. Resuming would carry the lived "
|
||||
"state into a different identity."
|
||||
)
|
||||
if self.config.strict_identity_continuity:
|
||||
from core.engine_identity import IdentityContinuityError
|
||||
|
||||
raise IdentityContinuityError(message)
|
||||
warnings.warn(message, RuntimeWarning, stacklevel=2)
|
||||
# Shape B+ (schema v2): restore the lived session state into the live
|
||||
# context so a reboot resumes the SAME life (field/vault/anchor/graph/
|
||||
# referents/dialogue). Opt-in (config.persist_session_state); None for a
|
||||
# v1 checkpoint -> fresh session (the historical Shape B behavior), so old
|
||||
# checkpoints stay loadable.
|
||||
if self.config.persist_session_state and self._context is not None:
|
||||
session_snapshot = store.load_session_state()
|
||||
if session_snapshot is not None:
|
||||
self._context.restore(session_snapshot)
|
||||
# W-024 / ADR-0158 — buffer reboot event for emission when sink attaches.
|
||||
from engine_state import _git_revision
|
||||
self._pending_reboot_payload = format_reboot_event_jsonl(
|
||||
|
|
@ -828,130 +719,7 @@ class ChatRuntime:
|
|||
for c in candidates_to_save
|
||||
]
|
||||
store.save_discovery_candidates(candidates_to_save)
|
||||
# Shape B+ (schema v2): persist the lived session state (field, vault,
|
||||
# anchor, graph, referents, dialogue) BEFORE the manifest, so the
|
||||
# manifest stays the last durable act — the commit marker for the turn.
|
||||
# Opt-in (config.persist_session_state): a deliberate resume mode, off by
|
||||
# default so one-shot runtimes don't pay the per-turn snapshot cost.
|
||||
if self._context is not None and self.config.persist_session_state:
|
||||
store.save_session_state(self._context.snapshot())
|
||||
# L11 — stamp the engine's identity and its lineage parent (the identity
|
||||
# of the prior checkpoint). Same substrate -> identity == parent (a stable
|
||||
# life); a ratified substrate change -> identity != parent (the bump).
|
||||
store.save_manifest(
|
||||
self._turn_count,
|
||||
engine_identity=self._engine_identity,
|
||||
parent_engine_identity=self._loaded_engine_identity,
|
||||
)
|
||||
self._loaded_engine_identity = self._engine_identity
|
||||
|
||||
def _count_pending_proposals(self) -> int:
|
||||
if self._proposal_log is None:
|
||||
return 0
|
||||
return sum(
|
||||
1
|
||||
for entry in self._proposal_log.current_state().values()
|
||||
if entry.get("state") == "pending"
|
||||
)
|
||||
|
||||
def idle_tick(self) -> "IdleTickResult":
|
||||
"""Advance learning during idle (NO user turn). Two disjoint passes:
|
||||
|
||||
1. PROPOSAL pass (the reviewed-learning flywheel): turn the pending discovery
|
||||
backlog into reviewable teaching proposals. Contemplate each pending
|
||||
candidate (enrichment) and run the replay-gated ``propose_from_candidate``,
|
||||
which leaves a PROPOSAL-ONLY ``pending`` entry in the persistent proposal
|
||||
log. An idle tick NEVER ratifies — ratification (appending to the corpus)
|
||||
stays HITL via ``teaching/review`` (CLAUDE.md teaching safety). The tick only
|
||||
*proposes*; the reviewed loop is not bypassed or duplicated.
|
||||
|
||||
2. CONSOLIDATION pass (Step D — CLOSE): the loop learns from *determined* facts.
|
||||
Run one semi-naive layer of the member/subset deductive closure over the held
|
||||
self (``generate.determine.consolidate_once``); each soundly-derived,
|
||||
proof_chain-verified fact is written back as a SPECULATIVE realized record so
|
||||
the next ``determine`` reaches it directly. SESSION memory (immediate,
|
||||
allowed) — NOT corpus mutation, NOT a proposal; the HITL path is untouched.
|
||||
Gated by ``config.consolidate_determinations``. The closure converges (a
|
||||
saturated tick consolidates nothing — ``at_fixed_point``).
|
||||
|
||||
The proposal log, the candidate backlog, and (with ``persist_session_state``)
|
||||
the consolidated facts all live in the engine-state dir, so this learning
|
||||
progress persists across reboot (CL-2).
|
||||
"""
|
||||
contemplated_count = 0
|
||||
created = 0
|
||||
facts_consolidated = 0
|
||||
did_work = False
|
||||
|
||||
# 1. Proposal pass — unchanged behavior, runs only with a log + backlog.
|
||||
if self._proposal_log is not None and self._pending_candidates:
|
||||
from teaching.contemplation import contemplate
|
||||
from teaching.proposals import (
|
||||
ProposalError,
|
||||
TeachingChainProposal,
|
||||
_current_revision,
|
||||
propose_from_candidate,
|
||||
)
|
||||
from teaching.source import ProposalSource
|
||||
|
||||
vault_probe = (
|
||||
_vault_probe_for_context(self._context) if self._context else None
|
||||
)
|
||||
contemplated = [
|
||||
contemplate(candidate, vault_probe=vault_probe)
|
||||
for candidate in self._pending_candidates
|
||||
]
|
||||
contemplated_count = len(contemplated)
|
||||
for candidate in contemplated:
|
||||
source = ProposalSource(
|
||||
kind="contemplation",
|
||||
source_id=candidate.candidate_id,
|
||||
emitted_at_revision=_current_revision(),
|
||||
)
|
||||
try:
|
||||
result = propose_from_candidate(
|
||||
candidate, log=self._proposal_log, source=source
|
||||
)
|
||||
except ProposalError:
|
||||
continue
|
||||
if isinstance(result, TeachingChainProposal):
|
||||
created += 1
|
||||
did_work = True
|
||||
|
||||
# 2. Consolidation pass (Step D) — runs independently of the backlog.
|
||||
if self.config.consolidate_determinations and self._context is not None:
|
||||
from generate.determine import consolidate_once
|
||||
|
||||
facts_consolidated = consolidate_once(self._context).consolidated
|
||||
did_work = True
|
||||
|
||||
# 3. Proposal-review sub-pass (IT) — READ-ONLY. Surfaces pending comprehension-failure
|
||||
# proposals (the contemplation pass's N5 artifacts) for review. It NEVER mutates an
|
||||
# artifact, NEVER sets ``did_work`` (no state change → no checkpoint), NEVER ratifies
|
||||
# or mounts. A reporter failure is CAPTURED, not propagated, so it cannot corrupt the
|
||||
# tick. Gated off by default.
|
||||
proposal_review = None
|
||||
if self.config.review_pending_proposals:
|
||||
try:
|
||||
proposal_review = idle_summary()
|
||||
except Exception as exc: # noqa: BLE001 — a reporter failure must not corrupt the tick
|
||||
proposal_review = ProposalReviewIdleSummary(
|
||||
safe=False, total=0, review_needed=0, malformed=0, by_family=(),
|
||||
errors=(f"proposal_review_failed:{type(exc).__name__}",),
|
||||
)
|
||||
|
||||
# Persist the advanced state once (backlog + lineage +, with
|
||||
# persist_session_state, the consolidated facts). Skipped on a no-op tick so an
|
||||
# idle engine with nothing to learn does not churn the checkpoint.
|
||||
if did_work:
|
||||
self.checkpoint_engine_state()
|
||||
return IdleTickResult(
|
||||
candidates_contemplated=contemplated_count,
|
||||
proposals_created=created,
|
||||
pending_proposals=self._count_pending_proposals(),
|
||||
facts_consolidated=facts_consolidated,
|
||||
proposal_review=proposal_review,
|
||||
)
|
||||
store.save_manifest(self._turn_count)
|
||||
|
||||
def record_recognition_example(
|
||||
self,
|
||||
|
|
@ -1004,164 +772,9 @@ class ChatRuntime:
|
|||
|
||||
def _checkpointed_response(self, response: ChatResponse) -> ChatResponse:
|
||||
self._turn_count += 1
|
||||
# Step B — inline realization, BEFORE the checkpoint so an accrued fact is in
|
||||
# the snapshot this turn (survives reboot with persist_session_state). Gated;
|
||||
# off by default. The surface contract is unchanged (the outcome is recorded,
|
||||
# not surfaced).
|
||||
if self.config.accrue_realized_knowledge:
|
||||
self._accrue_in_turn(self._last_input_text)
|
||||
response = self._maybe_surface_determination(response)
|
||||
self.checkpoint_engine_state()
|
||||
return response
|
||||
|
||||
def _maybe_surface_determination(self, response: ChatResponse) -> ChatResponse:
|
||||
"""Step B-2 / E — select the user-facing surface from the turn's accrual.
|
||||
|
||||
B-2: when the turn DETERMINED an answer over realized knowledge, select the
|
||||
rendered determination as ``surface`` (the realizer's ``articulation_surface``
|
||||
is retained as evidence). An ``Undetermined`` turn keeps the default surface.
|
||||
|
||||
E: when the turn is ``estimated`` (a refused converse query with a calibrated
|
||||
guess) AND ``estimation_enabled``, route the guess through the ADR-0206 bridge —
|
||||
``govern_response`` widens to APPROXIMATE iff the predicate-class holds a genuine
|
||||
SERVE license, and ``shape_surface`` DISCLOSES it as ``[approximate] …``. An
|
||||
unlicensed class stays STRICT (the surface is unchanged — the honest refusal).
|
||||
Off-flag turns never reach here. See ``docs/runtime_contracts.md``.
|
||||
"""
|
||||
accrual = self._last_turn_accrual
|
||||
if accrual is None:
|
||||
return response
|
||||
if accrual.kind == "determined":
|
||||
from generate.determine import Determined, render_determination
|
||||
|
||||
if not isinstance(accrual.determination, Determined):
|
||||
return response # Undetermined → keep the default surface
|
||||
return replace(response, surface=render_determination(accrual.determination))
|
||||
if accrual.kind == "estimated" and self.config.estimation_enabled:
|
||||
return self._surface_estimate(response, accrual)
|
||||
return response
|
||||
|
||||
def _surface_estimate(self, response: ChatResponse, accrual: "TurnAccrual") -> ChatResponse:
|
||||
"""Surface a licensed converse-guess as a DISCLOSED ``[approximate]`` estimate.
|
||||
|
||||
The license gates the widening (``govern_response`` returns STRICT for an
|
||||
unlicensed class → surface unchanged); ``shape_surface`` guarantees the
|
||||
disclosure prefix because a converse guess is ``UNVERIFIED_POSSIBLE``, never in
|
||||
APPROXIMATE's admissible (fully-grounded) set. So a wrong estimate is always a
|
||||
DISCLOSED wrong — wrong=0 (silent) is preserved.
|
||||
"""
|
||||
from core.epistemic_state import EpistemicState
|
||||
from core.response_governance import ReachLevel, govern_response, shape_surface
|
||||
from generate.determine import ConverseEstimate, render_estimate
|
||||
|
||||
estimate, license_decision = accrual.estimate, accrual.license
|
||||
if not isinstance(estimate, ConverseEstimate):
|
||||
return response
|
||||
policy = govern_response(
|
||||
epistemic_state=EpistemicState.UNVERIFIED_POSSIBLE,
|
||||
license_decision=license_decision,
|
||||
)
|
||||
if policy.level is ReachLevel.STRICT:
|
||||
return response # unlicensed → no widening, honest refusal stands
|
||||
disclosed = shape_surface(
|
||||
policy,
|
||||
committed_surface=response.surface,
|
||||
decode_state=EpistemicState.UNVERIFIED_POSSIBLE,
|
||||
disclosed_alternative=render_estimate(estimate),
|
||||
)
|
||||
return replace(response, surface=disclosed, reach_level=policy.level.value)
|
||||
|
||||
def last_turn_accrual(self) -> TurnAccrual | None:
|
||||
"""The most recent turn's inline-realization outcome (Step B), or None when
|
||||
accrual is off or did not complete. Introspection only — never surfaced."""
|
||||
return self._last_turn_accrual
|
||||
|
||||
def _accrue_in_turn(self, text: str) -> None:
|
||||
"""Inline realization (Step B): a comprehensible declarative turn accrues a
|
||||
realized fact into the held self (session vault, SPECULATIVE / as-told); a
|
||||
comprehensible question turn is determined over realized knowledge. Records the
|
||||
typed outcome on ``self._last_turn_accrual``.
|
||||
|
||||
Never raises into the turn — accrual is ADDITIVE, so any failure is a clean
|
||||
no-op (the turn's response is untouched). This is SESSION memory (immediate),
|
||||
NOT ratified learning: it proposes nothing and leaves the teaching/review HITL
|
||||
path untouched. Realization writes go through ``generate.realize`` (the INV-21
|
||||
allowed vault writer); DETERMINE and the readers are total (typed results, no
|
||||
raises), so the broad guard is a defensive backstop, not expected control flow.
|
||||
"""
|
||||
self._last_turn_accrual = None
|
||||
if self._context is None or not text or not text.strip():
|
||||
return
|
||||
try:
|
||||
from generate.determine import determine
|
||||
from generate.meaning_graph.reader import Comprehension, comprehend
|
||||
from generate.meaning_graph.relational import comprehend_relational
|
||||
from generate.realize import realize_comprehension
|
||||
|
||||
if self._relational_pack_lemmas is None:
|
||||
from generate.meaning_graph.relational import load_relational_pack_lemmas
|
||||
|
||||
self._relational_pack_lemmas = load_relational_pack_lemmas()
|
||||
|
||||
readings = (
|
||||
comprehend(text),
|
||||
comprehend_relational(text, self._relational_pack_lemmas),
|
||||
)
|
||||
comprehensions = [c for c in readings if isinstance(c, Comprehension)]
|
||||
|
||||
# A question turn (query-bearing) is DETERMINED over realized knowledge.
|
||||
for c in comprehensions:
|
||||
if c.queries:
|
||||
determination = determine(c, self._context)
|
||||
self._last_turn_accrual = self._accrue_estimate_if_refused(
|
||||
c, determination
|
||||
)
|
||||
return
|
||||
# A declarative turn (a single told fact) is REALIZED into the held self.
|
||||
for c in comprehensions:
|
||||
if not c.queries and c.meaning_graph.relations:
|
||||
self._last_turn_accrual = TurnAccrual(
|
||||
kind="realized", realized=realize_comprehension(c, self._context)
|
||||
)
|
||||
return
|
||||
self._last_turn_accrual = TurnAccrual(kind="none")
|
||||
except Exception: # additive: accrual must never crash a turn # noqa: BLE001
|
||||
self._last_turn_accrual = None
|
||||
|
||||
def _accrue_estimate_if_refused(self, comprehension: Any, determination: Any) -> "TurnAccrual":
|
||||
"""Step E: turn a REFUSED converse query into an ``estimated`` accrual.
|
||||
|
||||
When DETERMINE refused (``Undetermined``) a single non-negated binary query whose
|
||||
converse was told (``p(a,b)`` realized, ``p(b,a)`` asked), produce the calibrated
|
||||
converse-guess + its SERVE license. Off-flag (or any non-converse refusal) returns
|
||||
the plain ``determined`` accrual unchanged — nothing widens. The license is only
|
||||
*attached* here; the surface decision (and the disclosure) is the bridge's, in
|
||||
``_maybe_surface_determination``.
|
||||
"""
|
||||
from generate.determine import Undetermined
|
||||
|
||||
if not (self.config.estimation_enabled and isinstance(determination, Undetermined)):
|
||||
return TurnAccrual(kind="determined", determination=determination)
|
||||
queries = getattr(comprehension, "queries", ())
|
||||
if len(queries) != 1:
|
||||
return TurnAccrual(kind="determined", determination=determination)
|
||||
query = queries[0]
|
||||
if getattr(query, "negated", False) or len(getattr(query, "arguments", ())) != 2:
|
||||
return TurnAccrual(kind="determined", determination=determination)
|
||||
|
||||
from generate.determine import estimate_converse, serve_license
|
||||
|
||||
subject, target = query.arguments[0], query.arguments[1]
|
||||
estimate = estimate_converse(self._context, query.predicate, subject, target)
|
||||
if estimate is None: # no told converse to generalize from → plain refusal
|
||||
return TurnAccrual(kind="determined", determination=determination)
|
||||
return TurnAccrual(
|
||||
kind="estimated",
|
||||
determination=determination,
|
||||
estimate=estimate,
|
||||
license=serve_license(query.predicate),
|
||||
)
|
||||
|
||||
@property
|
||||
def session(self) -> SessionContext:
|
||||
return self._context
|
||||
|
|
|
|||
20
conftest.py
20
conftest.py
|
|
@ -25,26 +25,6 @@ from __future__ import annotations
|
|||
|
||||
import pytest
|
||||
|
||||
import engine_state
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _isolate_engine_state_default(tmp_path_factory, monkeypatch):
|
||||
"""Isolate the default engine-state checkpoint dir per test.
|
||||
|
||||
A bare ``ChatRuntime()`` (no ``engine_state_path``) falls back to
|
||||
``engine_state._DEFAULT_DIR`` — the shared repo ``engine_state/`` directory.
|
||||
Tests must not share that mutable dir: one test's checkpoint (recognizers,
|
||||
candidates, the stamped engine-identity, and — under resume mode — the lived
|
||||
session_state) leaks into another test's fresh-state assumptions (and, since
|
||||
L11, raises spurious identity-continuity-break warnings when a later test
|
||||
boots under a different identity over the same dir). Point the default at a
|
||||
fresh per-test temp dir. Tests passing an explicit ``engine_state_path`` are
|
||||
unaffected; within one test, repeated ``ChatRuntime()`` share this dir.
|
||||
"""
|
||||
isolated = tmp_path_factory.mktemp("engine_state_default")
|
||||
monkeypatch.setattr(engine_state, "_DEFAULT_DIR", isolated)
|
||||
|
||||
|
||||
QUARANTINE: frozenset[str] = frozenset()
|
||||
|
||||
|
|
|
|||
|
|
@ -1,64 +0,0 @@
|
|||
"""Bit-exact (de)serialization of numpy arrays for deterministic persistence.
|
||||
|
||||
A numpy array encodes to ``{"dtype", "shape", "b64"}`` where ``b64`` is base64 of
|
||||
the array's raw bytes. This is **bit-exact**: every float round-trips with zero
|
||||
precision loss, so a restored versor keeps ``versor_condition < 1e-6`` and a
|
||||
replayed turn keeps its ``trace_hash``.
|
||||
|
||||
NEVER serialize field arrays as decimal/JSON floats. Decimal truncates the
|
||||
mantissa and silently breaks both closure and deterministic replay — the Cl(4,1)
|
||||
float-truncation pitfall. ``dtype`` carries byte order (``'<f4'``/``'<f8'``), so
|
||||
the encoding is portable, and ``float32`` is never conflated with ``float64``.
|
||||
|
||||
This module is a leaf: it imports only numpy + base64, so every layer (field,
|
||||
vault, session, engine_state) can use it without an import cycle.
|
||||
|
||||
Zig-codec follow-up (tagged — NOT authorized). This bit-exact codec is the natural
|
||||
locked **reference contract** (ADR-0196 decision rule 1) for a future Ring-1 Zig
|
||||
byte-exact serialization component: deterministic buffer ownership, stable layout, and
|
||||
edge-native build are exactly Zig's profile. It is gated behind the G0–G8 ladder and
|
||||
is **only** worth proposing AFTER (1) persistence becomes incremental/append-only
|
||||
(O(Δ)/turn — the algorithmic fix, in Python), and (2) the edge-budget gate
|
||||
(``evals/edge_budget/``) proves the bounded per-turn codec is still the device
|
||||
bottleneck. A Zig rewrite of today's O(n) snapshot would only speed up the wrong
|
||||
asymptotics. See ``evals/edge_budget/contract.md``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
def encode_array(arr: np.ndarray) -> dict[str, Any]:
|
||||
"""Encode a numpy array to a bit-exact, JSON-safe dict."""
|
||||
contiguous = np.ascontiguousarray(arr)
|
||||
return {
|
||||
"dtype": contiguous.dtype.str, # byte-order-aware, e.g. '<f4', '<f8', '<i4'
|
||||
"shape": list(contiguous.shape),
|
||||
"b64": base64.b64encode(contiguous.tobytes()).decode("ascii"),
|
||||
}
|
||||
|
||||
|
||||
def decode_array(payload: dict[str, Any]) -> np.ndarray:
|
||||
"""Decode a payload produced by ``encode_array`` back to an exact array.
|
||||
|
||||
Returns a writable copy (``np.frombuffer`` is read-only) so the restored
|
||||
array can be composed and mutated like a freshly-built one.
|
||||
"""
|
||||
dtype = np.dtype(payload["dtype"])
|
||||
raw = base64.b64decode(payload["b64"])
|
||||
flat = np.frombuffer(raw, dtype=dtype)
|
||||
return flat.reshape(payload["shape"]).copy()
|
||||
|
||||
|
||||
def encode_optional_array(arr: np.ndarray | None) -> dict[str, Any] | None:
|
||||
"""Encode an array, or return ``None`` for ``None`` (e.g. optional holonomy)."""
|
||||
return None if arr is None else encode_array(arr)
|
||||
|
||||
|
||||
def decode_optional_array(payload: dict[str, Any] | None) -> np.ndarray | None:
|
||||
"""Decode an optional-array payload, or return ``None`` for ``None``."""
|
||||
return None if payload is None else decode_array(payload)
|
||||
57
core/cli.py
57
core/cli.py
|
|
@ -114,29 +114,6 @@ _TEST_SUITES: dict[str, tuple[str, ...]] = {
|
|||
"tests/test_versor_condition_rust_parity.py",
|
||||
"tests/test_versor_apply_rust_parity.py",
|
||||
),
|
||||
"sensorium": (
|
||||
"tests/test_sensorium_compiler_delta.py",
|
||||
"tests/test_audio_compiler.py",
|
||||
"tests/test_audio_crdt_merge.py",
|
||||
"tests/test_audio_eval_gates.py",
|
||||
"tests/test_audio_pack_manifest.py",
|
||||
"tests/test_audio_sensorium_mount.py",
|
||||
"tests/test_vision_compiler.py",
|
||||
"tests/test_event_vision_compiler.py",
|
||||
"tests/test_vision_crdt_merge.py",
|
||||
"tests/test_vision_eval_gates.py",
|
||||
"tests/test_vision_sensorium_mount.py",
|
||||
"tests/test_sensorimotor_contract.py",
|
||||
"tests/test_sensorimotor_pack_manifest.py",
|
||||
"tests/test_observation_frame_contract.py",
|
||||
"tests/test_observation_frame_harness.py",
|
||||
"tests/test_environment_falsification.py",
|
||||
"tests/test_environment_falsification_eval_cli.py",
|
||||
"tests/test_witness_log_importer.py",
|
||||
"tests/test_tabletop_lab_protocol.py",
|
||||
"tests/test_sensorium_eval_cli.py",
|
||||
"tests/test_efferent_gate.py",
|
||||
),
|
||||
"pulse": (
|
||||
"tests/test_pulse_integration.py",
|
||||
"tests/test_graph_diffusion.py",
|
||||
|
|
@ -188,9 +165,6 @@ _TEST_SUITES: dict[str, tuple[str, ...]] = {
|
|||
"math": (
|
||||
"tests/test_adr_0126_train_sample_runner.py",
|
||||
),
|
||||
"deductive": (
|
||||
"tests/test_deductive_logic_entail.py",
|
||||
),
|
||||
"full": ("tests/",),
|
||||
}
|
||||
|
||||
|
|
@ -2377,8 +2351,6 @@ def cmd_eval(args: argparse.Namespace) -> int:
|
|||
"""Run an eval lane by name, or list available lanes."""
|
||||
if getattr(args, "lane", None) == "sensorium":
|
||||
return cmd_eval_sensorium(args)
|
||||
if getattr(args, "lane", None) == "environment-falsification":
|
||||
return cmd_eval_environment_falsification(args)
|
||||
if getattr(args, "lane", None) == "math-contemplation":
|
||||
return cmd_eval_math_contemplation(args)
|
||||
|
||||
|
|
@ -2519,33 +2491,6 @@ def cmd_eval_sensorium(args: argparse.Namespace) -> int:
|
|||
return 0 if report["failed"] == 0 and report["gate_closed"] else 1
|
||||
|
||||
|
||||
def cmd_eval_environment_falsification(args: argparse.Namespace) -> int:
|
||||
"""Run deterministic environmental falsification replay reports."""
|
||||
from evals.environment_falsification import build_environment_falsification_report
|
||||
|
||||
report = build_environment_falsification_report()
|
||||
|
||||
if getattr(args, "json", False):
|
||||
print(json.dumps(report, ensure_ascii=False, indent=2, sort_keys=True))
|
||||
else:
|
||||
print(f"lane : {report['lane']}")
|
||||
print(f"version : {report['version']}")
|
||||
print(f"cases : {report['total']}")
|
||||
print(f"passed : {report['passed']}")
|
||||
print(f"failed : {report['failed']}")
|
||||
print(f"report_sha256 : {report['report_sha256']}")
|
||||
|
||||
if getattr(args, "report", None):
|
||||
report_path = Path(args.report)
|
||||
report_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
report_path.write_text(
|
||||
json.dumps(report, ensure_ascii=False, indent=2, sort_keys=True)
|
||||
)
|
||||
print(f"\nreport written: {report_path}", file=sys.stderr)
|
||||
|
||||
return 0 if report["failed"] == 0 and report["expected_report_hash_ok"] else 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ADR-0172 W3 — math-contemplation CLI lane
|
||||
# ---------------------------------------------------------------------------
|
||||
|
|
@ -4981,7 +4926,7 @@ def build_parser() -> argparse.ArgumentParser:
|
|||
eval_cmd.add_argument("--report", metavar="PATH", help="write JSON report to file")
|
||||
eval_cmd.add_argument(
|
||||
"--modality",
|
||||
choices=["audio", "vision", "event-vision", "sensorimotor"],
|
||||
choices=["audio", "vision", "sensorimotor"],
|
||||
default="vision",
|
||||
help="sensorium lane modality to evaluate (default: vision)",
|
||||
)
|
||||
|
|
|
|||
|
|
@ -22,7 +22,6 @@ from field.state import FieldState
|
|||
from core.cognition.result import CognitiveTurnResult
|
||||
from core.cognition.surface_resolution import resolve_surface
|
||||
from core.cognition.trace import compute_trace_hash, hash_admissibility_trace
|
||||
from core.reasoning.adapters import evidence_from_entailment_trace
|
||||
from generate.intent import classify_compound_intent
|
||||
from generate.intent_bridge import _is_useful_surface
|
||||
from generate.intent_ratifier import (
|
||||
|
|
@ -48,7 +47,6 @@ from generate.operators import (
|
|||
multi_relation_walk,
|
||||
transitive_walk,
|
||||
)
|
||||
from generate.proof_chain import EntailmentTrace, evaluate_entailment_with_trace
|
||||
from teaching.correction import CorrectionCandidate, extract_correction
|
||||
from teaching.epistemic import EpistemicStatus
|
||||
from teaching.review import ReviewedTeachingExample, review_correction
|
||||
|
|
@ -124,7 +122,7 @@ class CognitiveTurnPipeline:
|
|||
) -> None: # runtime: ChatRuntime (no import cycle)
|
||||
self.runtime = runtime
|
||||
self._last_node_id: str | None = None
|
||||
self.teaching_store = teaching_store if teaching_store is not None else TeachingStore()
|
||||
self.teaching_store = teaching_store or TeachingStore()
|
||||
if recognizer is not None:
|
||||
self._recognizer = recognizer
|
||||
elif hasattr(runtime, "first_admitted_recognizer"):
|
||||
|
|
@ -306,8 +304,6 @@ class CognitiveTurnPipeline:
|
|||
):
|
||||
compose_surface = CognitiveTurnPipeline._render_compose_surface(compose_result)
|
||||
|
||||
entailment_trace = self._maybe_entailment_trace(intent, triples)
|
||||
|
||||
resolved = resolve_surface(
|
||||
canonical_surface=canonical,
|
||||
pre_decoration_surface=pre_decoration,
|
||||
|
|
@ -397,14 +393,13 @@ class CognitiveTurnPipeline:
|
|||
epistemic_status = proposal.epistemic_status.value if proposal is not None else ""
|
||||
walk_serialised = CognitiveTurnPipeline._serialize_operator(walk_result)
|
||||
compose_serialised = CognitiveTurnPipeline._serialize_operator(compose_result)
|
||||
entailment_serialised = CognitiveTurnPipeline._serialize_entailment_trace(
|
||||
entailment_trace
|
||||
)
|
||||
# Deterministic concatenation: walk, compose, then entailment. Empty
|
||||
# strings are dropped so unaffected turns keep existing trace bytes.
|
||||
operator_invocation = "|".join(
|
||||
s for s in (walk_serialised, compose_serialised, entailment_serialised)
|
||||
if s
|
||||
# Deterministic concatenation: walk record, then compose record.
|
||||
# Empty strings are dropped so single-operator turns keep their
|
||||
# existing trace_hash byte-for-byte.
|
||||
operator_invocation = (
|
||||
f"{walk_serialised}|{compose_serialised}"
|
||||
if compose_serialised
|
||||
else walk_serialised
|
||||
)
|
||||
# ADR-0023 — admissibility trace + ratification provenance.
|
||||
# Comb pass 2026-05-21 — direct attribute access; the fields
|
||||
|
|
@ -695,39 +690,6 @@ class CognitiveTurnPipeline:
|
|||
relation=intent.relation,
|
||||
)
|
||||
|
||||
def _maybe_entailment_trace(
|
||||
self,
|
||||
intent,
|
||||
triples: tuple[tuple[str, str, str], ...],
|
||||
) -> EntailmentTrace | None:
|
||||
"""Compile exact verification triples into propositional entailment.
|
||||
|
||||
Telemetry-only v1: the result is folded into ``operator_invocation`` and
|
||||
never changes the user-facing surface. Runs only when classification
|
||||
exposes a precise positive ``subject relation object`` shape.
|
||||
"""
|
||||
if intent.tag is not IntentTag.VERIFICATION:
|
||||
return None
|
||||
if intent.negated or not intent.relation or not intent.object:
|
||||
return None
|
||||
head = self._proof_atom(intent.subject)
|
||||
tail = self._proof_atom(intent.object)
|
||||
if not head or not tail:
|
||||
return None
|
||||
|
||||
relation = intent.relation.strip().lower()
|
||||
premises: list[str] = []
|
||||
for h, r, t in triples:
|
||||
if r.strip().lower() != relation:
|
||||
continue
|
||||
h_atom = self._proof_atom(h)
|
||||
t_atom = self._proof_atom(t)
|
||||
if h_atom and t_atom:
|
||||
premises.append(f"{h_atom} -> {t_atom}")
|
||||
if not premises:
|
||||
return None
|
||||
return evaluate_entailment_with_trace(tuple(premises), f"{head} -> {tail}")
|
||||
|
||||
@staticmethod
|
||||
def _render_compose_surface(compose: FrameComposeResult) -> str:
|
||||
"""Render a frame-transfer composition suffix without selecting authority."""
|
||||
|
|
@ -759,19 +721,6 @@ class CognitiveTurnPipeline:
|
|||
return ""
|
||||
return json.dumps(op.as_dict(), sort_keys=True, ensure_ascii=False)
|
||||
|
||||
@staticmethod
|
||||
def _serialize_entailment_trace(trace: EntailmentTrace | None) -> str:
|
||||
if trace is None:
|
||||
return ""
|
||||
return f"entailment:{evidence_from_entailment_trace(trace).canonical_json()}"
|
||||
|
||||
@staticmethod
|
||||
def _proof_atom(text: str) -> str:
|
||||
parts = [p for p in _SUBJECT_SPLIT_RE.split(text.lower()) if p]
|
||||
if not parts:
|
||||
return ""
|
||||
return "atom_" + "_".join(parts)
|
||||
|
||||
@staticmethod
|
||||
def _render_walk_surface(walk: WalkResult) -> str:
|
||||
"""Render a chain-aware walk suffix without selecting authority."""
|
||||
|
|
|
|||
|
|
@ -1,57 +0,0 @@
|
|||
"""Shared typed record of a comprehension organ's attempt at a problem (N2).
|
||||
|
||||
The normalization layer the contemplation batch (N3 router, N4 registry, N6 pass manager) reasons
|
||||
over — uniform across the R1 and R2 setup compilers. Off-serving; imports no `evals`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from core.comprehension_attempt.classify import (
|
||||
classify_cmb,
|
||||
classify_r1,
|
||||
classify_r2,
|
||||
classify_r3,
|
||||
cmb_reason,
|
||||
)
|
||||
from core.comprehension_attempt.failure_family import (
|
||||
REGISTRY,
|
||||
FailureFamily,
|
||||
enrich_family,
|
||||
family_by_name,
|
||||
family_for_reason,
|
||||
)
|
||||
from core.comprehension_attempt.model import ComprehensionAttempt, Organ, Outcome
|
||||
from core.comprehension_attempt.proposal import (
|
||||
FailureProposal,
|
||||
build_proposal,
|
||||
emit_proposal,
|
||||
)
|
||||
from core.comprehension_attempt.router import (
|
||||
RouteResult,
|
||||
RouteStatus,
|
||||
cmb_is_authoritative,
|
||||
route_setup,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"REGISTRY",
|
||||
"ComprehensionAttempt",
|
||||
"FailureFamily",
|
||||
"FailureProposal",
|
||||
"build_proposal",
|
||||
"emit_proposal",
|
||||
"Organ",
|
||||
"Outcome",
|
||||
"RouteResult",
|
||||
"RouteStatus",
|
||||
"classify_cmb",
|
||||
"classify_r1",
|
||||
"classify_r2",
|
||||
"classify_r3",
|
||||
"cmb_is_authoritative",
|
||||
"cmb_reason",
|
||||
"enrich_family",
|
||||
"family_by_name",
|
||||
"family_for_reason",
|
||||
"route_setup",
|
||||
]
|
||||
|
|
@ -1,159 +0,0 @@
|
|||
"""Normalize R1/R2 organ output into a typed ``ComprehensionAttempt`` (N2, setup-level).
|
||||
|
||||
`classify_r1` / `classify_r2` run their organ and report **produce-mode** setup outcomes:
|
||||
`setup_refused` (with the organ's typed reason) or `setup_correct` (an admissible setup was
|
||||
produced, with a deterministic signature). They do NOT solve, do NOT compare to gold, and import
|
||||
nothing from `evals` (signatures are computed inline) — keeping this a thin, dependency-light
|
||||
normalizer. Answer-level outcomes are reached downstream (N6) when the solver/verifier run.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from core.comprehension_attempt.model import ComprehensionAttempt
|
||||
from generate.combined_rate_comprehension.model import CombinedRateProblem
|
||||
from generate.combined_rate_comprehension.reader import read_combined_rate_problem
|
||||
from generate.constraint_comprehension.model import ConstraintProblem
|
||||
from generate.constraint_comprehension.reader import read_constraint_problem
|
||||
from generate.meaning_graph.reader import Refusal
|
||||
from generate.quantitative_comprehension import comprehend_quantitative, to_relational_metric
|
||||
from generate.rate_comprehension.model import RateProblem
|
||||
from generate.rate_comprehension.reader import read_rate_problem
|
||||
|
||||
#: CMB reader/solver reasons that are namespaced ``cmb_*`` before the (reason-string-keyed) failure
|
||||
#: registry sees them, so a CMB boundary never inherits R2/R3's family for the SAME bare string
|
||||
#: (R3 ``rate_unit_mismatch`` is a growth surface; R2/R3 ``non_integer_solution`` carry other
|
||||
#: owners). The two ``input_shape`` reasons stay bare so they map to the cross ``input_shape``
|
||||
#: family (router hygiene). A ``cmb_``-prefixed attempt reason is also the signal that CMB
|
||||
#: *substantively* recognized the text (used by the CMB-over-R3 precedence rule in the router).
|
||||
_CMB_BARE_REASONS = frozenset({"not_combined_rate_shaped", "empty"})
|
||||
|
||||
|
||||
def cmb_reason(reason: str) -> str:
|
||||
"""Namespace a CMB refusal reason for the failure registry (see :data:`_CMB_BARE_REASONS`)."""
|
||||
return reason if reason in _CMB_BARE_REASONS else f"cmb_{reason}"
|
||||
|
||||
|
||||
def _r1_signature(relations: list[dict[str, Any]]) -> str:
|
||||
"""Deterministic, order-independent string signature of the projected R1 relations."""
|
||||
items: list[tuple] = []
|
||||
for r in relations:
|
||||
kind = r["kind"]
|
||||
if kind == "fact":
|
||||
items.append((kind, r["entity"], int(r["value"])))
|
||||
elif kind in ("more_than", "fewer_than"):
|
||||
items.append((kind, r["entity"], r["ref"], int(r["delta"])))
|
||||
elif kind == "times_as_many":
|
||||
items.append((kind, r["entity"], r["ref"], int(r["factor"])))
|
||||
elif kind == "divide_by":
|
||||
items.append((kind, r["entity"], r["ref"], int(r["divisor"])))
|
||||
elif kind == "sum_of":
|
||||
items.append((kind, r["entity"], tuple(sorted(r["parts"]))))
|
||||
else: # pragma: no cover - defensive
|
||||
items.append(("unhandled", kind, r.get("entity", "")))
|
||||
return repr(tuple(sorted(items, key=repr)))
|
||||
|
||||
|
||||
def _r2_signature(problem: ConstraintProblem) -> str:
|
||||
"""Deterministic, order-independent string signature of an R2 ConstraintProblem setup."""
|
||||
unknowns = tuple(sorted((u.symbol, u.unit, u.domain) for u in problem.unknowns))
|
||||
constraints: list[tuple] = []
|
||||
for c in problem.constraints:
|
||||
merged: dict[str, int] = {}
|
||||
for symbol, coeff in c.lhs.terms:
|
||||
merged[symbol] = merged.get(symbol, 0) + coeff
|
||||
terms = tuple(sorted((s, v) for s, v in merged.items() if v != 0))
|
||||
constraints.append((terms, c.relation, c.rhs - c.lhs.constant))
|
||||
query = (problem.query.symbol, problem.query.unit)
|
||||
return repr((unknowns, tuple(sorted(constraints, key=repr)), query))
|
||||
|
||||
|
||||
def classify_r1(text: str, *, case_id: str | None = None) -> ComprehensionAttempt:
|
||||
"""Attempt the R1 relational-arithmetic setup compiler on *text*."""
|
||||
comp = comprehend_quantitative(text)
|
||||
if isinstance(comp, Refusal):
|
||||
return ComprehensionAttempt(
|
||||
"r1_quantitative", "setup_refused", case_id=case_id, refusal_reason=comp.reason
|
||||
)
|
||||
projected = to_relational_metric(comp)
|
||||
if projected is None:
|
||||
return ComprehensionAttempt(
|
||||
"r1_quantitative", "setup_refused", case_id=case_id, refusal_reason="unprojectable"
|
||||
)
|
||||
relations, _query = projected
|
||||
return ComprehensionAttempt(
|
||||
"r1_quantitative", "setup_correct", case_id=case_id, setup_signature=_r1_signature(relations)
|
||||
)
|
||||
|
||||
|
||||
def classify_r2(text: str, *, case_id: str | None = None) -> ComprehensionAttempt:
|
||||
"""Attempt the R2 two-category constraint setup compiler on *text*."""
|
||||
problem = read_constraint_problem(text)
|
||||
if isinstance(problem, Refusal):
|
||||
return ComprehensionAttempt(
|
||||
"r2_constraints", "setup_refused", case_id=case_id, refusal_reason=problem.reason
|
||||
)
|
||||
return ComprehensionAttempt(
|
||||
"r2_constraints", "setup_correct", case_id=case_id, setup_signature=_r2_signature(problem)
|
||||
)
|
||||
|
||||
|
||||
def _r3_signature(problem: RateProblem) -> str:
|
||||
"""Deterministic string signature of an R3 single-rate setup."""
|
||||
return repr(
|
||||
(
|
||||
(problem.rate_unit.numerator, problem.rate_unit.denominator),
|
||||
("rate", problem.rate),
|
||||
("time", problem.time),
|
||||
("quantity", problem.quantity),
|
||||
problem.query,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def classify_r3(text: str, *, case_id: str | None = None) -> ComprehensionAttempt:
|
||||
"""Attempt the R3 single-rate setup compiler on *text*."""
|
||||
problem = read_rate_problem(text)
|
||||
if isinstance(problem, Refusal):
|
||||
return ComprehensionAttempt(
|
||||
"r3_rate", "setup_refused", case_id=case_id, refusal_reason=problem.reason
|
||||
)
|
||||
return ComprehensionAttempt(
|
||||
"r3_rate", "setup_correct", case_id=case_id, setup_signature=_r3_signature(problem)
|
||||
)
|
||||
|
||||
|
||||
def _cmb_signature(problem: CombinedRateProblem) -> str:
|
||||
"""Deterministic string signature of an R4 combined-rate setup. The leading ``"cmb"`` tag
|
||||
guarantees it never coincides with an R1/R2/R3 signature (cross-organ distinctness). ``sum`` is
|
||||
commutative, so its two rates are sorted; ``difference`` keeps order (which rate is the drain)."""
|
||||
rates = (problem.rate_a, problem.rate_b)
|
||||
if problem.combine_mode == "sum":
|
||||
rates = tuple(sorted(rates))
|
||||
return repr(
|
||||
(
|
||||
"cmb",
|
||||
problem.combine_mode,
|
||||
(problem.rate_unit.numerator, problem.rate_unit.denominator),
|
||||
rates,
|
||||
("time", problem.time),
|
||||
("quantity", problem.quantity),
|
||||
problem.query,
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def classify_cmb(text: str, *, case_id: str | None = None) -> ComprehensionAttempt:
|
||||
"""Attempt the R4 combined-rate setup compiler on *text* (refusal reasons namespaced ``cmb_*``)."""
|
||||
problem = read_combined_rate_problem(text)
|
||||
if isinstance(problem, Refusal):
|
||||
return ComprehensionAttempt(
|
||||
"r4_combined_rate", "setup_refused", case_id=case_id, refusal_reason=cmb_reason(problem.reason)
|
||||
)
|
||||
return ComprehensionAttempt(
|
||||
"r4_combined_rate", "setup_correct", case_id=case_id, setup_signature=_cmb_signature(problem)
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["classify_cmb", "classify_r1", "classify_r2", "classify_r3", "cmb_reason"]
|
||||
|
|
@ -1,250 +0,0 @@
|
|||
"""Failure-family registry (N4) — the heart of the contemplation batch.
|
||||
|
||||
Partitions every typed organ refusal reason (R1 reader/admissibility, R2 reader/solver/
|
||||
answer-choice) into a named **failure family** that declares:
|
||||
|
||||
- ``owner`` — which organ surfaces it (``r1`` / ``r2`` / ``cross``)
|
||||
- ``must_remain_refused`` — is this a correct wrong=0 boundary that must stay refused?
|
||||
- ``proposal_allowed`` — is this a genuine coverage gap a proposal may target?
|
||||
- ``safe_next_action`` — the human-readable next step
|
||||
- ``proposal_target`` — what artifact a proposal would suggest (e.g. ``r2_gold_fixture``)
|
||||
|
||||
Only three families are growth surfaces (``proposal_allowed = True``): the R2 ``missing_*``
|
||||
gaps. Everything else is a correct boundary — `correct refusal != missing capability`. The
|
||||
registry is a **partition**: every reachable reason maps to exactly one family (asserted by
|
||||
test), so ``family_for_reason`` is total and unambiguous. ``answer_key_contradiction`` carries
|
||||
no refusal reason — it is reached from the answer-choice ``contradiction`` *verdict* (N6).
|
||||
|
||||
Some R1 reasons are coarse (``unreadable_quantity_clause`` covers both the pronoun and distractor
|
||||
cases; ``admissibility_refused`` covers both ungrounded and unit-incompatible). v0 folds each to a
|
||||
single conservative family — the *action* (refuse, no proposal) is identical for the folded cases,
|
||||
so no wrong=0 signal is lost. The reserved families are forward-declared for R3 with no current
|
||||
reason mapping.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, replace
|
||||
from typing import Literal
|
||||
|
||||
from core.comprehension_attempt.model import ComprehensionAttempt
|
||||
|
||||
Owner = Literal["r1", "r2", "r3", "r4", "cross"]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class FailureFamily:
|
||||
"""A named class of comprehension failure with its growth/refusal policy."""
|
||||
|
||||
name: str
|
||||
owner: Owner
|
||||
must_remain_refused: bool
|
||||
proposal_allowed: bool
|
||||
safe_next_action: str
|
||||
proposal_target: str | None = None
|
||||
refusal_reasons: tuple[str, ...] = ()
|
||||
|
||||
|
||||
#: The registry. Every reachable organ refusal reason appears in exactly one family.
|
||||
REGISTRY: tuple[FailureFamily, ...] = (
|
||||
# --- correct wrong=0 boundaries (no proposal) ---------------------------------------- #
|
||||
FailureFamily(
|
||||
"input_shape", "cross", True, False,
|
||||
"refuse — the text is not a readable problem shape",
|
||||
refusal_reasons=(
|
||||
"empty", "no_quantity_template", "non_digit_quantity", "non_identifier_name",
|
||||
"unreadable_quantity_query", "invalid_binding_graph", "query_target_not_a_category",
|
||||
"unprojectable", "category_pair_not_found", "query_target_unrecognized", "no_query",
|
||||
"not_rate_shaped", "not_combined_rate_shaped",
|
||||
),
|
||||
),
|
||||
FailureFamily(
|
||||
"unsupported_clause_shape", "r1", True, False,
|
||||
"refuse — compound/pronoun clause the template cannot isolate (subsumes "
|
||||
"ambiguous_referent + unsupported_distractor_clause until a finer signal exists)",
|
||||
refusal_reasons=("unreadable_quantity_clause",),
|
||||
),
|
||||
FailureFamily(
|
||||
"ungrounded_base", "r1", True, False,
|
||||
"refuse — the asked quantity has no grounded anchor (underdetermined)",
|
||||
refusal_reasons=("no_single_quantity_query",),
|
||||
),
|
||||
FailureFamily(
|
||||
"admissibility_incompatible", "cross", True, False,
|
||||
"refuse — operands are ungrounded or unit-incompatible (cannot combine across dimensions)",
|
||||
refusal_reasons=("admissibility_refused", "coefficient_unit_mismatch", "coefficient_conflict"),
|
||||
),
|
||||
FailureFamily(
|
||||
"over_determined", "r1", True, False,
|
||||
"refuse — structurally incoherent (multiple bases / partition mismatch)",
|
||||
refusal_reasons=(
|
||||
"multiple_inverse_bases", "multiple_partitions",
|
||||
"partition_query_mismatch", "partition_container_mismatch",
|
||||
),
|
||||
),
|
||||
FailureFamily(
|
||||
"unsupported_system_size", "r2", True, False,
|
||||
"refuse — more than two categories; needs an n-variable solver (R3)",
|
||||
refusal_reasons=("too_many_categories",),
|
||||
),
|
||||
FailureFamily(
|
||||
"indistinguishable_system", "r2", True, False,
|
||||
"refuse — the system is singular/underdetermined; no unique solution",
|
||||
refusal_reasons=("indistinguishable_weights", "query_target_unsolved", "verification_failed"),
|
||||
),
|
||||
FailureFamily(
|
||||
"non_integer_solution", "r2", True, False,
|
||||
"refuse — no integer solution exists; never round",
|
||||
refusal_reasons=("non_integer_solution",),
|
||||
),
|
||||
FailureFamily(
|
||||
"negative_solution", "r2", True, False,
|
||||
"refuse — a solved count is negative; out of domain",
|
||||
refusal_reasons=("negative_solution",),
|
||||
),
|
||||
FailureFamily(
|
||||
"answer_choice_unresolved", "r2", True, False,
|
||||
"refuse — the proven value cannot be tied to exactly one option",
|
||||
refusal_reasons=(
|
||||
"no_matching_option", "ambiguous_options", "no_options",
|
||||
"unknown_provided_label", "unparseable_option",
|
||||
),
|
||||
),
|
||||
# --- growth surfaces (proposal allowed) ---------------------------------------------- #
|
||||
FailureFamily(
|
||||
"missing_total_count", "r2", False, True,
|
||||
"propose a total-count-constraint gold fixture for review",
|
||||
proposal_target="r2_gold_fixture", refusal_reasons=("missing_total_count",),
|
||||
),
|
||||
FailureFamily(
|
||||
"missing_weighted_total", "r2", False, True,
|
||||
"propose a weighted-total-constraint gold fixture for review",
|
||||
proposal_target="r2_gold_fixture", refusal_reasons=("missing_weighted_total",),
|
||||
),
|
||||
# --- verdict (not a refusal) --------------------------------------------------------- #
|
||||
FailureFamily(
|
||||
"answer_key_contradiction", "r2", False, False,
|
||||
"report the contradiction — the proven value disagrees with the supplied key",
|
||||
refusal_reasons=(),
|
||||
),
|
||||
# --- reserved / forward-declared for R3 (no current emitter) ------------------------- #
|
||||
FailureFamily(
|
||||
"missing_category_pair", "r2", False, True,
|
||||
"RESERVED — propose a category-pair fixture once the reader distinguishes a partial "
|
||||
"(one-category) R2 problem from non-R2 text; the raw category_pair_not_found reason is "
|
||||
"too broad to propose against safely (it fires on any non-R2 text), so it maps to "
|
||||
"input_shape until that split exists",
|
||||
proposal_target="r2_gold_fixture",
|
||||
),
|
||||
FailureFamily(
|
||||
"missing_attribute_coefficient", "r2", False, True,
|
||||
"RESERVED — propose an attribute-coefficient fixture (no emitter yet)",
|
||||
proposal_target="r2_gold_fixture",
|
||||
),
|
||||
# --- R3 single-rate organ (reachable; growth vs boundary by the anti-over-propose rule) -- #
|
||||
FailureFamily(
|
||||
"unsupported_rate_duration", "r3", False, True,
|
||||
"propose a rate fixture for a recognized-but-unsupported rate feature (unit conversion / "
|
||||
"multi-rate). GROWTH surface: rate_unit_mismatch + combined_rates are emitted ONLY after a "
|
||||
"rate clause is recognized, so they are always rate-like — never arbitrary text.",
|
||||
proposal_target="r3_gold_fixture",
|
||||
refusal_reasons=("rate_unit_mismatch", "combined_rates"),
|
||||
),
|
||||
FailureFamily(
|
||||
"rate_underdetermined", "r3", True, False,
|
||||
"refuse — a single-rate problem missing a needed value (underdetermined), like ungrounded_base",
|
||||
refusal_reasons=("missing_rate", "missing_time", "missing_quantity"),
|
||||
),
|
||||
FailureFamily(
|
||||
"unsupported_temporal_state", "r3", True, False,
|
||||
"refuse — elapsed clock-time is R3.x. NOT a growth surface: the clock-marker detector can "
|
||||
"fire on non-rate text, so temporal_state is not reliably rate-like.",
|
||||
refusal_reasons=("temporal_state",),
|
||||
),
|
||||
# --- R4 combined-rate organ (reasons namespaced cmb_*; CMB owns them over R3's same-string ---- #
|
||||
# reasons via the CMB-over-R3 precedence rule in the router) ------------------------------ #
|
||||
FailureFamily(
|
||||
"cmb_unit_mismatch", "r4", True, False,
|
||||
"refuse — the two rates' units are incompatible. must_remain_refused UNTIL a dimension "
|
||||
"registry exists: CMB v1 cannot distinguish a convertible pair (gallons/min vs gallons/hour) "
|
||||
"from a dimensionally-incompatible one (rooms/hour vs liters/minute), so a 'try conversion' "
|
||||
"proposal would be wrong=0-unsafe. NOT 'forever impossible' — a convertible split is future work.",
|
||||
refusal_reasons=("cmb_rate_unit_mismatch",),
|
||||
),
|
||||
FailureFamily(
|
||||
"cmb_combine_ambiguous", "r4", True, False,
|
||||
"refuse — two same-unit rates but no licensed sum/difference cue. An ambiguity, not a "
|
||||
"coverage gap: no fixture can teach which mode the text intends.",
|
||||
refusal_reasons=("cmb_combine_mode_ambiguous",),
|
||||
),
|
||||
FailureFamily(
|
||||
"cmb_underdetermined", "r4", True, False,
|
||||
"refuse — combined-shaped but a contributing rate is unstated (under-specified input); no "
|
||||
"reader enhancement can infer the missing rate.",
|
||||
refusal_reasons=("cmb_missing_second_rate",),
|
||||
),
|
||||
FailureFamily(
|
||||
"cmb_non_positive_net", "r4", True, False,
|
||||
"refuse — non-positive net rate; a quantity/time query cannot make progress (solver boundary).",
|
||||
refusal_reasons=("cmb_non_positive_net_rate",),
|
||||
),
|
||||
FailureFamily(
|
||||
"cmb_non_integer", "r4", True, False,
|
||||
"refuse — no exact integer answer; never round (solver boundary, exact-integer v1).",
|
||||
refusal_reasons=("cmb_non_integer_solution",),
|
||||
),
|
||||
# --- R4 growth surfaces (proposal allowed) — emitted ONLY after positive combined recognition -- #
|
||||
FailureFamily(
|
||||
"cmb_unsupported_rate_count", "r4", False, True,
|
||||
"propose a combined-rate fixture for ≥3 contributing rates (future capability)",
|
||||
proposal_target="cmb_gold_fixture", refusal_reasons=("cmb_three_or_more_rates",),
|
||||
),
|
||||
FailureFamily(
|
||||
"cmb_unsupported_reciprocal", "r4", False, True,
|
||||
"propose a reciprocal work-rate fixture (1/(1/a+1/b); reciprocal rates + rational arithmetic)",
|
||||
proposal_target="cmb_gold_fixture", refusal_reasons=("cmb_reciprocal_work_rate_deferred",),
|
||||
),
|
||||
FailureFamily(
|
||||
"cmb_unsupported_clock_interval", "r4", False, True,
|
||||
"propose a clock-interval fixture (elapsed-clock-time duration in a combined-rate problem)",
|
||||
proposal_target="cmb_gold_fixture", refusal_reasons=("cmb_clock_interval_deferred",),
|
||||
),
|
||||
)
|
||||
|
||||
_BY_NAME: dict[str, FailureFamily] = {f.name: f for f in REGISTRY}
|
||||
_BY_REASON: dict[str, FailureFamily] = {
|
||||
reason: family for family in REGISTRY for reason in family.refusal_reasons
|
||||
}
|
||||
|
||||
#: The verdict-derived family (no refusal reason maps to it).
|
||||
ANSWER_KEY_CONTRADICTION = _BY_NAME["answer_key_contradiction"]
|
||||
|
||||
|
||||
def family_for_reason(reason: str | None) -> FailureFamily | None:
|
||||
"""The single failure family a typed organ refusal reason belongs to (or ``None``)."""
|
||||
if reason is None:
|
||||
return None
|
||||
return _BY_REASON.get(reason)
|
||||
|
||||
|
||||
def family_by_name(name: str) -> FailureFamily | None:
|
||||
return _BY_NAME.get(name)
|
||||
|
||||
|
||||
def enrich_family(attempt: ComprehensionAttempt) -> ComprehensionAttempt:
|
||||
"""Return *attempt* with its ``family`` resolved from its refusal reason (or unchanged)."""
|
||||
family = family_for_reason(attempt.refusal_reason)
|
||||
if family is None:
|
||||
return attempt
|
||||
return replace(attempt, family=family.name)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ANSWER_KEY_CONTRADICTION",
|
||||
"FailureFamily",
|
||||
"Owner",
|
||||
"REGISTRY",
|
||||
"enrich_family",
|
||||
"family_by_name",
|
||||
"family_for_reason",
|
||||
]
|
||||
|
|
@ -1,63 +0,0 @@
|
|||
"""Typed, immutable record of one comprehension organ's attempt at a problem (N2).
|
||||
|
||||
A small normalization layer over the R1 (`generate.quantitative_comprehension`) and R2
|
||||
(`generate.constraint_comprehension`) setup compilers: it turns each organ's heterogeneous
|
||||
output (a typed setup, or a typed `Refusal`) into one uniform, frozen `ComprehensionAttempt`.
|
||||
Nothing here changes reader behavior — it only *describes* an outcome so the router (N3),
|
||||
failure-family registry (N4), and contemplation pass manager (N6) can reason over both organs
|
||||
uniformly.
|
||||
|
||||
Outcome semantics. `classify` (N2) produces **produce-mode** outcomes — what the organ did on
|
||||
its own gates, with no gold in hand: `setup_refused` (the organ refused) or `setup_correct`
|
||||
(an admissible setup was produced). The gold-relative outcomes (`setup_wrong`, `answer_wrong`)
|
||||
are representable here but are emitted only in **eval mode** by the lanes that hold gold — never
|
||||
fabricated by `classify`. `answer_*` / `contradiction` are reached when the solver / answer-choice
|
||||
verifier run downstream (N6), not at setup classification time.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal
|
||||
|
||||
from generate.binding_graph.model import SourceSpanLink
|
||||
|
||||
Organ = Literal["r1_quantitative", "r2_constraints", "r3_rate", "r4_combined_rate"]
|
||||
|
||||
Outcome = Literal[
|
||||
"setup_correct", # an admissible setup was produced (produce-mode) / matches gold (eval-mode)
|
||||
"setup_refused", # the organ refused to assemble a setup
|
||||
"setup_wrong", # eval-mode only: produced setup diverges from gold (a wrong=0 breach)
|
||||
"answer_correct", # a value was produced and self-verified / matches gold
|
||||
"answer_refused", # setup produced but the solver/verifier refused
|
||||
"answer_wrong", # eval-mode only: produced value diverges from gold
|
||||
"contradiction", # a verified value contradicts a supplied answer key
|
||||
]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ComprehensionAttempt:
|
||||
"""One organ's attempt at one problem. Immutable; carries the outcome, the refusal reason
|
||||
(if any), a deterministic setup signature (for cross-organ comparison), the answer (if a
|
||||
value was produced), and source-span evidence. ``family`` is left ``None`` by ``classify``
|
||||
and resolved later by the N4 failure-family registry."""
|
||||
|
||||
organ: Organ
|
||||
outcome: Outcome
|
||||
case_id: str | None = None
|
||||
refusal_reason: str | None = None
|
||||
family: str | None = None
|
||||
setup_signature: str | None = None
|
||||
answer: int | None = None
|
||||
evidence: tuple[SourceSpanLink, ...] = ()
|
||||
|
||||
@property
|
||||
def is_setup_correct(self) -> bool:
|
||||
return self.outcome == "setup_correct"
|
||||
|
||||
@property
|
||||
def is_refusal(self) -> bool:
|
||||
return self.outcome in ("setup_refused", "answer_refused")
|
||||
|
||||
|
||||
__all__ = ["ComprehensionAttempt", "Organ", "Outcome"]
|
||||
|
|
@ -1,148 +0,0 @@
|
|||
"""Proposal-only failure-artifact emitter (N5) — deliberately toothless.
|
||||
|
||||
When the contemplation pass (N6) meets a **growth-surface** failure family (``proposal_allowed
|
||||
= True``), it may emit a single content-addressed JSON artifact under
|
||||
``teaching/proposals/comprehension_failures/<hash>.json``. That artifact can *propose* a next
|
||||
fixture/rule for human review. It can do nothing else:
|
||||
|
||||
```text
|
||||
status is always "proposal_only"
|
||||
mounted is always false
|
||||
requires_review is always true
|
||||
serving never reads these files
|
||||
no reader/test is modified
|
||||
```
|
||||
|
||||
This routes into the existing proposal-only teaching flywheel (ADR-0055/0056/0057) — it is NOT a
|
||||
parallel correction path (CLAUDE.md teaching-safety). The alignment is
|
||||
``failure -> classification -> proposal -> review -> ratification``, never ``failure -> self-patch``.
|
||||
|
||||
Content-addressing: the filename is ``sha256(failure_family : sha256(problem_text))`` — so the
|
||||
same failure on the same text always writes the same path (idempotent), and the raw problem text
|
||||
is **hashed, never stored**. Deterministic; no clock, no randomness.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from core.comprehension_attempt.failure_family import FailureFamily
|
||||
from core.comprehension_attempt.model import ComprehensionAttempt
|
||||
|
||||
_PROPOSAL_STATUS = "proposal_only"
|
||||
|
||||
|
||||
def default_proposal_root() -> Path:
|
||||
"""``<repo>/teaching/proposals/comprehension_failures`` — the write-only proposal sink."""
|
||||
return Path(__file__).resolve().parents[2] / "teaching" / "proposals" / "comprehension_failures"
|
||||
|
||||
|
||||
def _sha256(text: str) -> str:
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _attempt_to_dict(attempt: ComprehensionAttempt) -> dict[str, Any]:
|
||||
return {
|
||||
"organ": attempt.organ,
|
||||
"outcome": attempt.outcome,
|
||||
"refusal_reason": attempt.refusal_reason,
|
||||
"family": attempt.family,
|
||||
"setup_signature": attempt.setup_signature,
|
||||
"answer": attempt.answer,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class FailureProposal:
|
||||
"""A proposal-only artifact. The invariant fields (``status``/``requires_review``/``mounted``)
|
||||
are enforced in ``__post_init__`` so even a hand-constructed proposal cannot be made
|
||||
serving-mountable."""
|
||||
|
||||
failure_family: str
|
||||
problem_text_sha256: str
|
||||
observed_attempts: tuple[dict[str, Any], ...]
|
||||
status: str = _PROPOSAL_STATUS
|
||||
suggested_next_fixture: None = None # v0: always None — a human authors the fixture on review
|
||||
requires_review: bool = True
|
||||
mounted: bool = False
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.status != _PROPOSAL_STATUS:
|
||||
raise ValueError(f"proposal status must be {_PROPOSAL_STATUS!r}; got {self.status!r}")
|
||||
if self.mounted:
|
||||
raise ValueError("a proposal can never be mounted")
|
||||
if not self.requires_review:
|
||||
raise ValueError("a proposal always requires review")
|
||||
|
||||
def content_hash(self) -> str:
|
||||
"""Deterministic content address: same failure on same text -> same hash."""
|
||||
return hashlib.sha256(
|
||||
f"{self.failure_family}:{self.problem_text_sha256}".encode("utf-8")
|
||||
).hexdigest()
|
||||
|
||||
def to_json_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"status": self.status,
|
||||
"failure_family": self.failure_family,
|
||||
"problem_text_sha256": self.problem_text_sha256,
|
||||
"observed_attempts": list(self.observed_attempts),
|
||||
"suggested_next_fixture": self.suggested_next_fixture,
|
||||
"requires_review": self.requires_review,
|
||||
"mounted": self.mounted,
|
||||
}
|
||||
|
||||
|
||||
def build_proposal(
|
||||
text: str, family: FailureFamily, attempts: tuple[ComprehensionAttempt, ...]
|
||||
) -> FailureProposal | None:
|
||||
"""Build a proposal for a growth-surface family, or ``None`` for a correct wrong=0 boundary.
|
||||
|
||||
A family with ``proposal_allowed = False`` (every must-remain-refused boundary) yields NO
|
||||
proposal — the loop never proposes against a faithful refusal.
|
||||
"""
|
||||
if not family.proposal_allowed:
|
||||
return None
|
||||
return FailureProposal(
|
||||
failure_family=family.name,
|
||||
problem_text_sha256=_sha256(text),
|
||||
observed_attempts=tuple(_attempt_to_dict(a) for a in attempts),
|
||||
)
|
||||
|
||||
|
||||
def proposal_path(proposal: FailureProposal, root: Path | None = None) -> Path:
|
||||
base = root if root is not None else default_proposal_root()
|
||||
return base / f"{proposal.content_hash()}.json"
|
||||
|
||||
|
||||
def emit_proposal(
|
||||
text: str,
|
||||
family: FailureFamily,
|
||||
attempts: tuple[ComprehensionAttempt, ...],
|
||||
*,
|
||||
root: Path | None = None,
|
||||
) -> Path | None:
|
||||
"""Write a proposal-only artifact for a growth-surface family; return its path, or ``None``.
|
||||
|
||||
Idempotent: the same failure on the same text writes the same content-addressed path with
|
||||
byte-identical content (``sort_keys``). Creates the sink directory on demand.
|
||||
"""
|
||||
proposal = build_proposal(text, family, attempts)
|
||||
if proposal is None:
|
||||
return None
|
||||
path = proposal_path(proposal, root)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(proposal.to_json_dict(), indent=2, sort_keys=True), encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
__all__ = [
|
||||
"FailureProposal",
|
||||
"build_proposal",
|
||||
"default_proposal_root",
|
||||
"emit_proposal",
|
||||
"proposal_path",
|
||||
]
|
||||
|
|
@ -1,77 +0,0 @@
|
|||
"""Deterministic multi-organ setup router (N3).
|
||||
|
||||
Boring on purpose: attempt the R1 and R2 setup compilers, collect their typed attempts, and
|
||||
select a setup ONLY when exactly one organ produced an admissible one. No dynamic "best"
|
||||
scoring, no priority heuristics.
|
||||
|
||||
```text
|
||||
exactly one setup_correct -> routed (use it)
|
||||
zero setup_correct -> all_refused (classify downstream)
|
||||
>= 2 setup_correct, signatures agree-> routed (organs concur)
|
||||
>= 2 setup_correct, signatures differ-> ambiguous (refuse — never pick)
|
||||
```
|
||||
|
||||
Cross-organ signatures are produced by different functions and never coincide, so in practice
|
||||
two admitting organs always resolve to ``ambiguous``. The router never solves and never emits
|
||||
``setup_wrong`` — that is an eval-only outcome; against gold the routed setup must match (the
|
||||
wrong=0 invariant, asserted by the router tests).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Literal
|
||||
|
||||
from core.comprehension_attempt.classify import classify_cmb, classify_r1, classify_r2, classify_r3
|
||||
from core.comprehension_attempt.model import ComprehensionAttempt
|
||||
|
||||
RouteStatus = Literal["routed", "all_refused", "ambiguous"]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RouteResult:
|
||||
"""The outcome of routing one problem across the organs: every attempt, the selected setup
|
||||
(or ``None``), and the routing status."""
|
||||
|
||||
attempts: tuple[ComprehensionAttempt, ...]
|
||||
selected: ComprehensionAttempt | None
|
||||
status: RouteStatus
|
||||
|
||||
|
||||
def cmb_is_authoritative(attempts: tuple[ComprehensionAttempt, ...]) -> bool:
|
||||
"""True when the R4 combined-rate organ **positively** recognized combined-rate shape — a setup,
|
||||
or a substantive ``cmb_*`` refusal (NOT the bare ``not_combined_rate_shaped`` / ``empty``
|
||||
step-aside). When it does, R3's broader single-rate read of the SAME text is inadmissible: a more
|
||||
specific positive recognition beats a broader partial one ("Anna and Ben paint together; Anna
|
||||
paints 3 rooms/hour" must not be answered as a single-rate 12). This is the narrow CMB↔R3
|
||||
instance of a future general domain-specificity adjudication; it does NOT mean "CMB always wins"
|
||||
(on ``input_shape`` CMB cedes to R3, e.g. a plain single-rate car problem)."""
|
||||
cmb = next((a for a in attempts if a.organ == "r4_combined_rate"), None)
|
||||
if cmb is None:
|
||||
return False
|
||||
return cmb.is_setup_correct or (cmb.refusal_reason or "").startswith("cmb_")
|
||||
|
||||
|
||||
def route_setup(text: str, *, case_id: str | None = None) -> RouteResult:
|
||||
"""Route *text* to the single organ that admits an honest setup, or refuse."""
|
||||
attempts = (
|
||||
classify_r1(text, case_id=case_id),
|
||||
classify_r2(text, case_id=case_id),
|
||||
classify_r3(text, case_id=case_id),
|
||||
classify_cmb(text, case_id=case_id),
|
||||
)
|
||||
# CMB-over-R3 domain precedence: a substantive CMB recognition vetoes R3's single-rate over-read.
|
||||
vetoed = cmb_is_authoritative(attempts)
|
||||
correct = tuple(
|
||||
a for a in attempts if a.is_setup_correct and not (vetoed and a.organ == "r3_rate")
|
||||
)
|
||||
if len(correct) == 1:
|
||||
return RouteResult(attempts, correct[0], "routed")
|
||||
if not correct:
|
||||
return RouteResult(attempts, None, "all_refused")
|
||||
if len({a.setup_signature for a in correct}) == 1:
|
||||
return RouteResult(attempts, correct[0], "routed")
|
||||
return RouteResult(attempts, None, "ambiguous")
|
||||
|
||||
|
||||
__all__ = ["RouteResult", "RouteStatus", "cmb_is_authoritative", "route_setup"]
|
||||
|
|
@ -276,74 +276,6 @@ class RuntimeConfig:
|
|||
# ADR-0151 — generate TeachingChainProposals from enriched candidates on load.
|
||||
auto_proposal_enabled: bool = False
|
||||
|
||||
# Shape B+ (L10 resume-as-same-life) — persist and restore the FULL lived
|
||||
# session state (field, vault, anchor, graph, referents, dialogue) across
|
||||
# reboot, not just recognizers/candidates (Shape B). OFF by default: it is a
|
||||
# deliberate always-on-runtime mode, and per-turn snapshotting has an O(turns)
|
||||
# cost, so demos/evals/one-shot runtimes must not pay for resume they don't
|
||||
# use. Enabled by the L10 continuity lane and the production L10 process.
|
||||
persist_session_state: bool = False
|
||||
|
||||
# L11 — on reboot, if the stamped checkpoint identity != the recomputed
|
||||
# engine identity (the ratified substrate changed during downtime), REFUSE to
|
||||
# start (raise IdentityContinuityError) rather than the default warn-and-flag.
|
||||
# OFF by default: reboot is recovery, not control flow (ADR-0157), and the
|
||||
# operator must not be bricked by a benign ratified pack swap. Deployments
|
||||
# wanting a hard identity-continuity guarantee opt in.
|
||||
strict_identity_continuity: bool = False
|
||||
|
||||
# Step B (inline realization) — when on, each turn ACCRUES knowledge into the
|
||||
# held self: a comprehensible declarative turn is realized into the session vault
|
||||
# (SPECULATIVE, as-told), and a comprehensible question turn is determined over
|
||||
# realized knowledge. This is the "one continuous life" telos made real — a
|
||||
# conversation accumulates knowledge it can recall and reason over. OFF by default
|
||||
# (one-shot/eval runtimes don't accrue); the production L10 process enables it
|
||||
# alongside persist_session_state so accrued facts survive reboot. Realization is
|
||||
# SESSION memory (immediate), NOT ratified learning — it proposes nothing; the
|
||||
# teaching/review HITL path is untouched.
|
||||
accrue_realized_knowledge: bool = False
|
||||
|
||||
# Step D (CLOSE) — when on, idle_tick consolidates soundly-derived determinations
|
||||
# back into the held self: one semi-naive layer of the member/subset deductive
|
||||
# closure (member∘subset, subset∘subset — NEVER member∘member) per tick, each
|
||||
# derived edge proof_chain-verified, written as a SPECULATIVE realized record with
|
||||
# derived-provenance. This is how the loop "learns from determined facts": derive
|
||||
# once, remember, reach one hop further next tick — the directly answerable set
|
||||
# climbs monotonically to the deductive-closure fixed point. OFF by default; the
|
||||
# production L10 process enables it alongside accrue_realized_knowledge +
|
||||
# persist_session_state. SESSION memory (immediate), NOT reviewed/corpus learning —
|
||||
# no proposal coupling, the teaching/review HITL path is untouched. Bounded by the
|
||||
# same _SUBSUMPTION_SUBSET_FACT_BUDGET; converges (a saturated tick is a no-op).
|
||||
consolidate_determinations: bool = False
|
||||
|
||||
# IT (proposal-review surface) — when on, idle_tick runs a READ-ONLY sub-pass that scans
|
||||
# the comprehension-failure proposal sink and surfaces a summary in
|
||||
# IdleTickResult.proposal_review. It NEVER mutates an artifact, sets did_work, checkpoints,
|
||||
# ratifies, mounts, or modifies readers; a reporter failure is captured, not propagated.
|
||||
# OFF by default — idle ticks don't pay for the scan unless a deployment wants the surface.
|
||||
review_pending_proposals: bool = False
|
||||
|
||||
# Step E (ESTIMATION) — when on, a converse query the engine would otherwise REFUSE
|
||||
# (told p(a,b), asked p(b,a)) may be answered with a DISCLOSED [approximate] estimate
|
||||
# IF the predicate-class has earned the SERVE license on the ratified, committed
|
||||
# reliability ledger (ADR-0175 license_for, θ_SERVE=0.99) — routed through the
|
||||
# ADR-0206 govern_response/shape_surface bridge. OFF by default; only meaningful with
|
||||
# accrue_realized_knowledge (the estimate is computed in the accrual path). wrong=0 is
|
||||
# preserved by construction: an estimate is ALWAYS disclosed ([approximate]), never
|
||||
# asserted as fact, and is offered only for a class whose committed track record
|
||||
# clears the Wilson floor. Absent a cleared license -> STRICT refuse (the safe
|
||||
# default); the engine never raises its own ceiling.
|
||||
estimation_enabled: bool = False
|
||||
|
||||
# ASK serving gate enable flag. When True, ASK serving is allowed.
|
||||
# Default False (dark).
|
||||
ask_serving_enabled: bool = False
|
||||
|
||||
# VERIFIED serving gate enable flag. When True, VERIFIED serving is allowed.
|
||||
# Default False (dark).
|
||||
verified_serving_enabled: bool = False
|
||||
|
||||
|
||||
|
||||
DEFAULT_IDENTITY_PACK: str = "default_general_v1"
|
||||
DEFAULT_ETHICS_PACK: str = "default_general_ethics_v1"
|
||||
|
|
|
|||
|
|
@ -2,10 +2,8 @@
|
|||
|
||||
from .contradiction_detection import mine_contradiction_detection_report
|
||||
from .frontier_compare import mine_frontier_compare_report
|
||||
from .learning_arena import mine_learning_arena_report
|
||||
|
||||
__all__ = [
|
||||
"mine_contradiction_detection_report",
|
||||
"mine_frontier_compare_report",
|
||||
"mine_learning_arena_report",
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,111 +0,0 @@
|
|||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from core.contemplation.schema import (
|
||||
ContemplationEvidenceRef,
|
||||
ContemplationFinding,
|
||||
FindingKind,
|
||||
)
|
||||
|
||||
|
||||
def mine_learning_arena_report(
|
||||
report_path: str | Path,
|
||||
*,
|
||||
substrate_hash: str,
|
||||
min_coverage: float = 0.25,
|
||||
) -> tuple[ContemplationFinding, ...]:
|
||||
"""Convert learning-arena report weaknesses into speculative findings.
|
||||
|
||||
Read-only: parses ``report_path`` and returns immutable findings. It never
|
||||
writes packs, teaching examples, proposals, or runtime state.
|
||||
"""
|
||||
path = Path(report_path)
|
||||
report = json.loads(path.read_text(encoding="utf-8"))
|
||||
source_id = str(path)
|
||||
findings: list[ContemplationFinding] = []
|
||||
|
||||
for class_name, row in sorted(_per_class(report).items()):
|
||||
committed = int(row.get("committed", 0) or 0)
|
||||
coverage = float(row.get("coverage", 0.0) or 0.0)
|
||||
t2_verified = int(row.get("t2_verified", 0) or 0)
|
||||
if coverage < min_coverage:
|
||||
evidence = ContemplationEvidenceRef(
|
||||
source_type="learning_arena_report",
|
||||
source_id=source_id,
|
||||
pointer=f"class={class_name}",
|
||||
summary=f"coverage={coverage};committed={committed};min={min_coverage}",
|
||||
)
|
||||
findings.append(
|
||||
ContemplationFinding(
|
||||
kind=FindingKind.COVERAGE_GAP,
|
||||
subject=class_name,
|
||||
predicate="weak_coverage",
|
||||
object=None,
|
||||
evidence_refs=(evidence,),
|
||||
proposed_action=(
|
||||
"Inspect refusal diagnoses for this capability class and add "
|
||||
"practice or reviewed operators only where the missing piece is named."
|
||||
),
|
||||
substrate_hash=substrate_hash,
|
||||
)
|
||||
)
|
||||
if committed > 0 and t2_verified == 0:
|
||||
evidence = ContemplationEvidenceRef(
|
||||
source_type="learning_arena_report",
|
||||
source_id=source_id,
|
||||
pointer=f"class={class_name}",
|
||||
summary=f"committed={committed};t2_verified=0",
|
||||
)
|
||||
findings.append(
|
||||
ContemplationFinding(
|
||||
kind=FindingKind.UNPROVED_RELATION,
|
||||
subject=class_name,
|
||||
predicate="missing_tier2_evidence",
|
||||
object=None,
|
||||
evidence_refs=(evidence,),
|
||||
proposed_action=(
|
||||
"Add convergent self-verification evidence before promoting this "
|
||||
"class beyond gold-scored practice."
|
||||
),
|
||||
substrate_hash=substrate_hash,
|
||||
)
|
||||
)
|
||||
|
||||
for record in report.get("elimination_records", ()) or ():
|
||||
if not isinstance(record, dict):
|
||||
continue
|
||||
case_id = str(record.get("case_id", "unknown_case"))
|
||||
class_name = str(record.get("class_name", "unknown_class"))
|
||||
evidence = ContemplationEvidenceRef(
|
||||
source_type="learning_arena_report",
|
||||
source_id=source_id,
|
||||
pointer=f"case={case_id}",
|
||||
summary=str(record.get("reason", "wrong attempt")),
|
||||
)
|
||||
findings.append(
|
||||
ContemplationFinding(
|
||||
kind=FindingKind.BENCHMARK_CASE,
|
||||
subject=f"{class_name}/{case_id}",
|
||||
predicate="wrong_attempt",
|
||||
object=str(record.get("gold", "")),
|
||||
evidence_refs=(evidence,),
|
||||
proposed_action=(
|
||||
"Use this gold-caught wrong attempt as elimination evidence; do not "
|
||||
"promote the class until the faulty derivation shape is pruned."
|
||||
),
|
||||
substrate_hash=substrate_hash,
|
||||
)
|
||||
)
|
||||
|
||||
return tuple(findings)
|
||||
|
||||
|
||||
def _per_class(report: dict[str, Any]) -> dict[str, dict[str, Any]]:
|
||||
rows = report.get("per_class", {})
|
||||
return {str(k): v for k, v in rows.items() if isinstance(v, dict)}
|
||||
|
||||
|
||||
__all__ = ["mine_learning_arena_report"]
|
||||
|
|
@ -1,134 +0,0 @@
|
|||
"""EngineIdentity — content-derived identity of the ratified substrate (L11).
|
||||
|
||||
``EngineIdentity`` is the sha256 of the canonical serialization of the engine's
|
||||
ratified PERSONALITY substrate — the active identity / safety / ethics / register
|
||||
/ anchor-lens pack files — plus the code revision. It is **content-derived, NOT
|
||||
entropy-based**: two engines with the same ratified substrate compute the SAME
|
||||
identity, because they ARE the same engine functionally (substrate is shareable).
|
||||
|
||||
This is the "who am I" hash. It is bumped only by a ratified change to the
|
||||
identity substrate (a new identity pack, a safety-axis change), NOT by lived
|
||||
learning (recall, teaching) — that is the engine's *experience*, carried across
|
||||
reboot by the Shape B+ lived-state lineage, not its identity. The git-like
|
||||
lineage chain (parent links on ratification) and the reboot identity verification
|
||||
build on this primitive.
|
||||
|
||||
Honest scope (per the EngineIdentity candidate note): this is a *convention*
|
||||
naming the existing per-pack content hashes, not a new pack format. The ratified
|
||||
teaching corpus / recognizer-registry head can be folded into the tuple later
|
||||
(additive — a new key changes the identity, which is the correct semantic for a
|
||||
ratified-substrate change).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from core.config import (
|
||||
DEFAULT_ANCHOR_LENS,
|
||||
DEFAULT_ETHICS_PACK,
|
||||
DEFAULT_IDENTITY_PACK,
|
||||
DEFAULT_REGISTER_PACK,
|
||||
RuntimeConfig,
|
||||
)
|
||||
|
||||
# The never-swappable safety pack default (packs/safety/loader.py).
|
||||
DEFAULT_SAFETY_PACK: str = "core_safety_axes_v1"
|
||||
|
||||
_PACKS_ROOT = Path(__file__).resolve().parents[1] / "packs"
|
||||
|
||||
#: role -> packs/ subdirectory holding ``<pack_id>.json``.
|
||||
_ROLE_DIRS: dict[str, str] = {
|
||||
"identity": "identity",
|
||||
"safety": "safety",
|
||||
"ethics": "ethics",
|
||||
"register": "register",
|
||||
"anchor_lens": "anchor_lens",
|
||||
}
|
||||
|
||||
#: The ratified roles that constitute the engine's identity, in canonical order.
|
||||
RATIFIED_ROLES: tuple[str, ...] = (
|
||||
"identity",
|
||||
"safety",
|
||||
"ethics",
|
||||
"register",
|
||||
"anchor_lens",
|
||||
)
|
||||
|
||||
|
||||
class EngineIdentityError(RuntimeError):
|
||||
"""A ratified pack named by the identity tuple could not be resolved."""
|
||||
|
||||
|
||||
class IdentityContinuityError(RuntimeError):
|
||||
"""A reboot found the stamped checkpoint identity != the recomputed identity.
|
||||
|
||||
The ratified substrate changed while the engine was down, so it would resume
|
||||
the lived state under a DIFFERENT identity than the checkpoint was written
|
||||
under. Raised only under strict identity continuity; the default surfaces a
|
||||
warning and a queryable break flag (reboot is recovery, not control flow).
|
||||
"""
|
||||
|
||||
|
||||
def _pack_content_hash(role: str, pack_id: str) -> str:
|
||||
path = _PACKS_ROOT / _ROLE_DIRS[role] / f"{pack_id}.json"
|
||||
if not path.exists():
|
||||
raise EngineIdentityError(
|
||||
f"ratified {role} pack not found: {_ROLE_DIRS[role]}/{pack_id}.json"
|
||||
)
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def ratified_substrate(pack_ids: dict[str, str], git_revision: str) -> dict[str, Any]:
|
||||
"""The canonical, auditable identity tuple.
|
||||
|
||||
``{role: {"pack_id", "sha256"}}`` for every ratified role plus
|
||||
``"code_revision"``. ``pack_ids`` must name a pack for every role in
|
||||
``RATIFIED_ROLES``.
|
||||
"""
|
||||
substrate: dict[str, Any] = {}
|
||||
for role in RATIFIED_ROLES:
|
||||
pack_id = pack_ids[role]
|
||||
substrate[role] = {
|
||||
"pack_id": pack_id,
|
||||
"sha256": _pack_content_hash(role, pack_id),
|
||||
}
|
||||
substrate["code_revision"] = git_revision
|
||||
return substrate
|
||||
|
||||
|
||||
def compute_engine_identity(pack_ids: dict[str, str], git_revision: str) -> str:
|
||||
"""The sha256 EngineIdentity over the ratified substrate tuple."""
|
||||
canonical = json.dumps(
|
||||
ratified_substrate(pack_ids, git_revision),
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
ensure_ascii=False,
|
||||
)
|
||||
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def _resolve_pack_ids(config: RuntimeConfig) -> dict[str, str]:
|
||||
"""Resolve each ratified role to its active pack id (config override or DEFAULT)."""
|
||||
return {
|
||||
"identity": config.identity_pack or DEFAULT_IDENTITY_PACK,
|
||||
"safety": DEFAULT_SAFETY_PACK, # never-swappable
|
||||
"ethics": config.ethics_pack or DEFAULT_ETHICS_PACK,
|
||||
"register": config.register_pack_id or DEFAULT_REGISTER_PACK,
|
||||
"anchor_lens": config.anchor_lens_id or DEFAULT_ANCHOR_LENS,
|
||||
}
|
||||
|
||||
|
||||
def engine_identity_for_config(config: RuntimeConfig, git_revision: str) -> str:
|
||||
"""EngineIdentity for a runtime config (resolving every role to its active pack)."""
|
||||
return compute_engine_identity(_resolve_pack_ids(config), git_revision)
|
||||
|
||||
|
||||
def ratified_substrate_for_config(
|
||||
config: RuntimeConfig, git_revision: str
|
||||
) -> dict[str, Any]:
|
||||
"""The auditable identity tuple for a runtime config."""
|
||||
return ratified_substrate(_resolve_pack_ids(config), git_revision)
|
||||
|
|
@ -1,87 +0,0 @@
|
|||
"""The Epistemic Disclosure spine (P0-1+).
|
||||
|
||||
CORE's served-surface governance: comprehension produces evidence, the limitation
|
||||
pass classifies *what kind of gap* is blocking resolution, and (in later slices) a
|
||||
disposition + disclosure-claim decide what reaches the user and under what epistemic
|
||||
claim. This package is the *owner* of that machine — ``generate/derivation/verify.py``
|
||||
and other serving sites may eventually *consume* it, but must never *define* it.
|
||||
|
||||
Shipped so far (all off-serving — nothing here imports ``generate.derivation`` /
|
||||
``core.reliability_gate``):
|
||||
|
||||
* :mod:`~core.epistemic_disclosure.limitation` (P0-1) — the typed limitation
|
||||
vocabulary, a CONSOLIDATING VIEW over the shipped failure-family registry +
|
||||
contemplation terminals (no fourth taxonomy).
|
||||
* :mod:`~core.epistemic_disclosure.disclosure_claim` (P0-2) — the ``DisclosureClaim``
|
||||
axis (the epistemic claim a response makes), kept SEPARATE from ``ReachLevel``.
|
||||
* :mod:`~core.epistemic_disclosure.disposition` (P0-3) — ``ServedDisposition`` and
|
||||
``choose_served_disposition``: the pure mapping
|
||||
``EpistemicState × LimitationAssessment × DisclosureClaim → ServedDisposition``.
|
||||
Mapping scaffold only — no rendering, no bus, no ``verify.py``; nothing consumes
|
||||
it yet.
|
||||
* :mod:`~core.epistemic_disclosure.ask_serving` — a narrow Q1-D served-ASK artifact
|
||||
adapter. It validates already-rendered question artifacts and returns a typed
|
||||
decision; it does not render prose and does not acquire runtime contemplation.
|
||||
* :mod:`~core.epistemic_disclosure.verified_contract` (P1-A) — the VERIFIED contract:
|
||||
the obligation, the proof shape, the validator, and the single sanctioned route to
|
||||
``EpistemicState.VERIFIED`` / ``DisclosureClaim.VERIFIED``. Contract only — no
|
||||
producer; a faithful solve of a WRONG read must not verify.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from core.epistemic_disclosure.ask_serving import (
|
||||
ServedAskDecision,
|
||||
evaluate_served_ask,
|
||||
)
|
||||
from core.epistemic_disclosure.disclosure_claim import (
|
||||
DEFAULT_DISCLOSURE_CLAIM,
|
||||
DisclosureClaim,
|
||||
)
|
||||
from core.epistemic_disclosure.disposition import (
|
||||
ServedDisposition,
|
||||
choose_served_disposition,
|
||||
)
|
||||
from core.epistemic_disclosure.limitation import (
|
||||
Q1B_ASK_CARVE_OUT,
|
||||
LimitationAssessment,
|
||||
LimitationKind,
|
||||
MissingSlot,
|
||||
ResolutionAction,
|
||||
assess_from_attempt,
|
||||
assess_from_family,
|
||||
terminal_for_action,
|
||||
)
|
||||
from core.epistemic_disclosure.verified_contract import (
|
||||
VERIFICATION_OBLIGATION,
|
||||
VerificationObligation,
|
||||
VerificationProof,
|
||||
VerificationResult,
|
||||
VerificationVerdict,
|
||||
disclosure_for_verification,
|
||||
evaluate_verification,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_DISCLOSURE_CLAIM",
|
||||
"Q1B_ASK_CARVE_OUT",
|
||||
"VERIFICATION_OBLIGATION",
|
||||
"DisclosureClaim",
|
||||
"LimitationAssessment",
|
||||
"LimitationKind",
|
||||
"MissingSlot",
|
||||
"ResolutionAction",
|
||||
"ServedAskDecision",
|
||||
"ServedDisposition",
|
||||
"VerificationObligation",
|
||||
"VerificationProof",
|
||||
"VerificationResult",
|
||||
"VerificationVerdict",
|
||||
"assess_from_attempt",
|
||||
"assess_from_family",
|
||||
"choose_served_disposition",
|
||||
"disclosure_for_verification",
|
||||
"evaluate_served_ask",
|
||||
"evaluate_verification",
|
||||
"terminal_for_action",
|
||||
]
|
||||
|
|
@ -1,171 +0,0 @@
|
|||
"""Stage 2 ASK served-surface artifact adapter.
|
||||
|
||||
This module is intentionally narrow: it validates a pre-rendered Q1-D
|
||||
``DeliveredQuestion`` artifact and decides whether that artifact is eligible to
|
||||
be exposed as a served ASK/QUESTION_NEEDED surface. It does not acquire
|
||||
contemplation results from runtime and does not render question prose.
|
||||
|
||||
Validation enforces the Q1-D artifact contract:
|
||||
|
||||
- top-level JSON object only;
|
||||
- ``status == "question_only"``;
|
||||
- ``requires_review is True``;
|
||||
- ``served is False``;
|
||||
- ``answer_binding`` is absent or ``None``;
|
||||
- ``question`` is an object;
|
||||
- ``question.text`` is a non-empty string;
|
||||
- ``question.slot_name`` is a non-empty string;
|
||||
- ``question_path`` exists on disk and differs from ``proposal_path``.
|
||||
|
||||
Any validation failure fails closed to the caller's fallback surface and
|
||||
standing disposition. The served text is consumed from the artifact exactly; no
|
||||
runtime prose construction or mutation happens here.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from core.epistemic_disclosure.disposition import ServedDisposition, choose_served_disposition
|
||||
from core.epistemic_disclosure.limitation import LimitationAssessment
|
||||
from core.epistemic_questions.serving_gate import ask_serving_enabled
|
||||
from core.epistemic_state import EpistemicState
|
||||
|
||||
_MISSING = object()
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ServedAskDecision:
|
||||
"""The adapter's served-ASK decision."""
|
||||
|
||||
served: bool
|
||||
terminal: str
|
||||
surface: str
|
||||
disposition: ServedDisposition
|
||||
|
||||
|
||||
def _terminal_value(contemplation_result: Any) -> str:
|
||||
terminal = getattr(contemplation_result, "terminal", None)
|
||||
if terminal is None:
|
||||
return "NO_PROGRESS"
|
||||
return str(getattr(terminal, "value", terminal))
|
||||
|
||||
|
||||
def _fallback_disposition(terminal: str) -> ServedDisposition:
|
||||
if terminal == "PROPOSAL_EMITTED":
|
||||
return ServedDisposition.PROPOSE
|
||||
if terminal == "SOLVED_VERIFIED":
|
||||
return ServedDisposition.COMMIT
|
||||
return ServedDisposition.REFUSE
|
||||
|
||||
|
||||
def _fallback_decision(contemplation_result: Any, fallback_surface: str) -> ServedAskDecision:
|
||||
terminal = _terminal_value(contemplation_result)
|
||||
return ServedAskDecision(
|
||||
served=False,
|
||||
terminal=terminal,
|
||||
surface=fallback_surface,
|
||||
disposition=_fallback_disposition(terminal),
|
||||
)
|
||||
|
||||
|
||||
def _validate_question_artifact(data: Any, *, question_path: Path, proposal_path: Any) -> str | None:
|
||||
"""Return the valid question text, or ``None`` for any contract violation."""
|
||||
|
||||
if not isinstance(data, dict):
|
||||
return None
|
||||
if data.get("status") != "question_only":
|
||||
return None
|
||||
if data.get("requires_review") is not True:
|
||||
return None
|
||||
served = data.get("served", _MISSING)
|
||||
if served is _MISSING or served is not False:
|
||||
return None
|
||||
answer_binding = data.get("answer_binding", _MISSING)
|
||||
if answer_binding is not _MISSING and answer_binding is not None:
|
||||
return None
|
||||
|
||||
question = data.get("question")
|
||||
if not isinstance(question, dict):
|
||||
return None
|
||||
|
||||
text = question.get("text")
|
||||
if not isinstance(text, str) or not text.strip():
|
||||
return None
|
||||
|
||||
slot_name = question.get("slot_name")
|
||||
if not isinstance(slot_name, str) or not slot_name.strip():
|
||||
return None
|
||||
|
||||
if proposal_path is not None and str(question_path) == str(proposal_path):
|
||||
return None
|
||||
|
||||
return text.strip()
|
||||
|
||||
|
||||
def evaluate_served_ask(
|
||||
config: Any,
|
||||
contemplation_result: Any,
|
||||
fallback_surface: str,
|
||||
) -> ServedAskDecision:
|
||||
"""Evaluate whether a Q1-D question artifact may be surfaced as ASK.
|
||||
|
||||
This is a bus/disposition adapter, not a renderer and not the runtime
|
||||
acquisition path. The caller supplies a contemplation result that already
|
||||
points to a delivered question artifact. When the gate is disabled or any
|
||||
artifact invariant fails, the adapter returns the fallback surface and the
|
||||
standing fallback disposition.
|
||||
"""
|
||||
|
||||
if not ask_serving_enabled(config):
|
||||
return _fallback_decision(contemplation_result, fallback_surface)
|
||||
|
||||
if _terminal_value(contemplation_result) != "QUESTION_NEEDED":
|
||||
return _fallback_decision(contemplation_result, fallback_surface)
|
||||
|
||||
question_path_value = getattr(contemplation_result, "question_path", None)
|
||||
proposal_path_value = getattr(contemplation_result, "proposal_path", None)
|
||||
if not question_path_value or question_path_value == proposal_path_value:
|
||||
return _fallback_decision(contemplation_result, fallback_surface)
|
||||
|
||||
question_path = Path(question_path_value)
|
||||
if not question_path.is_file():
|
||||
return _fallback_decision(contemplation_result, fallback_surface)
|
||||
|
||||
try:
|
||||
payload = json.loads(question_path.read_text(encoding="utf-8"))
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return _fallback_decision(contemplation_result, fallback_surface)
|
||||
|
||||
question_text = _validate_question_artifact(
|
||||
payload,
|
||||
question_path=question_path,
|
||||
proposal_path=proposal_path_value,
|
||||
)
|
||||
if question_text is None:
|
||||
return _fallback_decision(contemplation_result, fallback_surface)
|
||||
|
||||
limitation = LimitationAssessment(
|
||||
limitation_kind="missing_information",
|
||||
resolution_action="ask_question",
|
||||
epistemic_state=EpistemicState.UNDETERMINED,
|
||||
owner_organ=payload.get("owner_organ"),
|
||||
blocking_reason=str(payload.get("blocking_reason", "")),
|
||||
)
|
||||
disposition = choose_served_disposition(
|
||||
epistemic_state=EpistemicState.UNDETERMINED,
|
||||
limitation=limitation,
|
||||
)
|
||||
|
||||
return ServedAskDecision(
|
||||
served=True,
|
||||
terminal="QUESTION_NEEDED",
|
||||
surface=question_text,
|
||||
disposition=disposition,
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["ServedAskDecision", "evaluate_served_ask"]
|
||||
|
|
@ -1,65 +0,0 @@
|
|||
"""P0-2 — the DisclosureClaim axis (the epistemic claim a served response makes).
|
||||
|
||||
A served response carries two ORTHOGONAL governance properties:
|
||||
|
||||
* ``ReachLevel`` (``core/response_governance/policy.py``) — how far PAST
|
||||
fully-grounded fact the response reaches (STRICT < APPROXIMATE < EXTRAPOLATE
|
||||
< CREATIVE).
|
||||
* ``DisclosureClaim`` (here) — the EPISTEMIC CLAIM the response makes about its
|
||||
own truth status.
|
||||
|
||||
These are deliberately separate axes. ``verified`` is **not** a reach level — it is
|
||||
a claim about *proof state*, not about how far the response speculates. Conflating
|
||||
them (e.g. a hypothetical ``ReachLevel.VERIFIED``) would let a proven answer inherit
|
||||
an "approximate" surface, or an approximation inherit a "verified" badge. Keeping the
|
||||
axes distinct is the architectural commitment behind the Stage-2 lockfile
|
||||
(``docs/analysis/stage2-epistemic-disclosure-bus-verified-v1-scoping-2026-06-08.md`` §0).
|
||||
|
||||
**Discipline — no claim without a producer** (the spine enforces on itself what it
|
||||
enforces on answers). Every member has a real or imminent emitter:
|
||||
|
||||
* ``NONE`` — every response today (the baseline; STRICT + NONE).
|
||||
* ``APPROXIMATE`` — active: the cognition Step-E disclosed estimate
|
||||
(``estimation_enabled`` / ADR-0206 §5).
|
||||
* ``PROPOSAL_ONLY`` — active: ``teaching/proposals`` emits review-only proposals.
|
||||
* ``VERIFIED`` — the imminent frontier: its producer is Phase 1 (P1-A..),
|
||||
declared because it is the v1 target the bus is built around.
|
||||
|
||||
Two claims are intentionally ABSENT, because nothing can emit them — and the spine
|
||||
will not declare a label it cannot earn:
|
||||
|
||||
* ``PROVEN`` — a claim stronger than VERIFIED; no plan to build a producer.
|
||||
* ``ESTIMATED`` — a *future* split of ``APPROXIMATE`` into a distinct
|
||||
numeric-estimate claim, added ONLY once a real estimator producer
|
||||
exists. Until then the cognition estimate is ``APPROXIMATE``.
|
||||
|
||||
P0-2 ships ONLY the axis + its default. No bus behaviour, no mapping to a disposition
|
||||
(that is P0-3 / ServedDisposition). Off-serving: this module imports nothing.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import Enum, unique
|
||||
|
||||
|
||||
@unique
|
||||
class DisclosureClaim(str, Enum):
|
||||
"""The epistemic claim a served surface makes about its own truth status.
|
||||
|
||||
Orthogonal to ``ReachLevel``. ``str``-valued for stable serialization into
|
||||
telemetry / disposition records (the same convention as ``EpistemicState``).
|
||||
"""
|
||||
|
||||
NONE = "none" # no epistemic claim beyond the plain surface (the default)
|
||||
VERIFIED = "verified" # independently confirmed under a canonical-comparison contract
|
||||
APPROXIMATE = "approximate" # a disclosed best-estimate from incomplete evidence
|
||||
PROPOSAL_ONLY = "proposal_only" # offered as a proposal for review, not asserted
|
||||
|
||||
|
||||
#: The default claim: a surface asserts nothing about its truth status unless a
|
||||
#: producer upgrades it. ``STRICT`` reach + ``NONE`` claim is today's every-response
|
||||
#: baseline.
|
||||
DEFAULT_DISCLOSURE_CLAIM: DisclosureClaim = DisclosureClaim.NONE
|
||||
|
||||
|
||||
__all__ = ["DEFAULT_DISCLOSURE_CLAIM", "DisclosureClaim"]
|
||||
|
|
@ -1,110 +0,0 @@
|
|||
"""P0-3 — ServedDisposition: the served-surface decision (mapping scaffold only).
|
||||
|
||||
The third axis of the Epistemic Disclosure bus. Given (a) the epistemic state, (b)
|
||||
the limitation assessment (if resolution is blocked), and (c) the disclosure claim,
|
||||
:func:`choose_served_disposition` decides WHAT KIND OF MOVE the served surface makes:
|
||||
commit / disclose / ask / propose / report / explain / refuse / step-aside.
|
||||
|
||||
This is a PURE MAPPING. No rendering, no bus behaviour, no ``verify.py``, no
|
||||
``govern_response`` mutation — nothing consumes the result yet. P0-3 only fixes the
|
||||
decision table so a later slice / Phase 1 can wire it.
|
||||
|
||||
The load-bearing rule (the Phase-1 guard): a ``DisclosureClaim.VERIFIED`` discloses
|
||||
ONLY when the epistemic state is actually ``EpistemicState.VERIFIED``. An unbacked
|
||||
verified claim degrades to a plain ``COMMIT`` — it is NEVER served as verified before
|
||||
the producer exists. This protects the whole VERIFIED lane from accidental
|
||||
"verified-looking" serving.
|
||||
|
||||
Off-serving: imports no ``generate.derivation`` / ``core.reliability_gate``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import Enum, unique
|
||||
|
||||
from core.epistemic_disclosure.disclosure_claim import DisclosureClaim
|
||||
from core.epistemic_disclosure.limitation import LimitationAssessment
|
||||
from core.epistemic_state import EpistemicState
|
||||
|
||||
|
||||
@unique
|
||||
class ServedDisposition(str, Enum):
|
||||
"""What kind of move the served surface makes.
|
||||
|
||||
``str``-valued for stable serialization (the ``EpistemicState`` /
|
||||
``DisclosureClaim`` convention).
|
||||
"""
|
||||
|
||||
COMMIT = "commit" # serve a fully-grounded answer as-is, no epistemic caveat
|
||||
DISCLOSE = "disclose" # serve under a disclosure claim ([verified] / [approximate])
|
||||
ASK = "ask" # ask the user for the missing/ambiguous datum (Q1 tenant)
|
||||
PROPOSE = "propose" # offer a review-only capability proposal, do not assert
|
||||
REPORT = "report" # report a contradiction with a supplied answer key
|
||||
EXPLAIN = "explain" # explain that the ask is outside the current envelope (scope)
|
||||
REFUSE = "refuse" # hard-stop refusal (impossible / unreadable / known boundary)
|
||||
STEP_ASIDE = "step_aside" # not this organ's domain — cede
|
||||
|
||||
|
||||
def choose_served_disposition(
|
||||
*,
|
||||
epistemic_state: EpistemicState,
|
||||
limitation: LimitationAssessment | None,
|
||||
disclosure_claim: DisclosureClaim = DisclosureClaim.NONE,
|
||||
) -> ServedDisposition:
|
||||
"""Decide the served disposition. Pure, deterministic, total.
|
||||
|
||||
A blocking limitation dominates — you do not serve an answer, you ask / propose /
|
||||
report / explain / refuse / step aside per its resolution action. (A limitation
|
||||
whose action is ``answer`` is non-blocking and falls through to the serve
|
||||
decision.) With no blocking limitation, the disclosure claim + epistemic state pick
|
||||
the serve mode, under the Phase-1 ``VERIFIED`` guard.
|
||||
|
||||
NOTE (scaffold trust boundary): a blocking epistemic state (CONTRADICTED,
|
||||
AMBIGUOUS, UNDETERMINED) reaches this function AS a limitation (e.g. a contradiction
|
||||
arrives as ``report_contradiction``), not as ``limitation=None``. The serve branch
|
||||
trusts ``limitation=None`` to mean "servable".
|
||||
"""
|
||||
if limitation is not None:
|
||||
match limitation.resolution_action:
|
||||
case "ask_question":
|
||||
return ServedDisposition.ASK
|
||||
case "emit_proposal":
|
||||
return ServedDisposition.PROPOSE
|
||||
case "report_contradiction":
|
||||
return ServedDisposition.REPORT
|
||||
case "step_aside":
|
||||
return ServedDisposition.STEP_ASIDE
|
||||
case "refuse_known_boundary":
|
||||
# scope_boundary is the governed "outside the current envelope"
|
||||
# disposition — it may render as a refusal later, but must NOT collapse
|
||||
# into a hard boundary here.
|
||||
if limitation.limitation_kind == "scope_boundary":
|
||||
return ServedDisposition.EXPLAIN
|
||||
return ServedDisposition.REFUSE
|
||||
case "answer":
|
||||
pass # non-blocking — fall through to the serve decision
|
||||
|
||||
return _serve_disposition(epistemic_state, disclosure_claim)
|
||||
|
||||
|
||||
def _serve_disposition(
|
||||
epistemic_state: EpistemicState, disclosure_claim: DisclosureClaim
|
||||
) -> ServedDisposition:
|
||||
"""The no-blocking-limitation branch: claim + state pick the serve mode."""
|
||||
match disclosure_claim:
|
||||
case DisclosureClaim.VERIFIED:
|
||||
# The Phase-1 guard: disclose-as-verified ONLY with the backing state.
|
||||
if epistemic_state is EpistemicState.VERIFIED:
|
||||
return ServedDisposition.DISCLOSE
|
||||
return ServedDisposition.COMMIT
|
||||
case DisclosureClaim.APPROXIMATE:
|
||||
return ServedDisposition.DISCLOSE
|
||||
case DisclosureClaim.PROPOSAL_ONLY:
|
||||
return ServedDisposition.PROPOSE
|
||||
case DisclosureClaim.NONE:
|
||||
return ServedDisposition.COMMIT
|
||||
case _: # pragma: no cover - exhaustive over the 4-member enum; loud if extended
|
||||
raise AssertionError(f"unhandled disclosure_claim: {disclosure_claim!r}")
|
||||
|
||||
|
||||
__all__ = ["ServedDisposition", "choose_served_disposition"]
|
||||
|
|
@ -1,378 +0,0 @@
|
|||
"""P0-1 — the pre-question limitation pass, as a CONSOLIDATING VIEW (session §1.5).
|
||||
|
||||
The intake gate of the Epistemic Disclosure spine: before contemplation chooses a
|
||||
served disposition, it classifies *what KIND of limitation* is blocking resolution.
|
||||
Asking a question is only one possible resolution, and asking is wrong unless the
|
||||
limitation is specifically the kind a question resolves (missing/ambiguous external
|
||||
information). Mis-classify and intake breaks two ways — refuse-when-should-ask (lose
|
||||
the unlocking datum) or ask-when-should-propose (waste the channel).
|
||||
|
||||
This module is the first slice: the typed vocabulary (:data:`LimitationKind` /
|
||||
:data:`ResolutionAction` / :class:`LimitationAssessment`) and the *mapping* that
|
||||
DERIVES each from the already-shipped failure-family registry
|
||||
(:data:`core.comprehension_attempt.failure_family.REGISTRY`) and contemplation
|
||||
:class:`~generate.contemplation.findings.Terminal` set.
|
||||
|
||||
**Hard invariant (no fourth taxonomy).** Every assessment is mechanically derived
|
||||
from an existing ``FailureFamily``; the *only* genuinely new resolution action is
|
||||
``ask_question`` (the Q1/ASK tenant). Through Q1-C it was the one action with no
|
||||
shipped terminal; Q1-D adds :attr:`~generate.contemplation.findings.Terminal.QUESTION_NEEDED`
|
||||
(sibling of ``PROPOSAL_EMITTED``), so :func:`terminal_for_action` is now total — every
|
||||
action maps onto a shipped terminal, which is what makes this a consolidating view.
|
||||
|
||||
**Q1-B transitional carve-out (this slice).** Two shipped families —
|
||||
``missing_total_count`` and ``missing_weighted_total`` — are classified here as
|
||||
``missing_information`` / ``ask_question``: the user *could state the total* and
|
||||
unblock solving, so they are missing data, not capability gaps. The shipped
|
||||
:data:`REGISTRY` still flags them ``proposal_allowed = True`` so that existing
|
||||
consumers (:mod:`core.comprehension_attempt.proposal` /
|
||||
:mod:`generate.contemplation.pass_manager`) keep emitting proposal-only artifacts
|
||||
to the pile — until Q1-C/Q1-D wire ASK delivery to a served surface there is
|
||||
nowhere for an ``ask_question`` to go, and silently dropping the proposal signal
|
||||
would be a capability regression with no compensating intake. Once ASK is serving,
|
||||
the REGISTRY flag flips to ``False`` and the carve-out
|
||||
(:data:`Q1B_ASK_CARVE_OUT`) retires. The carve-out is named, enumerated, and
|
||||
covered by an explicit test (``tests/test_limitation_assessment.py``) so its
|
||||
removal is a conscious act, not a silent re-key.
|
||||
|
||||
**Off-serving.** Imports no ``generate.derivation`` / ``core.reliability_gate``; it
|
||||
cannot move the sealed GSM8K metric. Nothing consumes ``resolution_action`` to change
|
||||
serving yet — this slice only *classifies* and *describes residue*.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Literal
|
||||
|
||||
from core.comprehension_attempt.failure_family import (
|
||||
FailureFamily,
|
||||
family_for_reason,
|
||||
)
|
||||
from core.comprehension_attempt.model import ComprehensionAttempt
|
||||
from core.epistemic_state import EpistemicState
|
||||
from generate.contemplation.findings import Terminal
|
||||
|
||||
#: Why resolution is blocked. The split from :data:`ResolutionAction` keeps *state*
|
||||
#: ("what is true / missing") distinct from *action* ("what to do about it").
|
||||
LimitationKind = Literal[
|
||||
"missing_information", # a needed datum is absent — could be supplied → ask
|
||||
"ambiguous_structure", # data present, relationship unclear → ask (when user-resolvable)
|
||||
"scope_boundary", # exceeds the current capability/evidence envelope → refuse/explain
|
||||
"capability_gap", # info present, CORE lacks the transform → propose (or refuse if signal too coarse)
|
||||
"hard_boundary", # mathematically/logically impossible or undefined → refuse
|
||||
"contradiction", # evidence conflicts with a claimed answer → report
|
||||
"renderability_gap", # asking is right but terms aren't grounded enough to name safely → ask-generic
|
||||
"input_shape", # not this organ's domain → step aside
|
||||
]
|
||||
|
||||
#: What to do about a limitation. ``ask_question`` is the genuinely new action this
|
||||
#: spine introduces; the other five each correspond to a shipped contemplation
|
||||
#: terminal (:func:`terminal_for_action`).
|
||||
ResolutionAction = Literal[
|
||||
"answer",
|
||||
"ask_question",
|
||||
"emit_proposal",
|
||||
"refuse_known_boundary",
|
||||
"report_contradiction",
|
||||
"step_aside",
|
||||
]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class MissingSlot:
|
||||
"""One typed slot the ASK limitation identifies as missing.
|
||||
|
||||
A *structural* description (NOT user-renderable prose): ``slot_name`` is the
|
||||
family-defined slot identifier (e.g. ``"total_count"``); ``expected_unit_or_type``
|
||||
is the family-typed expectation a future bound answer must satisfy (e.g.
|
||||
``"count_int"``); ``binding_target`` is the structural role the slot fills in
|
||||
the organ's setup (e.g. ``"collective_unit_total"``), which Q2 answer-binding
|
||||
re-enters the gate against (scoping §4). Renderable, user-facing strings come
|
||||
later from grounded text spans (:attr:`LimitationAssessment.grounded_terms`),
|
||||
NEVER from these snake_case identifiers — that split is what keeps the renderer
|
||||
wrong=0-safe (scoping §2).
|
||||
"""
|
||||
|
||||
slot_name: str
|
||||
expected_unit_or_type: str
|
||||
binding_target: str
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class LimitationAssessment:
|
||||
"""One typed classification of the limitation blocking a comprehension attempt.
|
||||
|
||||
``epistemic_state`` (what is true) and ``resolution_action`` (what to do) are
|
||||
deliberately distinct: e.g. ``UNDETERMINED`` + ``ask_question`` for a missing
|
||||
datum. ``blocking_reason`` is the failure-family key (the partition key), so the
|
||||
assessment is back-traceable to the shipped registry.
|
||||
|
||||
``missing_slots`` and ``grounded_terms`` are the ASK *typed residue* — populated
|
||||
only for ``ask_question`` resolutions by :func:`assess_from_attempt`. Both
|
||||
default to empty so existing P0-1 callers using :func:`assess_from_family`
|
||||
continue to work unchanged. The wrong=0 invariant (scoping §2): a slot here
|
||||
carries family-typed structural identifiers only; renderable prose must come
|
||||
from ``grounded_terms`` (verbatim text spans), never fabricated.
|
||||
"""
|
||||
|
||||
limitation_kind: LimitationKind
|
||||
resolution_action: ResolutionAction
|
||||
epistemic_state: EpistemicState
|
||||
owner_organ: str | None
|
||||
blocking_reason: str
|
||||
missing_slots: tuple[MissingSlot, ...] = field(default_factory=tuple)
|
||||
grounded_terms: tuple[str, ...] = field(default_factory=tuple)
|
||||
|
||||
|
||||
# --- The consolidating mapping (derived from the shipped registry, not invented) ---
|
||||
|
||||
#: ``EpistemicState`` each kind asserts. ``scope_boundary`` lights up the RESERVED
|
||||
#: ``SCOPE_BOUNDARY`` state with a real producer; ``contradiction`` / ``ambiguous``
|
||||
#: reuse the ACTIVE states. The refuse/propose kinds share ``UNDETERMINED`` — the
|
||||
#: *kind* carries the distinction, the *state* only says "no answer determined".
|
||||
_KIND_TO_STATE: dict[LimitationKind, EpistemicState] = {
|
||||
"missing_information": EpistemicState.UNDETERMINED,
|
||||
"ambiguous_structure": EpistemicState.AMBIGUOUS,
|
||||
"scope_boundary": EpistemicState.SCOPE_BOUNDARY,
|
||||
"capability_gap": EpistemicState.UNDETERMINED,
|
||||
"hard_boundary": EpistemicState.UNDETERMINED,
|
||||
"contradiction": EpistemicState.CONTRADICTED,
|
||||
"renderability_gap": EpistemicState.UNDETERMINED,
|
||||
"input_shape": EpistemicState.UNDETERMINED,
|
||||
}
|
||||
|
||||
#: The shipped contemplation ``Terminal`` each action corresponds to. Through Q1-C
|
||||
#: ``ask_question`` mapped to ``None`` (the one action the spine added with no terminal
|
||||
#: yet); Q1-D ships ``QUESTION_NEEDED`` (sibling of ``PROPOSAL_EMITTED``), so the map is
|
||||
#: now TOTAL — all six actions correspond to shipped terminals. This totality is the
|
||||
#: proof the limitation pass is a consolidating view, not a new universe.
|
||||
#:
|
||||
#: NOTE: this is the action's *home* terminal "in principle". The terminal a Q1-D
|
||||
#: *delivery* actually emits depends on renderability — an unrenderable ``ask_question``
|
||||
#: falls back to the family's standing disposition (the D2 guard in
|
||||
#: :mod:`core.epistemic_questions.delivery`), it does NOT emit a contentless
|
||||
#: ``QUESTION_NEEDED``.
|
||||
_ACTION_TO_TERMINAL: dict[ResolutionAction, Terminal] = {
|
||||
"answer": Terminal.SOLVED_VERIFIED,
|
||||
"emit_proposal": Terminal.PROPOSAL_EMITTED,
|
||||
"refuse_known_boundary": Terminal.REFUSED_KNOWN_BOUNDARY,
|
||||
"report_contradiction": Terminal.CONTRADICTION_DETECTED,
|
||||
"step_aside": Terminal.NO_PROGRESS,
|
||||
"ask_question": Terminal.QUESTION_NEEDED,
|
||||
}
|
||||
|
||||
#: **Transitional carve-out (Q1-B).** Families this slice classifies as
|
||||
#: ``ask_question`` in the limitation layer while their shipped
|
||||
#: :data:`REGISTRY` entries keep ``proposal_allowed = True`` so the contemplation
|
||||
#: pass and proposal pile keep working unchanged. This is an honest migration seam:
|
||||
#: the disclosure layer speaks the truthful classification now; the operational
|
||||
#: layer keeps the current signal so nothing regresses before Q1-C/Q1-D wires ASK
|
||||
#: delivery. Retirement: once ASK is serving, flip ``proposal_allowed = False`` on
|
||||
#: these two families in :mod:`core.comprehension_attempt.failure_family`, drop
|
||||
#: this set, and amend the ``proposal_allowed`` invariant in tests.
|
||||
Q1B_ASK_CARVE_OUT: frozenset[str] = frozenset(
|
||||
{"missing_total_count", "missing_weighted_total"}
|
||||
)
|
||||
|
||||
#: family name → (LimitationKind, ResolutionAction). Keys must equal the REGISTRY's
|
||||
#: family names exactly (asserted total by test). Classification rationale per family:
|
||||
#: - growth surfaces (``proposal_allowed``) → ``capability_gap`` / ``emit_proposal``
|
||||
#: EXCEPT :data:`Q1B_ASK_CARVE_OUT` — see that constant's docstring
|
||||
#: - underdetermined / missing-datum refusals → ``missing_information`` / ``ask_question``
|
||||
#: - same-unit-but-no-cue ambiguity the user could resolve → ``ambiguous_structure`` / ask
|
||||
#: - math/logic impossibility, incoherence, coarse-signal parse gaps → ``hard_boundary`` / refuse
|
||||
#: - beyond-current-solver-envelope → ``scope_boundary`` / refuse
|
||||
#: - the answer-key verdict → ``contradiction`` / ``report_contradiction``
|
||||
#: - foreign text → ``input_shape`` / ``step_aside``
|
||||
_FAMILY_TO_LIMITATION: dict[str, tuple[LimitationKind, ResolutionAction]] = {
|
||||
# cross-organ
|
||||
"input_shape": ("input_shape", "step_aside"),
|
||||
"admissibility_incompatible": ("hard_boundary", "refuse_known_boundary"),
|
||||
# R1
|
||||
"unsupported_clause_shape": ("capability_gap", "refuse_known_boundary"),
|
||||
"ungrounded_base": ("missing_information", "ask_question"),
|
||||
"over_determined": ("hard_boundary", "refuse_known_boundary"),
|
||||
# R2 boundaries
|
||||
"unsupported_system_size": ("scope_boundary", "refuse_known_boundary"),
|
||||
"indistinguishable_system": ("hard_boundary", "refuse_known_boundary"),
|
||||
"non_integer_solution": ("hard_boundary", "refuse_known_boundary"),
|
||||
"negative_solution": ("hard_boundary", "refuse_known_boundary"),
|
||||
"answer_choice_unresolved": ("ambiguous_structure", "refuse_known_boundary"),
|
||||
# R2 growth surfaces — Q1-B reclassification (see Q1B_ASK_CARVE_OUT):
|
||||
# disclosure says ask; REGISTRY still emits proposals to the pile.
|
||||
"missing_total_count": ("missing_information", "ask_question"),
|
||||
"missing_weighted_total": ("missing_information", "ask_question"),
|
||||
"missing_category_pair": ("capability_gap", "emit_proposal"),
|
||||
"missing_attribute_coefficient": ("capability_gap", "emit_proposal"),
|
||||
# R2 verdict
|
||||
"answer_key_contradiction": ("contradiction", "report_contradiction"),
|
||||
# R3
|
||||
"unsupported_rate_duration": ("capability_gap", "emit_proposal"),
|
||||
"rate_underdetermined": ("missing_information", "ask_question"),
|
||||
"unsupported_temporal_state": ("scope_boundary", "refuse_known_boundary"),
|
||||
# R4 combined-rate boundaries
|
||||
"cmb_unit_mismatch": ("hard_boundary", "refuse_known_boundary"),
|
||||
"cmb_combine_ambiguous": ("ambiguous_structure", "ask_question"),
|
||||
"cmb_underdetermined": ("missing_information", "ask_question"),
|
||||
"cmb_non_positive_net": ("hard_boundary", "refuse_known_boundary"),
|
||||
"cmb_non_integer": ("hard_boundary", "refuse_known_boundary"),
|
||||
# R4 growth surfaces
|
||||
"cmb_unsupported_rate_count": ("capability_gap", "emit_proposal"),
|
||||
"cmb_unsupported_reciprocal": ("capability_gap", "emit_proposal"),
|
||||
"cmb_unsupported_clock_interval": ("capability_gap", "emit_proposal"),
|
||||
}
|
||||
|
||||
#: family name → typed slots an ASK assessment for that family identifies as missing.
|
||||
#: Only families with a *known* slot structure appear here; an ask-mapped family without
|
||||
#: an entry yields an empty residue (the "minimal" stance — never fabricate a slot the
|
||||
#: family's contract has not pinned down yet). Slot semantics, per family:
|
||||
#: - ``missing_total_count`` : the collective-unit total count (R2 constraint C7);
|
||||
#: ``binding_target`` matches the equation slot the augmented input would fill
|
||||
#: (Q2 re-entry, scoping §4).
|
||||
#: - ``missing_weighted_total``: the measured-unit weighted total (R2 constraint C8).
|
||||
#: Other ask-mapped families (``ungrounded_base``, ``rate_underdetermined``,
|
||||
#: ``cmb_underdetermined``, ``cmb_combine_ambiguous``) get slots in later Q1 sub-PRs
|
||||
#: once their per-family slot signatures are pinned with tests.
|
||||
_FAMILY_TO_MISSING_SLOTS: dict[str, tuple[MissingSlot, ...]] = {
|
||||
"missing_total_count": (
|
||||
MissingSlot(
|
||||
slot_name="total_count",
|
||||
expected_unit_or_type="count_int",
|
||||
binding_target="collective_unit_total",
|
||||
),
|
||||
),
|
||||
"missing_weighted_total": (
|
||||
MissingSlot(
|
||||
slot_name="weighted_total",
|
||||
expected_unit_or_type="measured_unit_int",
|
||||
binding_target="weighted_total_value",
|
||||
),
|
||||
),
|
||||
}
|
||||
|
||||
# A conservative refusal for any family/reason that is not in the mapping. The total
|
||||
# mapping (asserted by test against REGISTRY) means this is dead in practice; if a new
|
||||
# family ever lands unmapped, it refuses — it NEVER silently becomes an answerable
|
||||
# question (the wrong=0-safe default).
|
||||
_CONSERVATIVE_DEFAULT: tuple[LimitationKind, ResolutionAction] = (
|
||||
"hard_boundary",
|
||||
"refuse_known_boundary",
|
||||
)
|
||||
|
||||
|
||||
def assess_from_family(family: FailureFamily) -> LimitationAssessment:
|
||||
"""The limitation a failure family expresses, as a typed assessment.
|
||||
|
||||
Total over :data:`REGISTRY` (asserted by test). An unmapped family falls to the
|
||||
conservative refuse default — never ``ask_question``. Residue defaults to empty;
|
||||
populating typed slots / grounded terms requires a specific attempt (use
|
||||
:func:`assess_from_attempt`).
|
||||
"""
|
||||
kind, action = _FAMILY_TO_LIMITATION.get(family.name, _CONSERVATIVE_DEFAULT)
|
||||
return LimitationAssessment(
|
||||
limitation_kind=kind,
|
||||
resolution_action=action,
|
||||
epistemic_state=_KIND_TO_STATE[kind],
|
||||
owner_organ=family.owner,
|
||||
blocking_reason=family.name,
|
||||
)
|
||||
|
||||
|
||||
def _residue_from_attempt(
|
||||
attempt: ComprehensionAttempt,
|
||||
family: FailureFamily,
|
||||
action: ResolutionAction,
|
||||
) -> tuple[tuple[MissingSlot, ...], tuple[str, ...]]:
|
||||
"""Typed ASK residue for an ask-mapped attempt — empty for any other action.
|
||||
|
||||
Wrong=0 invariant (scoping §2): ``grounded_terms`` carries only verbatim text from
|
||||
:attr:`ComprehensionAttempt.evidence` SourceSpanLinks. If the attempt carries no
|
||||
evidence (today: every refused attempt — :mod:`core.comprehension_attempt.classify`
|
||||
leaves ``evidence = ()``), ``grounded_terms`` is empty, NEVER fabricated from the
|
||||
family or the refusal reason. ``missing_slots`` is family-derived (snake_case
|
||||
structural identifiers, not renderable prose) so it is always safe to emit; absent
|
||||
from :data:`_FAMILY_TO_MISSING_SLOTS` ⇒ no slots (minimal stance — never invent a
|
||||
slot the family contract has not pinned down).
|
||||
"""
|
||||
if action != "ask_question":
|
||||
return ((), ())
|
||||
slots = _FAMILY_TO_MISSING_SLOTS.get(family.name, ())
|
||||
grounded = tuple(link.text for link in attempt.evidence)
|
||||
return (slots, grounded)
|
||||
|
||||
|
||||
def assess_from_attempt(attempt: ComprehensionAttempt) -> LimitationAssessment | None:
|
||||
"""Classify the limitation a comprehension attempt expresses, or ``None``.
|
||||
|
||||
- ``contradiction`` outcome → report it (no refusal reason carries this; it is the
|
||||
answer-choice verdict).
|
||||
- a refusal → classify via its failure family; an *unclassified* refusal reason
|
||||
falls to the conservative refuse default (never ``ask_question``).
|
||||
- any non-limitation outcome (solved/admissible setup, or eval-only ``*_wrong``)
|
||||
→ ``None``: there is no resolvable limitation to act on.
|
||||
|
||||
For ask-mapped refusals, the returned assessment carries typed residue
|
||||
(:attr:`~LimitationAssessment.missing_slots` / ``grounded_terms``) per
|
||||
:func:`_residue_from_attempt`'s wrong=0 invariant.
|
||||
"""
|
||||
if attempt.outcome == "contradiction":
|
||||
return LimitationAssessment(
|
||||
limitation_kind="contradiction",
|
||||
resolution_action="report_contradiction",
|
||||
epistemic_state=EpistemicState.CONTRADICTED,
|
||||
owner_organ=attempt.organ,
|
||||
blocking_reason="answer_key_contradiction",
|
||||
)
|
||||
if not attempt.is_refusal:
|
||||
return None
|
||||
family = family_for_reason(attempt.refusal_reason)
|
||||
if family is None:
|
||||
kind, action = _CONSERVATIVE_DEFAULT
|
||||
return LimitationAssessment(
|
||||
limitation_kind=kind,
|
||||
resolution_action=action,
|
||||
epistemic_state=_KIND_TO_STATE[kind],
|
||||
owner_organ=attempt.organ,
|
||||
blocking_reason=attempt.refusal_reason or "unclassified",
|
||||
)
|
||||
base = assess_from_family(family)
|
||||
missing_slots, grounded_terms = _residue_from_attempt(
|
||||
attempt, family, base.resolution_action
|
||||
)
|
||||
if not missing_slots and not grounded_terms:
|
||||
return base
|
||||
return LimitationAssessment(
|
||||
limitation_kind=base.limitation_kind,
|
||||
resolution_action=base.resolution_action,
|
||||
epistemic_state=base.epistemic_state,
|
||||
owner_organ=attempt.organ,
|
||||
blocking_reason=base.blocking_reason,
|
||||
missing_slots=missing_slots,
|
||||
grounded_terms=grounded_terms,
|
||||
)
|
||||
|
||||
|
||||
def terminal_for_action(action: ResolutionAction) -> Terminal:
|
||||
"""The shipped contemplation ``Terminal`` an action corresponds to — total.
|
||||
|
||||
Every action maps to a shipped terminal; ``ask_question`` resolves to
|
||||
``QUESTION_NEEDED`` (added by Q1-D, sibling of ``PROPOSAL_EMITTED``). That totality
|
||||
is what makes this a consolidating view rather than a new taxonomy. This is the
|
||||
action's *home* terminal; the terminal a Q1-D delivery actually emits may differ
|
||||
when the question is unrenderable (the D2 fallback — see
|
||||
:mod:`core.epistemic_questions.delivery`).
|
||||
"""
|
||||
return _ACTION_TO_TERMINAL[action]
|
||||
|
||||
|
||||
__all__ = [
|
||||
"Q1B_ASK_CARVE_OUT",
|
||||
"LimitationAssessment",
|
||||
"LimitationKind",
|
||||
"MissingSlot",
|
||||
"ResolutionAction",
|
||||
"assess_from_attempt",
|
||||
"assess_from_family",
|
||||
"terminal_for_action",
|
||||
]
|
||||
|
|
@ -1,33 +0,0 @@
|
|||
"""VERIFIED serving gate helper — default-dark, no served-surface wiring.
|
||||
|
||||
This module centralizes the future kill-switch predicate for VERIFIED serving.
|
||||
It is default-dark / fail-closed: if the config field is missing or malformed,
|
||||
it must evaluate to False.
|
||||
|
||||
This helper only centralizes the future kill-switch predicate so that future serving code
|
||||
has one audited predicate. It does not wire any served-surface or implement served
|
||||
VERIFIED behavior. Missing field means False.
|
||||
|
||||
Note that eval-gold-backed producers (such as the verification producer in
|
||||
evals/constraint_oracle/verified_producer.py) are not serving-eligible.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from core.config import DEFAULT_CONFIG, RuntimeConfig
|
||||
|
||||
|
||||
def verified_serving_enabled(config: RuntimeConfig | Any | None = None) -> bool:
|
||||
"""Return whether served VERIFIED delivery is explicitly enabled.
|
||||
|
||||
Missing attribute means False. This is the load-bearing dark-gate invariant:
|
||||
the served VERIFIED path cannot light merely because the helper exists or because
|
||||
an older RuntimeConfig instance lacks the future field.
|
||||
"""
|
||||
cfg = DEFAULT_CONFIG if config is None else config
|
||||
return bool(getattr(cfg, "verified_serving_enabled", False))
|
||||
|
||||
|
||||
__all__ = ["verified_serving_enabled"]
|
||||
|
|
@ -1,211 +0,0 @@
|
|||
"""P1-A — the VERIFIED contract (the meaning of "verified", before any producer).
|
||||
|
||||
This module defines WHAT IT MEANS for a result to earn ``DisclosureClaim.VERIFIED`` /
|
||||
``EpistemicState.VERIFIED`` — the soundness≠correctness gate the whole VERIFIED lane
|
||||
rests on. It ships ONLY the contract: the obligation, the proof SHAPE a producer must
|
||||
fill, the validator that enforces the obligation, and the single sanctioned route from
|
||||
a validated proof to the verified state/claim.
|
||||
|
||||
It does NOT produce proofs. There is no reader, no solver, no back-substitution
|
||||
computation, no serving, no ``verify.py`` call, no ``verified_serving_enabled``. P1-B+
|
||||
fill :class:`VerificationProof` with real digests; P1-A only fixes the rules a proof
|
||||
must satisfy — and, above all, the rule that makes the lane safe:
|
||||
|
||||
A faithful solve of a WRONG read must NOT verify.
|
||||
|
||||
**The mechanism.** Verification requires TWO INDEPENDENT reads (distinct reader
|
||||
lineages) that CONVERGE on the same canonical structure:
|
||||
|
||||
* a *wrong* primary read is caught because the independent read **disagrees** —
|
||||
back-substitution alone cannot catch a read error (it only proves the solver was
|
||||
faithful to whatever structure it was handed);
|
||||
* a single reader run twice ("same answer twice") is rejected as **not independent**.
|
||||
|
||||
Neither gold-agreement, nor absence-of-refusal, nor a second solver over ONE read can
|
||||
earn VERIFIED; only this contract can. (See [[VERIFIED-canonical-comparison-scoping-2026-06-06]]:
|
||||
"independence must be in the READING, not the solving".)
|
||||
|
||||
**Discipline.** ``EpistemicState.VERIFIED`` must be reached ONLY via
|
||||
:func:`disclosure_for_verification` over a VERIFIED :class:`VerificationResult` — never
|
||||
constructed directly by producer/serving code. A future architectural invariant can
|
||||
scan for direct emission; P1-A establishes the route.
|
||||
|
||||
Off-serving: imports no ``generate.derivation`` / ``core.reliability_gate``.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import Enum, unique
|
||||
|
||||
from core.epistemic_disclosure.disclosure_claim import DisclosureClaim
|
||||
from core.epistemic_disclosure.limitation import LimitationAssessment
|
||||
from core.epistemic_state import EpistemicState
|
||||
|
||||
|
||||
@unique
|
||||
class VerificationVerdict(str, Enum):
|
||||
"""Whether a result earns the VERIFIED claim. There is no middle state — a result
|
||||
either survives every obligation or it does not verify (refuse-preferring)."""
|
||||
|
||||
VERIFIED = "verified"
|
||||
NOT_VERIFIED = "not_verified"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class VerificationObligation:
|
||||
"""The declarative contract: which checks a proof must survive to be VERIFIED.
|
||||
|
||||
A schema naming proof obligations (CLAUDE.md "Schema-Defined Proof Obligations").
|
||||
The canonical :data:`VERIFICATION_OBLIGATION` sets every flag ``True``; the flags
|
||||
exist so a test can prove each obligation is LOAD-BEARING — relax exactly one and
|
||||
the corresponding poison case slips through (see ``test_verified_contract.py``).
|
||||
"""
|
||||
|
||||
requires_independent_read: bool # two DISTINCT reader lineages — not the same read twice
|
||||
rejects_wrong_read_even_if_solved: bool # the two reads must CONVERGE — catches a wrong read
|
||||
requires_bound_slots: bool # the answer binds to the STATED slots (query/unknowns), not phantom ones
|
||||
requires_back_substitution: bool # the answer back-substitutes into the canonical structure
|
||||
requires_boundary_clear: bool # no organ boundary fired in the chain
|
||||
|
||||
|
||||
#: The canonical, fully-strict obligation. VERIFIED requires ALL of it.
|
||||
VERIFICATION_OBLIGATION: VerificationObligation = VerificationObligation(
|
||||
requires_independent_read=True,
|
||||
rejects_wrong_read_even_if_solved=True,
|
||||
requires_bound_slots=True,
|
||||
requires_back_substitution=True,
|
||||
requires_boundary_clear=True,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class VerificationProof:
|
||||
"""The replayable proof shape a producer (P1-B+) must fill. P1-A defines the shape
|
||||
and the rules; it computes none of these digests.
|
||||
|
||||
The two reads are kept separate ON PURPOSE: ``primary_reader_lineage`` /
|
||||
``independent_reader_lineage`` must DIFFER (independence), and ``primary_read_digest``
|
||||
/ ``independent_read_digest`` must MATCH (convergence on one canonical structure).
|
||||
Independence + convergence together are what reject a faithful solve of a wrong read.
|
||||
|
||||
The "from the stated quantities" obligation is split into THREE separable digests
|
||||
(P1-C hardening — for replay, audit, and failure localization):
|
||||
``derivation_digest`` (a solve happened from the structure), ``bound_slots_digest``
|
||||
(the answer binds to a STATED slot — the asked unknown — not a phantom), and
|
||||
``back_substitution_digest`` (the answer satisfies the constraints). A complete
|
||||
derivation does NOT imply the answer bound to a declared slot, so the two are
|
||||
distinct obligations (``test_derivation_digest_alone_is_insufficient_without_bound_slots``).
|
||||
"""
|
||||
|
||||
source_problem_digest: str # provenance: hash of the problem text
|
||||
primary_reader_lineage: str # identity of the primary reader
|
||||
independent_reader_lineage: str # identity of the independent cross-check reader
|
||||
primary_read_digest: str # canonical structure the primary read produced
|
||||
independent_read_digest: str # canonical structure the independent read produced
|
||||
derivation_digest: str # the derivation (solve) from the STATED quantities
|
||||
bound_slots_digest: str # the answer binds to the STATED slots (asked unknown), not a phantom
|
||||
back_substitution_digest: str # back-substitution into the canonical structure
|
||||
boundary_clear: bool # no organ boundary fired
|
||||
contradiction_clear: bool # no contradiction family fired
|
||||
|
||||
|
||||
# Reason codes for a failed obligation — each names exactly one violated rule.
|
||||
REASON_READS_NOT_INDEPENDENT = "reads_not_independent" # same reader lineage twice
|
||||
REASON_READS_DISAGREE = "reads_disagree" # the wrong-read catcher
|
||||
REASON_NO_BOUND_SLOTS = "no_bound_slots" # answer did not bind to a stated slot
|
||||
REASON_NO_BACK_SUBSTITUTION = "no_back_substitution"
|
||||
REASON_BOUNDARY_FIRED = "boundary_fired"
|
||||
REASON_CONTRADICTION_PRESENT = "contradiction_present"
|
||||
REASON_INCOMPLETE_PROOF = "incomplete_proof_digest"
|
||||
REASON_UNRESOLVED_LIMITATION = "unresolved_limitation"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class VerificationResult:
|
||||
"""The verdict plus the specific obligations that failed (empty iff VERIFIED)."""
|
||||
|
||||
verdict: VerificationVerdict
|
||||
failed_checks: tuple[str, ...]
|
||||
|
||||
|
||||
def evaluate_verification(
|
||||
proof: VerificationProof,
|
||||
*,
|
||||
limitation: LimitationAssessment | None,
|
||||
obligation: VerificationObligation = VERIFICATION_OBLIGATION,
|
||||
) -> VerificationResult:
|
||||
"""Apply the VERIFIED contract to a proof. Refuse-preferring: VERIFIED only if
|
||||
EVERY obligation survives; otherwise NOT_VERIFIED with the failed checks.
|
||||
|
||||
Pure logic over the proof fields — it does NOT compute or trust any digest, it only
|
||||
enforces the rules a producer's proof must satisfy. ``limitation`` is the
|
||||
contemplation outcome: a verified answer cannot coexist with an unresolved
|
||||
limitation (a contradiction, an ambiguity, a missing datum).
|
||||
"""
|
||||
failed: list[str] = []
|
||||
|
||||
if obligation.requires_independent_read and (
|
||||
proof.primary_reader_lineage == proof.independent_reader_lineage
|
||||
):
|
||||
failed.append(REASON_READS_NOT_INDEPENDENT)
|
||||
|
||||
if obligation.rejects_wrong_read_even_if_solved and (
|
||||
proof.primary_read_digest != proof.independent_read_digest
|
||||
):
|
||||
failed.append(REASON_READS_DISAGREE)
|
||||
|
||||
if obligation.requires_bound_slots and not proof.bound_slots_digest:
|
||||
failed.append(REASON_NO_BOUND_SLOTS)
|
||||
|
||||
if obligation.requires_back_substitution and not proof.back_substitution_digest:
|
||||
failed.append(REASON_NO_BACK_SUBSTITUTION)
|
||||
|
||||
if obligation.requires_boundary_clear and not proof.boundary_clear:
|
||||
failed.append(REASON_BOUNDARY_FIRED)
|
||||
|
||||
# Non-negotiable checks (NOT gated by the obligation flags):
|
||||
if not proof.contradiction_clear:
|
||||
failed.append(REASON_CONTRADICTION_PRESENT)
|
||||
|
||||
if not (
|
||||
proof.source_problem_digest
|
||||
and proof.primary_read_digest
|
||||
and proof.derivation_digest
|
||||
):
|
||||
failed.append(REASON_INCOMPLETE_PROOF)
|
||||
|
||||
if limitation is not None:
|
||||
failed.append(REASON_UNRESOLVED_LIMITATION)
|
||||
|
||||
verdict = (
|
||||
VerificationVerdict.VERIFIED if not failed else VerificationVerdict.NOT_VERIFIED
|
||||
)
|
||||
return VerificationResult(verdict=verdict, failed_checks=tuple(failed))
|
||||
|
||||
|
||||
def disclosure_for_verification(
|
||||
result: VerificationResult,
|
||||
) -> tuple[EpistemicState, DisclosureClaim]:
|
||||
"""The ONLY sanctioned route to a verified state/claim.
|
||||
|
||||
VERIFIED verdict → (``EpistemicState.VERIFIED``, ``DisclosureClaim.VERIFIED``);
|
||||
anything else → (``UNDETERMINED``, ``NONE``). Producer/serving code must reach
|
||||
``EpistemicState.VERIFIED`` THROUGH this gate, never by constructing it directly —
|
||||
that is what keeps "gold agreement" / "same answer twice" / "no refusal" from ever
|
||||
masquerading as verified.
|
||||
"""
|
||||
if result.verdict is VerificationVerdict.VERIFIED:
|
||||
return EpistemicState.VERIFIED, DisclosureClaim.VERIFIED
|
||||
return EpistemicState.UNDETERMINED, DisclosureClaim.NONE
|
||||
|
||||
|
||||
__all__ = [
|
||||
"VERIFICATION_OBLIGATION",
|
||||
"VerificationObligation",
|
||||
"VerificationProof",
|
||||
"VerificationResult",
|
||||
"VerificationVerdict",
|
||||
"disclosure_for_verification",
|
||||
"evaluate_verification",
|
||||
]
|
||||
|
|
@ -1,40 +0,0 @@
|
|||
"""The epistemic question organ — render (Q1-C) and off-serving delivery (Q1-D).
|
||||
|
||||
Q1-C (:mod:`core.epistemic_questions.render`) turns an ASK
|
||||
:class:`~core.epistemic_disclosure.limitation.LimitationAssessment` into an
|
||||
:class:`~core.epistemic_questions.render.EpistemicQuestion` under the wrong=0
|
||||
grounded-rendering invariant (scoping §2) — render only, no fabrication.
|
||||
|
||||
Q1-D (:mod:`core.epistemic_questions.delivery`) routes that rendered question onto
|
||||
the contemplation bus as the ``QUESTION_NEEDED`` tenant and writes a proposal-only
|
||||
artifact to the ``teaching/questions`` sink — consuming the renderer verbatim, never
|
||||
rendering. Off-serving: no served surface, no ``ask_serving_enabled`` yet.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from core.epistemic_questions.delivery import (
|
||||
AnswerBinding,
|
||||
DeliveredQuestion,
|
||||
DeliveryOutcome,
|
||||
default_question_root,
|
||||
deliver_ask,
|
||||
emit_question,
|
||||
question_path,
|
||||
)
|
||||
from core.epistemic_questions.render import (
|
||||
EpistemicQuestion,
|
||||
render_question,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"AnswerBinding",
|
||||
"DeliveredQuestion",
|
||||
"DeliveryOutcome",
|
||||
"EpistemicQuestion",
|
||||
"default_question_root",
|
||||
"deliver_ask",
|
||||
"emit_question",
|
||||
"question_path",
|
||||
"render_question",
|
||||
]
|
||||
|
|
@ -1,272 +0,0 @@
|
|||
"""Q1-D — off-serving ASK delivery: route a rendered question onto the bus.
|
||||
|
||||
The fourth and final Q1 rung. Given an ``ask_question``
|
||||
:class:`~core.epistemic_disclosure.limitation.LimitationAssessment`, decide the
|
||||
contemplation :class:`~generate.contemplation.findings.Terminal` and, when a question
|
||||
can honestly be asked, produce a :class:`DeliveredQuestion` artifact for the
|
||||
review-gated, **question-only** ``teaching/questions`` sink. A question is an *intake
|
||||
request*, NOT a capability proposal — ``question_only`` is its own lane, distinct from
|
||||
the ``proposal_only`` lane of :mod:`core.comprehension_attempt.proposal`. This is the
|
||||
ASK *analogue* of that emitter (and just as toothless: it never serves, never mounts,
|
||||
always requires review), but the artifacts must never be conflated.
|
||||
|
||||
**The rung separation (the steer).** Q1-D *consumes* the Q1-C
|
||||
:class:`~core.epistemic_questions.render.EpistemicQuestion` — it does NOT render.
|
||||
:func:`render_question` is called exactly once (in :func:`deliver_ask`) and its
|
||||
result is wrapped verbatim; Q1-D constructs no user-facing prose of its own, so the
|
||||
Q1-C grounded-rendering wrong=0 guard is never bypassed by a second surface.
|
||||
|
||||
Q1-B: typed residue (what is missing, as typed slots)
|
||||
Q1-C: renderability (can it be asked without fabricating? → EpistemicQuestion)
|
||||
Q1-D: delivery (route the rendered question onto the bus) ← here
|
||||
|
||||
**D2 — the delivery-side wrong=0 guard.** A contentless ``QUESTION_NEEDED`` is worse
|
||||
than useless: it is a *false intake surface* (the user is invited to answer a question
|
||||
that names nothing). So when :func:`render_question` returns ``unrenderable``
|
||||
(``renderability_gap`` / ``multi_slot_not_supported`` / ``no_missing_slot`` / the
|
||||
fabrication backstop), :func:`deliver_ask` does NOT emit ``QUESTION_NEEDED``. It falls
|
||||
back to the family's *standing disposition*:
|
||||
|
||||
family still proposes (``proposal_allowed``) → ``PROPOSAL_EMITTED``
|
||||
otherwise → ``NO_PROGRESS``
|
||||
|
||||
This guard is enforced twice: in :func:`deliver_ask`'s branch, and structurally — a
|
||||
:class:`DeliveredQuestion` *cannot wrap an unrenderable question* (its
|
||||
``__post_init__`` refuses), so ``QUESTION_NEEDED`` is unreachable without a real,
|
||||
rendered question.
|
||||
|
||||
**D3 — the carve-out stays.** Q1-D delivery is OFF-SERVING. It does not flip the
|
||||
``Q1B_ASK_CARVE_OUT`` (``missing_total_count`` / ``missing_weighted_total`` keep
|
||||
``proposal_allowed = True`` in the registry, so the proposal pile keeps working). Both
|
||||
signals coexist during the carve-out — the off-serving question artifact AND the
|
||||
existing proposal — so no operational signal is lost. The flip to ``ask`` waits on a
|
||||
future ``ask_serving_enabled`` gate, not on this rung.
|
||||
|
||||
**D5 — single-slot only.** Q1-C refuses multi-slot rendering, so Q1-D delivers at most
|
||||
one ``DeliveredQuestion`` per assessment; a multi-slot assessment is ``unrenderable``
|
||||
(``multi_slot_not_supported``) and takes the D2 fallback. No fan-out here.
|
||||
|
||||
**Off-serving.** Imports nothing from ``generate.derivation`` / ``core.reliability_gate``;
|
||||
it cannot move the sealed GSM8K metric, and there is NO served surface — no
|
||||
``ask_serving_enabled``, no ``chat/runtime.py`` wiring. The artifacts land in the
|
||||
review-gated teaching sink and nowhere else.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from core.comprehension_attempt.failure_family import family_by_name
|
||||
from core.epistemic_disclosure.limitation import LimitationAssessment
|
||||
from core.epistemic_questions.render import EpistemicQuestion, render_question
|
||||
from generate.contemplation.findings import Terminal
|
||||
|
||||
#: The sink status — the ASK analogue of the proposal emitter's ``"proposal_only"``.
|
||||
#: A delivered question is review-gated intake, never a served answer.
|
||||
_QUESTION_STATUS = "question_only"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AnswerBinding:
|
||||
"""RESERVED for Q2 — the typed seat a future answer round-trip binds into.
|
||||
|
||||
Q1-D produces NO ``AnswerBinding`` (every :class:`DeliveredQuestion` carries
|
||||
``answer_binding = None``; see that class's ``__post_init__``). The seat exists so
|
||||
the Q2 binder — which must re-enter the limitation gate with augmented input, never
|
||||
mutate the model mid-flight (scoping §4) — is wireable without reshaping the
|
||||
artifact. ``target_slot`` / ``binding_target`` mirror the
|
||||
:class:`~core.epistemic_disclosure.limitation.MissingSlot` the answer fills.
|
||||
"""
|
||||
|
||||
target_organ: str
|
||||
target_slot: str
|
||||
binding_target: str
|
||||
parser: str
|
||||
unit: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class DeliveredQuestion:
|
||||
"""A review-gated, question-only ASK artifact wrapping a rendered Q1-C question.
|
||||
|
||||
``question_only`` is its OWN lane (an intake request), not the ``proposal_only``
|
||||
lane — the analogy to :class:`~core.comprehension_attempt.proposal.FailureProposal`
|
||||
is structural (toothless, review-gated, never served), never semantic.
|
||||
|
||||
The invariant fields are enforced in ``__post_init__`` so even a hand-constructed
|
||||
instance cannot become a contentless question, a served question, or an
|
||||
answer-bound (Q2) question — illegal states are unrepresentable:
|
||||
|
||||
- it can never wrap an ``unrenderable`` question (the D2 guard, structurally) —
|
||||
so a ``QUESTION_NEEDED`` terminal always carries real, rendered text;
|
||||
- it can never be ``served`` (off-serving; ``ask_serving_enabled`` does not exist);
|
||||
- it always ``requires_review``;
|
||||
- its ``answer_binding`` is always ``None`` in Q1-D (the Q2 seat is reserved, unbound).
|
||||
"""
|
||||
|
||||
question: EpistemicQuestion
|
||||
owner_organ: str | None
|
||||
blocking_reason: str
|
||||
answer_binding: AnswerBinding | None = None
|
||||
status: str = _QUESTION_STATUS
|
||||
requires_review: bool = True
|
||||
served: bool = False
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.question.unrenderable or self.question.text is None:
|
||||
raise ValueError(
|
||||
"a DeliveredQuestion cannot wrap an unrenderable question "
|
||||
f"(reason={self.question.reason!r}); the D2 fallback handles it"
|
||||
)
|
||||
if self.status != _QUESTION_STATUS:
|
||||
raise ValueError(
|
||||
f"question status must be {_QUESTION_STATUS!r}; got {self.status!r}"
|
||||
)
|
||||
if self.served:
|
||||
raise ValueError("a Q1-D delivered question is never served")
|
||||
if not self.requires_review:
|
||||
raise ValueError("a delivered question always requires review")
|
||||
if self.answer_binding is not None:
|
||||
raise ValueError("answer_binding is reserved for Q2; Q1-D emits None")
|
||||
|
||||
def content_hash(self) -> str:
|
||||
"""Deterministic content address: same question on the same blocking family and
|
||||
slot always yields the same hash (idempotent sink writes). No clock, no
|
||||
randomness. The rendered ``text`` is included so a template change re-addresses.
|
||||
"""
|
||||
slot_name = self.question.slot.slot_name if self.question.slot else ""
|
||||
payload = f"{self.blocking_reason}:{slot_name}:{self.question.text}"
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||
|
||||
def to_json_dict(self) -> dict[str, Any]:
|
||||
slot = self.question.slot
|
||||
return {
|
||||
"status": self.status,
|
||||
"blocking_reason": self.blocking_reason,
|
||||
"owner_organ": self.owner_organ,
|
||||
"question": {
|
||||
"text": self.question.text,
|
||||
"reason": self.question.reason,
|
||||
"slot_name": slot.slot_name if slot else None,
|
||||
"expected_unit_or_type": slot.expected_unit_or_type if slot else None,
|
||||
"binding_target": slot.binding_target if slot else None,
|
||||
},
|
||||
"answer_binding": None, # reserved (Q2)
|
||||
"requires_review": self.requires_review,
|
||||
"served": self.served,
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class DeliveryOutcome:
|
||||
"""The result of routing one ``ask_question`` assessment.
|
||||
|
||||
``question`` is the artifact iff ``terminal is QUESTION_NEEDED`` (a renderable ask);
|
||||
otherwise it is ``None`` and ``fallback_reason`` carries the unrenderable reason that
|
||||
triggered the D2 standing-disposition fallback.
|
||||
"""
|
||||
|
||||
terminal: Terminal
|
||||
question: DeliveredQuestion | None
|
||||
fallback_reason: str | None
|
||||
|
||||
|
||||
def _standing_disposition(blocking_reason: str) -> Terminal:
|
||||
"""The D2 fallback terminal for an unrenderable ask: the family's standing move.
|
||||
|
||||
A family that still proposes (``proposal_allowed``) falls back to
|
||||
``PROPOSAL_EMITTED`` — its existing operational signal, preserved (D3). Anything
|
||||
else (an unknown reason, or a family that does not propose) falls back to
|
||||
``NO_PROGRESS``. Never a contentless ``QUESTION_NEEDED``.
|
||||
"""
|
||||
family = family_by_name(blocking_reason)
|
||||
if family is not None and family.proposal_allowed:
|
||||
return Terminal.PROPOSAL_EMITTED
|
||||
return Terminal.NO_PROGRESS
|
||||
|
||||
|
||||
def deliver_ask(assessment: LimitationAssessment) -> DeliveryOutcome:
|
||||
"""Route an ``ask_question`` assessment to a terminal — render via Q1-C, never here.
|
||||
|
||||
Renderable → ``QUESTION_NEEDED`` + a :class:`DeliveredQuestion` wrapping the Q1-C
|
||||
result verbatim. Unrenderable → the D2 standing-disposition fallback (no artifact).
|
||||
|
||||
Raises ``ValueError`` if called on a non-ASK assessment: the bus routes only
|
||||
``ask_question`` resolutions here, so any other action is a wiring error, not a
|
||||
runtime input — fail loudly rather than silently mis-deliver.
|
||||
"""
|
||||
if assessment.resolution_action != "ask_question":
|
||||
raise ValueError(
|
||||
"deliver_ask is only valid for ask_question assessments; got "
|
||||
f"{assessment.resolution_action!r}"
|
||||
)
|
||||
|
||||
question = render_question(assessment)
|
||||
if question.unrenderable:
|
||||
return DeliveryOutcome(
|
||||
terminal=_standing_disposition(assessment.blocking_reason),
|
||||
question=None,
|
||||
fallback_reason=question.reason,
|
||||
)
|
||||
|
||||
delivered = DeliveredQuestion(
|
||||
question=question,
|
||||
owner_organ=assessment.owner_organ,
|
||||
blocking_reason=assessment.blocking_reason,
|
||||
)
|
||||
return DeliveryOutcome(
|
||||
terminal=Terminal.QUESTION_NEEDED,
|
||||
question=delivered,
|
||||
fallback_reason=None,
|
||||
)
|
||||
|
||||
|
||||
def default_question_root() -> Path:
|
||||
"""``<repo>/teaching/questions`` — the write-only, review-gated ASK sink.
|
||||
|
||||
A sibling of ``teaching/proposals`` (D4): questions are intake requests, not
|
||||
capability proposals, so they do not overload the proposal pile.
|
||||
"""
|
||||
return Path(__file__).resolve().parents[2] / "teaching" / "questions"
|
||||
|
||||
|
||||
def question_path(delivered: DeliveredQuestion, root: Path | None = None) -> Path:
|
||||
base = root if root is not None else default_question_root()
|
||||
return base / f"{delivered.content_hash()}.json"
|
||||
|
||||
|
||||
def emit_question(
|
||||
assessment: LimitationAssessment, *, root: Path | None = None
|
||||
) -> Path | None:
|
||||
"""Deliver an ask assessment and, iff it renders, write the artifact to the sink.
|
||||
|
||||
Returns the artifact path for a ``QUESTION_NEEDED`` delivery, or ``None`` when the
|
||||
ask was unrenderable and fell back (D2) — no contentless artifact is ever written.
|
||||
Idempotent: the same question writes the same content-addressed path with
|
||||
byte-identical content (``sort_keys``). Creates the sink directory on demand.
|
||||
"""
|
||||
outcome = deliver_ask(assessment)
|
||||
if outcome.question is None:
|
||||
return None
|
||||
path = question_path(outcome.question, root)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(
|
||||
json.dumps(outcome.question.to_json_dict(), indent=2, sort_keys=True),
|
||||
encoding="utf-8",
|
||||
)
|
||||
return path
|
||||
|
||||
|
||||
__all__ = [
|
||||
"AnswerBinding",
|
||||
"DeliveredQuestion",
|
||||
"DeliveryOutcome",
|
||||
"default_question_root",
|
||||
"deliver_ask",
|
||||
"emit_question",
|
||||
"question_path",
|
||||
]
|
||||
|
|
@ -1,195 +0,0 @@
|
|||
"""The grounded-only question renderer (Q1-C) — wrong=0-safe by construction.
|
||||
|
||||
This is the renderer rung of the Epistemic Disclosure ASK spine: given a
|
||||
:class:`~core.epistemic_disclosure.limitation.LimitationAssessment` whose
|
||||
``resolution_action == "ask_question"`` and which carries at least one typed
|
||||
:class:`~core.epistemic_disclosure.limitation.MissingSlot`, produce an
|
||||
:class:`EpistemicQuestion` — a rendered user-facing question, or an explicit
|
||||
``question_unrenderable`` verdict. Nothing here delivers, serves, or chooses a
|
||||
disposition (that is Q1-D); this is *only* surface realization of the residue.
|
||||
|
||||
**The wrong=0 invariant (scoping §2 / session §1.5.7) — the whole point.**
|
||||
|
||||
A question may name an entity, slot, unit, or relation only if it appears
|
||||
*verbatim* in the assessment's ``grounded_terms``. When the grounded terms
|
||||
lack what a question needs, degrade to a generic question or emit
|
||||
``question_unrenderable`` — never a named guess.
|
||||
|
||||
Two substrate facts force the conservative policy below:
|
||||
|
||||
1. ``grounded_terms`` is empty for every assessment produced today — the readers
|
||||
do not yet emit verbatim evidence on refusal (scoping §3, the substrate gap
|
||||
Q1 must close first). So there is no grounded problem-entity to name.
|
||||
2. A *missing* slot's referent is, by definition, absent from the comprehension
|
||||
trace — the missing thing can never appear in ``grounded_terms`` even once
|
||||
readers do emit evidence. ``grounded_terms`` can only supply *context*
|
||||
entities, and binding a slot to its context entity needs an alignment step
|
||||
that Q1-C does not have (a later rung).
|
||||
|
||||
**Chosen rendering policy — generic-structural, names zero problem entities.**
|
||||
|
||||
Because Q1-C can neither (today) read grounded context nor (ever, for the slot
|
||||
itself) name the missing referent from grounded text, the only wrong=0-safe
|
||||
artifact it can render is a *generic* question whose sole variable content is a
|
||||
controlled English phrase for the slot's structural *type*
|
||||
(``expected_unit_or_type``), drawn from the closed, audited
|
||||
:data:`_CLOSED_TYPE_PHRASES` map below. Concretely the renderer:
|
||||
|
||||
- NEVER surfaces ``slot_name`` or ``binding_target`` — these are snake_case
|
||||
structural identifiers, and user-facing prose must never come from them
|
||||
(limitation.py ``MissingSlot`` docstring; session §1.5.7). ``slot_name`` is
|
||||
also the field most likely to *read* like a fabricated entity (``ben_rate`` →
|
||||
the forbidden "Ben"), so it is never touched.
|
||||
- NEVER prettifies a snake_case identifier into a natural-language entity — no
|
||||
capitalization, no possessive, no splitting on underscores.
|
||||
- Translates ``expected_unit_or_type`` through the closed map only; an unmapped
|
||||
type degrades to ``question_unrenderable`` (a ``renderability_gap``) rather
|
||||
than dumping raw snake_case or guessing.
|
||||
- Names no problem-specific entity at all. The closed type phrases ("a
|
||||
whole-number count") are generic structural descriptors that assert nothing
|
||||
about *this* problem — distinct from problem-specific names, which would need
|
||||
grounding. This is the line scoping §2 draws ("generic, all terms grounded"
|
||||
vs. the fabricated "Ben").
|
||||
|
||||
A post-render fabrication guard (:func:`_names_only_grounded`) re-checks that
|
||||
every word in the rendered text is closed-vocabulary scaffold or appears in
|
||||
``grounded_terms`` — defense in depth so a fabricated token can never escape even
|
||||
if the template were later edited carelessly.
|
||||
|
||||
**Off-serving.** Imports nothing from ``generate.derivation`` or
|
||||
``core.reliability_gate``; it cannot move the sealed GSM8K metric.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from core.epistemic_disclosure.limitation import (
|
||||
LimitationAssessment,
|
||||
MissingSlot,
|
||||
)
|
||||
|
||||
#: Closed, audited map from a family-pinned ``expected_unit_or_type`` to a
|
||||
#: controlled English phrase. Keys are exactly the slot types that ship today
|
||||
#: (``core.epistemic_disclosure.limitation._FAMILY_TO_MISSING_SLOTS``). A slot
|
||||
#: whose type is absent here is NOT rendered — the renderer refuses with a
|
||||
#: ``renderability_gap`` rather than surfacing raw snake_case. New slot types
|
||||
#: earn a phrase here (with a test) when their family lands, never by guessing.
|
||||
_CLOSED_TYPE_PHRASES: dict[str, str] = {
|
||||
"count_int": "a whole-number count",
|
||||
"measured_unit_int": "a whole-number quantity",
|
||||
}
|
||||
|
||||
#: The fixed question scaffold. The only hole is the closed type phrase; every
|
||||
#: other word is constant, problem-independent English. It names no entity.
|
||||
_TEMPLATE = (
|
||||
"To answer this, one more value is still needed — {phrase} — that the "
|
||||
"problem does not state. What is it?"
|
||||
)
|
||||
|
||||
#: Machine reasons for an :class:`EpistemicQuestion`. Closed set.
|
||||
_REASON_RENDERED = "rendered"
|
||||
_REASON_NOT_ASK = "not_ask"
|
||||
_REASON_NO_SLOT = "no_missing_slot"
|
||||
_REASON_MULTI_SLOT = "multi_slot_not_supported"
|
||||
_REASON_RENDERABILITY_GAP = "renderability_gap"
|
||||
_REASON_FABRICATION_GUARD = "fabrication_guard"
|
||||
|
||||
|
||||
def _tokens(text: str) -> set[str]:
|
||||
"""Lowercased maximal alphabetic runs — the unit the fabrication guard checks."""
|
||||
return set(re.findall(r"[a-z]+", text.lower()))
|
||||
|
||||
|
||||
#: Every word the renderer is allowed to emit *without* grounding: the scaffold
|
||||
#: words plus the words of every closed type phrase. Built once at import.
|
||||
_ALLOWED_SCAFFOLD_WORDS: frozenset[str] = frozenset(
|
||||
_tokens(_TEMPLATE.replace("{phrase}", " "))
|
||||
| {w for phrase in _CLOSED_TYPE_PHRASES.values() for w in _tokens(phrase)}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class EpistemicQuestion:
|
||||
"""The rendered ASK artifact, or an explicit unrenderable verdict.
|
||||
|
||||
``slot`` is the bound :class:`MissingSlot` the question is about — present
|
||||
whenever a slot was selected, ``None`` when the assessment carried no slot to
|
||||
bind (non-ASK, or zero slots). ``text`` is the rendered question, or ``None``
|
||||
when ``unrenderable``. ``reason`` is a closed-set machine string (one of the
|
||||
``_REASON_*`` constants) explaining the verdict; for a renderable question it
|
||||
is :data:`_REASON_RENDERED`.
|
||||
"""
|
||||
|
||||
slot: MissingSlot | None
|
||||
text: str | None
|
||||
unrenderable: bool
|
||||
reason: str
|
||||
|
||||
|
||||
def _unrenderable(reason: str, slot: MissingSlot | None = None) -> EpistemicQuestion:
|
||||
"""A ``question_unrenderable`` verdict with no text."""
|
||||
return EpistemicQuestion(slot=slot, text=None, unrenderable=True, reason=reason)
|
||||
|
||||
|
||||
def _names_only_grounded(text: str, grounded_terms: tuple[str, ...]) -> bool:
|
||||
"""True iff every word in ``text`` is closed-vocab scaffold or grounded.
|
||||
|
||||
The wrong=0 guard, enforced post-render as defense in depth: a fabricated
|
||||
entity (a word neither in the closed scaffold/phrase vocabulary nor verbatim
|
||||
in ``grounded_terms``) makes this return ``False``, and the renderer refuses.
|
||||
With today's empty ``grounded_terms`` and a fully closed-vocab template this
|
||||
holds by construction; the guard exists so that can never silently change.
|
||||
"""
|
||||
allowed = _ALLOWED_SCAFFOLD_WORDS | _tokens(" ".join(grounded_terms))
|
||||
return _tokens(text) <= allowed
|
||||
|
||||
|
||||
def render_question(assessment: LimitationAssessment) -> EpistemicQuestion:
|
||||
"""Render a single-slot generic ASK question, or refuse to render.
|
||||
|
||||
Strictly single-slot: Q1-C renders only when the assessment carries
|
||||
*exactly one* missing slot. The fixed template asserts "one more value is
|
||||
still needed" — a globally-quantified claim that exactly one value is
|
||||
missing — so rendering the first of several slots and ignoring the rest
|
||||
would make that sentence subtly false (it would imply the single rendered
|
||||
value closes the gap when others remain). Rather than weaken the template to
|
||||
an honest-but-vaguer plural, a multi-slot assessment refuses with
|
||||
``multi_slot_not_supported`` (slot ``None``); one-question-per-slot fan-out
|
||||
is a later rung. The renderer also refuses (``question_unrenderable``) when
|
||||
the assessment is not an ASK, carries no slot, or the slot's structural type
|
||||
is outside the closed phrase map. It NEVER fabricates a natural-language
|
||||
entity name — see the module docstring for the policy and the wrong=0
|
||||
rationale.
|
||||
"""
|
||||
if assessment.resolution_action != "ask_question":
|
||||
return _unrenderable(_REASON_NOT_ASK)
|
||||
if not assessment.missing_slots:
|
||||
return _unrenderable(_REASON_NO_SLOT)
|
||||
if len(assessment.missing_slots) > 1:
|
||||
# Strictly single-slot: the template claims exactly one value is
|
||||
# missing, which is false when several slots remain. Refuse rather than
|
||||
# render the first and silently drop the rest.
|
||||
return _unrenderable(_REASON_MULTI_SLOT, slot=None)
|
||||
|
||||
slot = assessment.missing_slots[0]
|
||||
phrase = _CLOSED_TYPE_PHRASES.get(slot.expected_unit_or_type)
|
||||
if phrase is None:
|
||||
# Unknown structural type: refuse rather than surface raw snake_case.
|
||||
return _unrenderable(_REASON_RENDERABILITY_GAP, slot=slot)
|
||||
|
||||
text = _TEMPLATE.format(phrase=phrase)
|
||||
if not _names_only_grounded(text, assessment.grounded_terms):
|
||||
# Unreachable by construction; the guard is the wrong=0 backstop.
|
||||
return _unrenderable(_REASON_FABRICATION_GUARD, slot=slot)
|
||||
|
||||
return EpistemicQuestion(
|
||||
slot=slot, text=text, unrenderable=False, reason=_REASON_RENDERED
|
||||
)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"EpistemicQuestion",
|
||||
"render_question",
|
||||
]
|
||||
|
|
@ -1,32 +0,0 @@
|
|||
"""ASK serving gate helper — default-dark, no served-surface wiring.
|
||||
|
||||
This module is the first code slice after the ASK serving-integration scoping brief.
|
||||
It intentionally does **not** call ``deliver_ask``/``emit_question``, does not import
|
||||
``chat.runtime``, and does not expose any user-facing surface. It only centralizes the
|
||||
kill-switch read so future serving code has one audited predicate.
|
||||
|
||||
The planned config field is ``RuntimeConfig.ask_serving_enabled``. During this dark-gate
|
||||
slice the predicate is conservative: absent field == ``False``. That lets the helper land
|
||||
without widening behavior and preserves the current default for every existing
|
||||
``RuntimeConfig`` instance.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from core.config import DEFAULT_CONFIG, RuntimeConfig
|
||||
|
||||
|
||||
def ask_serving_enabled(config: RuntimeConfig | Any | None = None) -> bool:
|
||||
"""Return whether served ASK delivery is explicitly enabled.
|
||||
|
||||
Missing attribute means ``False``. That is the load-bearing dark-gate invariant:
|
||||
the served ASK path cannot light merely because the helper exists or because an
|
||||
older ``RuntimeConfig`` instance lacks the future field.
|
||||
"""
|
||||
cfg = DEFAULT_CONFIG if config is None else config
|
||||
return bool(getattr(cfg, "ask_serving_enabled", False))
|
||||
|
||||
|
||||
__all__ = ["ask_serving_enabled"]
|
||||
|
|
@ -15,7 +15,6 @@ from core.learning_arena.protocols import (
|
|||
DomainSolver,
|
||||
GoldTether,
|
||||
Problem,
|
||||
Tier2Verifier,
|
||||
)
|
||||
from core.learning_arena.report import (
|
||||
REFUSAL_DIAGNOSES,
|
||||
|
|
@ -32,7 +31,6 @@ __all__ = [
|
|||
"DomainSolver",
|
||||
"GoldTether",
|
||||
"Problem",
|
||||
"Tier2Verifier",
|
||||
"REFUSAL_DIAGNOSES",
|
||||
"EliminationRecord",
|
||||
"PracticeReport",
|
||||
|
|
|
|||
|
|
@ -15,20 +15,14 @@ Invariants (the load-bearing ADR-0199 mandates, enforced structurally here):
|
|||
never touches a serving path or the active teaching corpus. Promotion is the
|
||||
caller's separate ``propose_from_ledger`` step into the reviewed corridor.
|
||||
- **L-4 (determinism).** Pure fold over the input order; identical
|
||||
(problems, solver, tether, diagnose, tier2_verifier) -> identical report.
|
||||
(problems, solver, tether, diagnose) -> identical report.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Callable, Sequence
|
||||
|
||||
from core.learning_arena.protocols import (
|
||||
Attempt,
|
||||
DomainProblem,
|
||||
DomainSolver,
|
||||
GoldTether,
|
||||
Tier2Verifier,
|
||||
)
|
||||
from core.learning_arena.protocols import Attempt, DomainProblem, DomainSolver, GoldTether
|
||||
from core.learning_arena.report import EliminationRecord, PracticeReport
|
||||
from core.reliability_gate import ClassTally
|
||||
|
||||
|
|
@ -49,7 +43,6 @@ def run_practice(
|
|||
tether: GoldTether,
|
||||
*,
|
||||
diagnose: Callable[[str], str] = _default_diagnose,
|
||||
tier2_verifier: Tier2Verifier | None = None,
|
||||
) -> PracticeReport:
|
||||
"""Sealed practice: attempt -> gold-tether score -> per-class ledger.
|
||||
|
||||
|
|
@ -67,36 +60,21 @@ def run_practice(
|
|||
for problem in problems:
|
||||
cls = problem.class_name
|
||||
attempt: Attempt = solver.attempt(problem)
|
||||
gold_correct = False
|
||||
|
||||
if not attempt.committed:
|
||||
verdict = "refused"
|
||||
elif tether.is_correct(attempt, problem):
|
||||
verdict = "correct"
|
||||
else:
|
||||
gold_correct = tether.is_correct(attempt, problem)
|
||||
verdict = "correct" if gold_correct else "wrong"
|
||||
verdict = "wrong"
|
||||
|
||||
counts[verdict] = counts.get(verdict, 0) + 1
|
||||
tally = ledger.get(cls) or ClassTally(cls)
|
||||
t2_verified = 0
|
||||
t2_agrees_gold = 0
|
||||
if attempt.committed and tier2_verifier is not None:
|
||||
t2_verdict = tier2_verifier.verify(attempt, problem)
|
||||
if t2_verdict.verified:
|
||||
t2_verified = 1
|
||||
t2_agrees_gold = 1 if gold_correct else 0
|
||||
|
||||
if verdict == "correct":
|
||||
tally = tally.record(
|
||||
correct=1,
|
||||
t2_verified=t2_verified,
|
||||
t2_agrees_gold=t2_agrees_gold,
|
||||
)
|
||||
tally = tally.record(correct=1)
|
||||
elif verdict == "wrong":
|
||||
tally = tally.record(
|
||||
wrong=1,
|
||||
t2_verified=t2_verified,
|
||||
t2_agrees_gold=t2_agrees_gold,
|
||||
)
|
||||
tally = tally.record(wrong=1)
|
||||
elims.append(
|
||||
EliminationRecord(
|
||||
case_id=attempt.case_id,
|
||||
|
|
|
|||
|
|
@ -21,8 +21,6 @@ from __future__ import annotations
|
|||
from dataclasses import dataclass, field
|
||||
from typing import Any, Protocol, runtime_checkable
|
||||
|
||||
from core.reasoning.evidence import Tier2Verdict
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class DomainProblem(Protocol):
|
||||
|
|
@ -88,15 +86,6 @@ class GoldTether(Protocol):
|
|||
def gold_answer(self, problem: DomainProblem) -> Any: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class Tier2Verifier(Protocol):
|
||||
"""Optional convergent self-verifier for a domain attempt."""
|
||||
|
||||
domain_id: str
|
||||
|
||||
def verify(self, attempt: Attempt, problem: DomainProblem) -> Tier2Verdict: ...
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Problem:
|
||||
"""Concrete :class:`DomainProblem` a domain adapter can build directly."""
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
"""ADR-0199 / ADR-0175 — the domain-agnostic practice report.
|
||||
|
||||
Extracted from ``evals/gsm8k_math/practice/v1/runner.py`` so every subject's
|
||||
arena emits the same report shape. The report now also exposes the existing
|
||||
Tier-2 ledger counts when a domain supplies convergent self-verification
|
||||
evidence; domains without a Tier-2 verifier report zeros.
|
||||
Extracted verbatim (schema-preserving) from
|
||||
``evals/gsm8k_math/practice/v1/runner.py`` so every subject's arena emits the
|
||||
same report shape. ``PracticeReport.as_dict`` is byte-stable with the original
|
||||
GSM8K report so existing goldens and ``report.json`` are unaffected.
|
||||
|
||||
The three refusal-diagnosis axes are the universal ADR-0175 §8 router
|
||||
(skill / knowledge / ambiguity), not a domain quantity — so they live here.
|
||||
|
|
@ -59,9 +59,6 @@ class PracticeReport:
|
|||
"committed": t.committed,
|
||||
"reliability": t.reliability,
|
||||
"coverage": t.coverage,
|
||||
"t2_verified": t.t2_verified,
|
||||
"t2_agrees_gold": t.t2_agrees_gold,
|
||||
"t2_precision": t.t2_precision,
|
||||
}
|
||||
for cls, t in sorted(self.ledger.items())
|
||||
},
|
||||
|
|
|
|||
|
|
@ -1,38 +0,0 @@
|
|||
"""Read-only proposal review reporter (RPT) — surfaces comprehension-failure proposals for review.
|
||||
|
||||
Observes ``teaching/proposals/comprehension_failures/*.json`` (emitted by the contemplation pass,
|
||||
N5), validates them, and reports pending review obligations. It is **read-only**: it does not
|
||||
advance the teaching loop, ratify, mount, modify readers, or affect serving. It is **not** an
|
||||
``idle_tick`` (``ChatRuntime.idle_tick`` remains the only one) and **not** L10 — it is the review
|
||||
surface that keeps proposal artifacts from becoming inert files. A future PR may call this reporter
|
||||
from ``idle_tick`` as a read-only sub-pass.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from core.proposal_review.model import MalformedArtifact, PendingProposal
|
||||
from core.proposal_review.report import (
|
||||
ProposalReviewReport,
|
||||
build_report,
|
||||
report_json,
|
||||
report_text,
|
||||
)
|
||||
from core.proposal_review.safety import SafetyVerdict, dry_check
|
||||
from core.proposal_review.scan import DEFAULT_SINK, default_sink, scan
|
||||
from core.proposal_review.summary import ProposalReviewIdleSummary, idle_summary
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_SINK",
|
||||
"MalformedArtifact",
|
||||
"PendingProposal",
|
||||
"ProposalReviewIdleSummary",
|
||||
"ProposalReviewReport",
|
||||
"SafetyVerdict",
|
||||
"build_report",
|
||||
"default_sink",
|
||||
"dry_check",
|
||||
"idle_summary",
|
||||
"report_json",
|
||||
"report_text",
|
||||
"scan",
|
||||
]
|
||||
|
|
@ -1,60 +0,0 @@
|
|||
"""CLI for the read-only proposal review reporter (RPT-c).
|
||||
|
||||
python -m core.proposal_review # text report + safety dry-check
|
||||
python -m core.proposal_review comprehension-failures
|
||||
python -m core.proposal_review --json # machine-readable
|
||||
python -m core.proposal_review --root <path> # override the sink
|
||||
|
||||
Read-only: scans, reports, and dry-checks the comprehension-failure proposal sink. **Mutates
|
||||
nothing.** Exit 0 iff the safety dry-check passes (every artifact inert + serving unconsumed).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
from core.proposal_review.report import build_report, report_text
|
||||
from core.proposal_review.safety import dry_check
|
||||
from core.proposal_review.scan import scan
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="python -m core.proposal_review",
|
||||
description="Read-only comprehension-failure proposal review (observes; never mounts/ratifies).",
|
||||
)
|
||||
parser.add_argument(
|
||||
"target", nargs="?", default="comprehension-failures", choices=["comprehension-failures"]
|
||||
)
|
||||
parser.add_argument("--json", action="store_true", help="emit JSON instead of text")
|
||||
parser.add_argument("--root", default=None, help="override the sink path")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
root = Path(args.root) if args.root else None
|
||||
proposals, malformed = scan(root)
|
||||
report = build_report(proposals, malformed)
|
||||
verdict = dry_check(proposals, malformed, root=root)
|
||||
|
||||
if args.json:
|
||||
print(
|
||||
json.dumps(
|
||||
{
|
||||
"report": report.to_json_dict(),
|
||||
"safety": {"ok": verdict.ok, "violations": list(verdict.violations)},
|
||||
},
|
||||
indent=2,
|
||||
sort_keys=True,
|
||||
)
|
||||
)
|
||||
else:
|
||||
print(report_text(report))
|
||||
print(f" safety: {'OK' if verdict.ok else f'VIOLATIONS ({len(verdict.violations)})'}")
|
||||
for v in verdict.violations:
|
||||
print(f" ! {v}")
|
||||
return 0 if verdict.ok else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -1,39 +0,0 @@
|
|||
"""Typed records for the read-only proposal review reporter (RPT-a).
|
||||
|
||||
A ``PendingProposal`` is the parsed view of one ``teaching/proposals/comprehension_failures/
|
||||
<hash>.json`` artifact (emitted by the contemplation pass, N5). A ``MalformedArtifact`` is a file
|
||||
that could not be parsed into one. Pure value data — the reporter never mutates the sink.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class PendingProposal:
|
||||
"""A parsed proposal artifact awaiting human review. ``content_hash`` is the filename stem
|
||||
(the content address). Safety fields (``status`` / ``mounted`` / ``requires_review``) are
|
||||
carried verbatim so the dry-check (RPT-c) can verify them independently of the emitter."""
|
||||
|
||||
path: str
|
||||
content_hash: str
|
||||
failure_family: str
|
||||
status: str
|
||||
mounted: bool
|
||||
requires_review: bool
|
||||
problem_text_sha256: str
|
||||
observed_attempts: tuple[dict[str, Any], ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class MalformedArtifact:
|
||||
"""A file under the sink that is not a parseable proposal (bad JSON / missing or mistyped
|
||||
fields). Surfaced so a human notices corruption rather than the reporter silently skipping it."""
|
||||
|
||||
path: str
|
||||
reason: str
|
||||
|
||||
|
||||
__all__ = ["MalformedArtifact", "PendingProposal"]
|
||||
|
|
@ -1,80 +0,0 @@
|
|||
"""Deterministic review report over the scanned proposals (RPT-b).
|
||||
|
||||
A pure projection of the scan results into a summary: total pending, counts by failure_family,
|
||||
counts by status, the malformed count, and the review-needed list. Deterministic given the sink
|
||||
contents (no clock): counts are sorted, the review-needed list is sorted by content hash.
|
||||
|
||||
Time-based "oldest/newest" is intentionally **omitted**: the proposal artifacts are
|
||||
content-addressed and carry no timestamp (the emitter is clock-free for idempotence), so an honest
|
||||
temporal ordering is not available from the data — only from non-deterministic filesystem mtime,
|
||||
which would make this report non-deterministic. A human can sort the sink by mtime if needed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from core.proposal_review.model import MalformedArtifact, PendingProposal
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProposalReviewReport:
|
||||
"""A deterministic snapshot of the review obligations in the proposal sink."""
|
||||
|
||||
total: int
|
||||
by_family: dict[str, int]
|
||||
by_status: dict[str, int]
|
||||
malformed: int
|
||||
review_needed: tuple[str, ...]
|
||||
|
||||
def to_json_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"total": self.total,
|
||||
"by_family": self.by_family,
|
||||
"by_status": self.by_status,
|
||||
"malformed": self.malformed,
|
||||
"review_needed": list(self.review_needed),
|
||||
}
|
||||
|
||||
|
||||
def build_report(
|
||||
proposals: list[PendingProposal], malformed: list[MalformedArtifact]
|
||||
) -> ProposalReviewReport:
|
||||
by_family: dict[str, int] = {}
|
||||
by_status: dict[str, int] = {}
|
||||
review_needed: list[str] = []
|
||||
for p in proposals:
|
||||
by_family[p.failure_family] = by_family.get(p.failure_family, 0) + 1
|
||||
by_status[p.status] = by_status.get(p.status, 0) + 1
|
||||
if p.requires_review:
|
||||
review_needed.append(p.content_hash)
|
||||
return ProposalReviewReport(
|
||||
total=len(proposals),
|
||||
by_family=dict(sorted(by_family.items())),
|
||||
by_status=dict(sorted(by_status.items())),
|
||||
malformed=len(malformed),
|
||||
review_needed=tuple(sorted(review_needed)),
|
||||
)
|
||||
|
||||
|
||||
def report_json(report: ProposalReviewReport) -> str:
|
||||
"""Deterministic JSON (sorted keys)."""
|
||||
return json.dumps(report.to_json_dict(), indent=2, sort_keys=True)
|
||||
|
||||
|
||||
def report_text(report: ProposalReviewReport) -> str:
|
||||
"""Human-readable summary."""
|
||||
lines = [
|
||||
f"comprehension-failure proposals: {report.total} pending · {report.malformed} malformed",
|
||||
" by family:",
|
||||
*(f" {fam}: {n}" for fam, n in report.by_family.items()),
|
||||
" by status:",
|
||||
*(f" {status}: {n}" for status, n in report.by_status.items()),
|
||||
f" review-needed: {len(report.review_needed)}",
|
||||
]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
__all__ = ["ProposalReviewReport", "build_report", "report_json", "report_text"]
|
||||
|
|
@ -1,102 +0,0 @@
|
|||
"""Safety dry-check over the proposal sink (RPT-c) — the load-bearing part of the reporter.
|
||||
|
||||
The reporter's value is not just visibility; it is **independent safety verification**. The
|
||||
dry-check confirms — without trusting the emitter — that every artifact in the sink is inert:
|
||||
|
||||
```text
|
||||
status == "proposal_only"
|
||||
mounted == false
|
||||
requires_review == true
|
||||
content-address consistent: filename == sha256(failure_family : problem_text_sha256)
|
||||
path under the sink
|
||||
no malformed artifact (an unverifiable file in a safety-critical sink is a violation)
|
||||
serving never imports/reads the sink
|
||||
```
|
||||
|
||||
Any failure is a violation; the CLI exits non-zero. **Pure read** — verifies, never repairs.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from core.proposal_review.model import MalformedArtifact, PendingProposal
|
||||
from core.proposal_review.scan import DEFAULT_SINK
|
||||
|
||||
#: Serving-path roots that must never read the proposal sink (CLAUDE.md forbidden/serving sites).
|
||||
_SERVING_TARGETS = (
|
||||
("generate", "stream.py"),
|
||||
("field", "propagate.py"),
|
||||
("vault", "store.py"),
|
||||
("generate", "derivation"),
|
||||
("core", "reliability_gate"),
|
||||
)
|
||||
_SINK_MARKER = "comprehension_failures"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SafetyVerdict:
|
||||
"""The outcome of the dry-check: ``ok`` iff there are no violations."""
|
||||
|
||||
ok: bool
|
||||
violations: tuple[str, ...]
|
||||
|
||||
|
||||
def _repo_root() -> Path:
|
||||
return Path(__file__).resolve().parents[2]
|
||||
|
||||
|
||||
def _serving_references_sink(repo_root: Path) -> list[str]:
|
||||
violations: list[str] = []
|
||||
for parts in _SERVING_TARGETS:
|
||||
target = repo_root.joinpath(*parts)
|
||||
if not target.exists():
|
||||
continue
|
||||
files = [target] if target.is_file() else sorted(target.rglob("*.py"))
|
||||
for f in files:
|
||||
try:
|
||||
if _SINK_MARKER in f.read_text(encoding="utf-8"):
|
||||
violations.append(f"serving reads the sink: {f.relative_to(repo_root)}")
|
||||
except (UnicodeDecodeError, OSError): # pragma: no cover - defensive
|
||||
continue
|
||||
return violations
|
||||
|
||||
|
||||
def dry_check(
|
||||
proposals: list[PendingProposal],
|
||||
malformed: list[MalformedArtifact],
|
||||
*,
|
||||
root: Path | None = None,
|
||||
repo_root: Path | None = None,
|
||||
) -> SafetyVerdict:
|
||||
"""Verify every artifact is inert and the sink is serving-unconsumed. Returns a SafetyVerdict."""
|
||||
base = (root if root is not None else DEFAULT_SINK).resolve()
|
||||
violations: list[str] = []
|
||||
|
||||
for p in proposals:
|
||||
tag = p.content_hash
|
||||
if p.status != "proposal_only":
|
||||
violations.append(f"{tag}: status={p.status!r} (must be 'proposal_only')")
|
||||
if p.mounted:
|
||||
violations.append(f"{tag}: mounted=True (must be False)")
|
||||
if not p.requires_review:
|
||||
violations.append(f"{tag}: requires_review=False (must be True)")
|
||||
expected = hashlib.sha256(
|
||||
f"{p.failure_family}:{p.problem_text_sha256}".encode("utf-8")
|
||||
).hexdigest()
|
||||
if p.content_hash != expected:
|
||||
violations.append(f"{tag}: content-address mismatch (expected {expected})")
|
||||
if not str(Path(p.path).resolve()).startswith(str(base)):
|
||||
violations.append(f"{tag}: path outside the sink")
|
||||
|
||||
for m in malformed:
|
||||
violations.append(f"malformed (unverifiable): {m.path} — {m.reason}")
|
||||
|
||||
violations.extend(_serving_references_sink(repo_root if repo_root is not None else _repo_root()))
|
||||
|
||||
return SafetyVerdict(not violations, tuple(violations))
|
||||
|
||||
|
||||
__all__ = ["SafetyVerdict", "dry_check"]
|
||||
|
|
@ -1,85 +0,0 @@
|
|||
"""Read-only scanner over the comprehension-failure proposal sink (RPT-a).
|
||||
|
||||
Reads ``teaching/proposals/comprehension_failures/*.json`` into typed ``PendingProposal`` records;
|
||||
any file that does not parse into one is reported as a ``MalformedArtifact`` (never silently
|
||||
dropped). **Pure read** — opens files, never writes/moves/deletes. Deterministic: results are
|
||||
sorted by filename. The sink location is computed here independently of the emitter, so the
|
||||
reporter verifies the artifact contract without importing the writer.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from core.proposal_review.model import MalformedArtifact, PendingProposal
|
||||
|
||||
#: The proposal sink the contemplation pass (N5) writes to — known independently of the emitter.
|
||||
DEFAULT_SINK = (
|
||||
Path(__file__).resolve().parents[2] / "teaching" / "proposals" / "comprehension_failures"
|
||||
)
|
||||
|
||||
#: Required fields and their JSON types for a well-formed proposal artifact.
|
||||
_REQUIRED: tuple[tuple[str, type | tuple[type, ...]], ...] = (
|
||||
("status", str),
|
||||
("failure_family", str),
|
||||
("problem_text_sha256", str),
|
||||
("mounted", bool),
|
||||
("requires_review", bool),
|
||||
("observed_attempts", list),
|
||||
)
|
||||
|
||||
|
||||
def default_sink() -> Path:
|
||||
return DEFAULT_SINK
|
||||
|
||||
|
||||
def _parse(path: Path) -> PendingProposal | MalformedArtifact:
|
||||
try:
|
||||
raw = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (json.JSONDecodeError, UnicodeDecodeError) as exc:
|
||||
return MalformedArtifact(str(path), f"invalid_json: {exc}")
|
||||
if not isinstance(raw, dict):
|
||||
return MalformedArtifact(str(path), "not_a_json_object")
|
||||
for key, typ in _REQUIRED:
|
||||
if key not in raw:
|
||||
return MalformedArtifact(str(path), f"missing_field: {key}")
|
||||
# bool is a subclass of int; check bools explicitly so 0/1 don't pass as bool.
|
||||
if typ is bool and not isinstance(raw[key], bool):
|
||||
return MalformedArtifact(str(path), f"bad_type: {key}")
|
||||
if typ is not bool and not isinstance(raw[key], typ):
|
||||
return MalformedArtifact(str(path), f"bad_type: {key}")
|
||||
attempts: tuple[dict[str, Any], ...] = tuple(
|
||||
a for a in raw["observed_attempts"] if isinstance(a, dict)
|
||||
)
|
||||
return PendingProposal(
|
||||
path=str(path),
|
||||
content_hash=path.stem,
|
||||
failure_family=raw["failure_family"],
|
||||
status=raw["status"],
|
||||
mounted=raw["mounted"],
|
||||
requires_review=raw["requires_review"],
|
||||
problem_text_sha256=raw["problem_text_sha256"],
|
||||
observed_attempts=attempts,
|
||||
)
|
||||
|
||||
|
||||
def scan(root: Path | None = None) -> tuple[list[PendingProposal], list[MalformedArtifact]]:
|
||||
"""Scan the sink (default: the comprehension-failure sink). Returns ``(proposals, malformed)``,
|
||||
each sorted by path. A missing sink yields two empty lists (nothing to review yet)."""
|
||||
base = root if root is not None else DEFAULT_SINK
|
||||
if not base.exists():
|
||||
return [], []
|
||||
proposals: list[PendingProposal] = []
|
||||
malformed: list[MalformedArtifact] = []
|
||||
for path in sorted(base.glob("*.json")):
|
||||
parsed = _parse(path)
|
||||
if isinstance(parsed, PendingProposal):
|
||||
proposals.append(parsed)
|
||||
else:
|
||||
malformed.append(parsed)
|
||||
return proposals, malformed
|
||||
|
||||
|
||||
__all__ = ["DEFAULT_SINK", "default_sink", "scan"]
|
||||
|
|
@ -1,52 +0,0 @@
|
|||
"""Pure idle-use summary of the proposal sink (IT-a).
|
||||
|
||||
``idle_summary`` composes the landed read-only pieces — ``scan`` (RPT-a) + ``build_report``
|
||||
(RPT-b) + ``dry_check`` (RPT-c) — into one small, JSON-safe value the runtime's ``idle_tick``
|
||||
can surface (IT-b) without importing the reporter's internals. Pure read: no mutation, no clock.
|
||||
|
||||
The summary is deliberately primitives-only (no paths, no raw file content, no mutable dicts) so
|
||||
it is trivially serializable if ``IdleTickResult`` is ever persisted. ``errors`` carries the
|
||||
reason the sink is not ``safe`` — the dry-check violations here; IT-b additionally uses it to
|
||||
record a captured reporter exception (``proposal_review_failed:<type>``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from core.proposal_review.report import build_report
|
||||
from core.proposal_review.safety import dry_check
|
||||
from core.proposal_review.scan import scan
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProposalReviewIdleSummary:
|
||||
"""A JSON-safe snapshot of the proposal sink for idle surfacing. ``safe`` is the dry-check
|
||||
verdict; ``errors`` is empty iff safe (or carries ``proposal_review_failed:<type>`` when the
|
||||
reporter itself raised, set by the runtime sub-pass)."""
|
||||
|
||||
safe: bool
|
||||
total: int
|
||||
review_needed: int
|
||||
malformed: int
|
||||
by_family: tuple[tuple[str, int], ...]
|
||||
errors: tuple[str, ...] = ()
|
||||
|
||||
|
||||
def idle_summary(root: Path | None = None) -> ProposalReviewIdleSummary:
|
||||
"""Scan → report → dry-check the proposal sink into a JSON-safe idle summary. Pure read."""
|
||||
proposals, malformed = scan(root)
|
||||
report = build_report(proposals, malformed)
|
||||
verdict = dry_check(proposals, malformed, root=root)
|
||||
return ProposalReviewIdleSummary(
|
||||
safe=verdict.ok,
|
||||
total=report.total,
|
||||
review_needed=len(report.review_needed),
|
||||
malformed=report.malformed,
|
||||
by_family=tuple(report.by_family.items()),
|
||||
errors=verdict.violations,
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["ProposalReviewIdleSummary", "idle_summary"]
|
||||
|
|
@ -1,35 +0,0 @@
|
|||
"""Shared deterministic reasoning evidence contracts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from core.reasoning.evidence import (
|
||||
COMMITMENT_DISAGREEMENT,
|
||||
DUPLICATE_STRUCTURAL_SIGNATURE,
|
||||
INSUFFICIENT_EVIDENCE,
|
||||
MISSING_COMMITMENT,
|
||||
SAME_READER_LINEAGE,
|
||||
TIER2_VERIFIED,
|
||||
EvidenceBundle,
|
||||
OperatorEvidence,
|
||||
Tier2Verdict,
|
||||
verify_tier2_agreement,
|
||||
)
|
||||
from core.reasoning.adapters import (
|
||||
evidence_from_entailment_trace,
|
||||
evidence_from_math_solution,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"COMMITMENT_DISAGREEMENT",
|
||||
"DUPLICATE_STRUCTURAL_SIGNATURE",
|
||||
"INSUFFICIENT_EVIDENCE",
|
||||
"MISSING_COMMITMENT",
|
||||
"SAME_READER_LINEAGE",
|
||||
"TIER2_VERIFIED",
|
||||
"EvidenceBundle",
|
||||
"OperatorEvidence",
|
||||
"Tier2Verdict",
|
||||
"verify_tier2_agreement",
|
||||
"evidence_from_entailment_trace",
|
||||
"evidence_from_math_solution",
|
||||
]
|
||||
|
|
@ -1,119 +0,0 @@
|
|||
"""Adapters from existing operator traces into shared reasoning evidence."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from typing import Any
|
||||
|
||||
from core.reasoning.evidence import OperatorEvidence
|
||||
from generate.math_problem_graph import MathProblemGraph
|
||||
from generate.math_solver import SolutionTrace
|
||||
from generate.proof_chain import Entailment, EntailmentTrace
|
||||
|
||||
|
||||
def evidence_from_entailment_trace(trace: EntailmentTrace) -> OperatorEvidence:
|
||||
"""Convert propositional entailment trace evidence to the shared contract."""
|
||||
query_key = trace.query_key or ""
|
||||
check_keys = tuple(
|
||||
key for key in (
|
||||
trace.conjunction_key,
|
||||
trace.entailment_check_key,
|
||||
trace.refutation_check_key,
|
||||
)
|
||||
if key
|
||||
)
|
||||
if trace.outcome is Entailment.REFUSED:
|
||||
commitment_key = ""
|
||||
else:
|
||||
commitment_key = f"entailment:{trace.outcome.value}:{query_key}"
|
||||
structural_signature = _sha256_text(
|
||||
json.dumps(
|
||||
{
|
||||
"operator": "propositional_entailment",
|
||||
"premise_keys": list(trace.premise_keys),
|
||||
"check_keys": list(check_keys),
|
||||
},
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
)
|
||||
return OperatorEvidence(
|
||||
domain="mathematics_logic",
|
||||
operator="propositional_entailment",
|
||||
outcome=trace.outcome.value,
|
||||
reason=trace.reason,
|
||||
input_keys=(*trace.premise_keys, query_key),
|
||||
check_keys=check_keys,
|
||||
commitment_key=commitment_key,
|
||||
structural_signature=structural_signature,
|
||||
reader_lineage="proof_chain.entail",
|
||||
payload={"entailment_trace": trace.as_dict()},
|
||||
)
|
||||
|
||||
|
||||
def evidence_from_math_solution(
|
||||
*,
|
||||
graph: MathProblemGraph,
|
||||
trace: SolutionTrace,
|
||||
reader_trace: tuple[str, ...] = (),
|
||||
operator: str = "math_problem_graph_solve_verify",
|
||||
reason: str = "solver_verifier_passed",
|
||||
) -> OperatorEvidence:
|
||||
"""Convert a verified MathProblemGraph solution to shared evidence."""
|
||||
graph_hash = hashlib.sha256(graph.canonical_bytes()).hexdigest()
|
||||
trace_hash = hashlib.sha256(trace.canonical_bytes()).hexdigest()
|
||||
operation_kinds = tuple(step.operation_kind for step in trace.steps)
|
||||
commitment_key = json.dumps(
|
||||
{
|
||||
"answer_entity": trace.answer_entity,
|
||||
"answer_unit": trace.answer_unit,
|
||||
"answer_value": trace.answer_value,
|
||||
},
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
structural_signature = _sha256_text(
|
||||
json.dumps(
|
||||
{
|
||||
"graph_hash": graph_hash,
|
||||
"operation_kinds": list(operation_kinds),
|
||||
"operator": operator,
|
||||
"pack_id": trace.pack_id,
|
||||
},
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
)
|
||||
return OperatorEvidence(
|
||||
domain="mathematics_logic",
|
||||
operator=operator,
|
||||
outcome="verified",
|
||||
reason=reason,
|
||||
input_keys=(graph_hash,),
|
||||
check_keys=(trace_hash, trace.graph_canonical_hash),
|
||||
commitment_key=commitment_key,
|
||||
structural_signature=structural_signature,
|
||||
reader_lineage="math_problem_graph.solve_verify",
|
||||
payload={
|
||||
"answer_entity": trace.answer_entity,
|
||||
"answer_unit": trace.answer_unit,
|
||||
"answer_value": trace.answer_value,
|
||||
"graph_hash": graph_hash,
|
||||
"operation_kinds": list(operation_kinds),
|
||||
"pack_id": trace.pack_id,
|
||||
"reader_trace": tuple(_reader_event(ev) for ev in reader_trace),
|
||||
"trace_hash": trace_hash,
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _reader_event(event: str) -> Any:
|
||||
try:
|
||||
return json.loads(event)
|
||||
except json.JSONDecodeError:
|
||||
return event
|
||||
|
||||
|
||||
def _sha256_text(text: str) -> str:
|
||||
return hashlib.sha256(text.encode("utf-8")).hexdigest()
|
||||
|
|
@ -1,250 +0,0 @@
|
|||
"""Domain-neutral reasoning evidence and Tier-2 agreement checks.
|
||||
|
||||
This module is deliberately pure data plus deterministic serialization. It is
|
||||
the shared evidence shape for proof, reconstruction, contemplation, and sealed
|
||||
learning arenas; it does not authorize serving behavior by itself.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
from collections import Counter
|
||||
from collections.abc import Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from types import MappingProxyType
|
||||
from typing import Any, Final
|
||||
|
||||
TIER2_VERIFIED: Final[str] = "tier2_verified"
|
||||
INSUFFICIENT_EVIDENCE: Final[str] = "insufficient_evidence"
|
||||
DUPLICATE_STRUCTURAL_SIGNATURE: Final[str] = "duplicate_structural_signature"
|
||||
COMMITMENT_DISAGREEMENT: Final[str] = "commitment_disagreement"
|
||||
MISSING_COMMITMENT: Final[str] = "missing_commitment"
|
||||
SAME_READER_LINEAGE: Final[str] = "same_reader_lineage"
|
||||
|
||||
TIER2_REASONS: Final[frozenset[str]] = frozenset({
|
||||
TIER2_VERIFIED,
|
||||
INSUFFICIENT_EVIDENCE,
|
||||
DUPLICATE_STRUCTURAL_SIGNATURE,
|
||||
COMMITMENT_DISAGREEMENT,
|
||||
MISSING_COMMITMENT,
|
||||
SAME_READER_LINEAGE,
|
||||
})
|
||||
|
||||
|
||||
def _freeze_json_value(value: Any) -> Any:
|
||||
"""Recursively freeze JSON-like payloads for immutable evidence storage."""
|
||||
if isinstance(value, Mapping):
|
||||
frozen = {str(k): _freeze_json_value(v) for k, v in value.items()}
|
||||
return MappingProxyType(frozen)
|
||||
if isinstance(value, list | tuple):
|
||||
return tuple(_freeze_json_value(v) for v in value)
|
||||
if value is None or isinstance(value, str | int | float | bool):
|
||||
return value
|
||||
raise TypeError(f"unsupported evidence payload value: {type(value).__name__}")
|
||||
|
||||
|
||||
def _json_value(value: Any) -> Any:
|
||||
"""Return a JSON-serializable copy of a frozen payload value."""
|
||||
if isinstance(value, Mapping):
|
||||
return {str(k): _json_value(v) for k, v in value.items()}
|
||||
if isinstance(value, list | tuple):
|
||||
return [_json_value(v) for v in value]
|
||||
if value is None or isinstance(value, str | int | float | bool):
|
||||
return value
|
||||
raise TypeError(f"unsupported frozen evidence value: {type(value).__name__}")
|
||||
|
||||
|
||||
def _canonical_json(payload: Mapping[str, Any]) -> str:
|
||||
return json.dumps(
|
||||
_json_value(payload),
|
||||
ensure_ascii=False,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class OperatorEvidence:
|
||||
"""Replayable evidence for one deterministic operator invocation."""
|
||||
|
||||
domain: str
|
||||
operator: str
|
||||
outcome: str
|
||||
reason: str
|
||||
input_keys: tuple[str, ...]
|
||||
check_keys: tuple[str, ...]
|
||||
commitment_key: str
|
||||
structural_signature: str
|
||||
# reader_lineage — the mechanistic identity of the decoding PATHWAY that
|
||||
# produced this evidence (e.g. "proof_chain.entail"). The Tier-2 gate keys
|
||||
# independence on this, not on the cosmetic structural_signature string, so a
|
||||
# single reader cannot self-verify by relabeling. Deliberately EXCLUDED from
|
||||
# ``as_dict``/``canonical_json`` (and therefore the evidence/trace hash): it is
|
||||
# gate-routing provenance, not replayable evidence content, and including it
|
||||
# would perturb the entailment trace_hash with no replay benefit. The
|
||||
# static guarantee that distinct lineages are import-disjoint (cannot share a
|
||||
# decoding pathway) is enforced by the architectural reader-disjointness
|
||||
# invariant, not by this string alone.
|
||||
reader_lineage: str = ""
|
||||
payload: Mapping[str, Any] = field(default_factory=dict)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
for field_name in (
|
||||
"domain",
|
||||
"operator",
|
||||
"outcome",
|
||||
"reason",
|
||||
"structural_signature",
|
||||
"reader_lineage",
|
||||
):
|
||||
value = getattr(self, field_name)
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
raise ValueError(f"OperatorEvidence.{field_name} is required")
|
||||
object.__setattr__(self, field_name, value.strip())
|
||||
if not isinstance(self.commitment_key, str):
|
||||
raise ValueError("OperatorEvidence.commitment_key must be a string")
|
||||
object.__setattr__(self, "input_keys", tuple(str(k) for k in self.input_keys))
|
||||
object.__setattr__(self, "check_keys", tuple(str(k) for k in self.check_keys))
|
||||
object.__setattr__(self, "payload", _freeze_json_value(dict(self.payload)))
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"domain": self.domain,
|
||||
"operator": self.operator,
|
||||
"outcome": self.outcome,
|
||||
"reason": self.reason,
|
||||
"input_keys": list(self.input_keys),
|
||||
"check_keys": list(self.check_keys),
|
||||
"commitment_key": self.commitment_key,
|
||||
"structural_signature": self.structural_signature,
|
||||
"payload": _json_value(self.payload),
|
||||
}
|
||||
|
||||
def canonical_json(self) -> str:
|
||||
return _canonical_json(self.as_dict())
|
||||
|
||||
@property
|
||||
def evidence_hash(self) -> str:
|
||||
return hashlib.sha256(self.canonical_json().encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class EvidenceBundle:
|
||||
"""Ordered collection of operator evidence with stable serialization."""
|
||||
|
||||
evidences: tuple[OperatorEvidence, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
object.__setattr__(self, "evidences", tuple(self.evidences))
|
||||
if not all(isinstance(ev, OperatorEvidence) for ev in self.evidences):
|
||||
raise ValueError("EvidenceBundle.evidences must contain OperatorEvidence")
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {"evidences": [ev.as_dict() for ev in self.evidences]}
|
||||
|
||||
def canonical_json(self) -> str:
|
||||
return _canonical_json(self.as_dict())
|
||||
|
||||
@property
|
||||
def evidence_hash(self) -> str:
|
||||
return hashlib.sha256(self.canonical_json().encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Tier2Verdict:
|
||||
"""Result of a domain-neutral convergent self-verification check."""
|
||||
|
||||
verified: bool
|
||||
reason: str
|
||||
commitment_key: str = ""
|
||||
evidence_hash: str = ""
|
||||
structural_signatures: tuple[str, ...] = ()
|
||||
reader_lineages: tuple[str, ...] = ()
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.reason not in TIER2_REASONS:
|
||||
raise ValueError(f"unknown Tier2Verdict.reason: {self.reason!r}")
|
||||
object.__setattr__(
|
||||
self,
|
||||
"structural_signatures",
|
||||
tuple(str(s) for s in self.structural_signatures),
|
||||
)
|
||||
object.__setattr__(
|
||||
self,
|
||||
"reader_lineages",
|
||||
tuple(str(s) for s in self.reader_lineages),
|
||||
)
|
||||
|
||||
def as_dict(self) -> dict[str, Any]:
|
||||
return {
|
||||
"verified": self.verified,
|
||||
"reason": self.reason,
|
||||
"commitment_key": self.commitment_key,
|
||||
"evidence_hash": self.evidence_hash,
|
||||
"structural_signatures": list(self.structural_signatures),
|
||||
"reader_lineages": list(self.reader_lineages),
|
||||
}
|
||||
|
||||
|
||||
def verify_tier2_agreement(
|
||||
evidences: tuple[OperatorEvidence, ...] | list[OperatorEvidence],
|
||||
) -> Tier2Verdict:
|
||||
"""Require two **independently-read** structures converging on one commitment.
|
||||
|
||||
Independence is keyed on ``reader_lineage`` — the decoding PATHWAY — not on the
|
||||
cosmetic ``structural_signature``. The same reader cannot self-verify by emitting
|
||||
two differently-labeled signatures for one case: that is refused with
|
||||
``SAME_READER_LINEAGE``. The static guarantee that distinct lineages are
|
||||
import-disjoint (so distinct lineage ⟹ no shared decoding pathway) is enforced
|
||||
by the architectural reader-disjointness invariant; this gate enforces the
|
||||
runtime half (distinct producing pathways + distinct structure + one commitment).
|
||||
"""
|
||||
bundle = EvidenceBundle(tuple(evidences))
|
||||
if len(bundle.evidences) < 2:
|
||||
return Tier2Verdict(False, INSUFFICIENT_EVIDENCE)
|
||||
|
||||
if any(not ev.commitment_key for ev in bundle.evidences):
|
||||
return Tier2Verdict(False, MISSING_COMMITMENT, evidence_hash=bundle.evidence_hash)
|
||||
|
||||
lineages = tuple(ev.reader_lineage for ev in bundle.evidences)
|
||||
signatures = tuple(ev.structural_signature for ev in bundle.evidences)
|
||||
|
||||
# Mechanistic independence firewall: at least two DISTINCT decoding pathways.
|
||||
if len(set(lineages)) < 2:
|
||||
return Tier2Verdict(
|
||||
False,
|
||||
SAME_READER_LINEAGE,
|
||||
evidence_hash=bundle.evidence_hash,
|
||||
structural_signatures=tuple(sorted(set(signatures))),
|
||||
reader_lineages=tuple(sorted(set(lineages))),
|
||||
)
|
||||
|
||||
if len(set(signatures)) < 2:
|
||||
return Tier2Verdict(
|
||||
False,
|
||||
DUPLICATE_STRUCTURAL_SIGNATURE,
|
||||
evidence_hash=bundle.evidence_hash,
|
||||
structural_signatures=tuple(sorted(set(signatures))),
|
||||
reader_lineages=tuple(sorted(set(lineages))),
|
||||
)
|
||||
|
||||
commitments = Counter(ev.commitment_key for ev in bundle.evidences)
|
||||
shared = [key for key, count in commitments.items() if count >= 2]
|
||||
if len(shared) != 1 or len(commitments) != 1:
|
||||
return Tier2Verdict(
|
||||
False,
|
||||
COMMITMENT_DISAGREEMENT,
|
||||
evidence_hash=bundle.evidence_hash,
|
||||
structural_signatures=tuple(sorted(set(signatures))),
|
||||
reader_lineages=tuple(sorted(set(lineages))),
|
||||
)
|
||||
|
||||
return Tier2Verdict(
|
||||
True,
|
||||
TIER2_VERIFIED,
|
||||
commitment_key=shared[0],
|
||||
evidence_hash=bundle.evidence_hash,
|
||||
structural_signatures=tuple(sorted(set(signatures))),
|
||||
reader_lineages=tuple(sorted(set(lineages))),
|
||||
)
|
||||
|
|
@ -26,7 +26,6 @@ from __future__ import annotations
|
|||
|
||||
from core.response_governance.policy import (
|
||||
ACTIVE_STATES,
|
||||
APPROXIMATE_POLICY,
|
||||
RECONCILE_STATES,
|
||||
RESERVED_STATES,
|
||||
STRICT_POLICY,
|
||||
|
|
@ -38,7 +37,6 @@ from core.response_governance.policy import (
|
|||
|
||||
__all__ = [
|
||||
"ACTIVE_STATES",
|
||||
"APPROXIMATE_POLICY",
|
||||
"RECONCILE_STATES",
|
||||
"RESERVED_STATES",
|
||||
"STRICT_POLICY",
|
||||
|
|
|
|||
|
|
@ -121,19 +121,7 @@ STRICT_POLICY: ReachPolicy = ReachPolicy(
|
|||
license_ratio=0.0,
|
||||
)
|
||||
|
||||
# Step E (ADR-0206 §5) — the first widening rung. APPROXIMATE keeps the SAME admissible
|
||||
# set as STRICT ({DECODED}): a fully-grounded surface commits verbatim, but anything less
|
||||
# grounded (a converse GUESS is ``UNVERIFIED_POSSIBLE``) is surfaced by ``shape_surface``
|
||||
# as a DISCLOSED ``[approximate]`` alternative. So a licensed estimate is never committed
|
||||
# silently — admitting its state here would defeat the disclosure the rung exists for.
|
||||
APPROXIMATE_POLICY: ReachPolicy = ReachPolicy(
|
||||
level=ReachLevel.APPROXIMATE,
|
||||
admissible_states=_STRICT_ADMISSIBLE,
|
||||
rationale="license-gated widening (ADR-0206 §5 / Step E — SERVE earned on a committed ClassTally)",
|
||||
license_ratio=1.0,
|
||||
)
|
||||
|
||||
# Disclosure prefixes for the widening levels. Real
|
||||
# Disclosure prefixes for the (currently unreachable) widening levels. Real
|
||||
# code so the higher-level branch of shape_surface is genuinely
|
||||
# policy-sensitive, exercised by the live-wiring test.
|
||||
_DISCLOSURE_PREFIX: dict[ReachLevel, str] = {
|
||||
|
|
@ -143,25 +131,6 @@ _DISCLOSURE_PREFIX: dict[ReachLevel, str] = {
|
|||
}
|
||||
|
||||
|
||||
def _serve_licensed(license_decision: object | None) -> bool:
|
||||
"""True iff ``license_decision`` is a GENUINE, licensed ``Action.SERVE`` decision.
|
||||
|
||||
Strict by type on purpose: only a real :class:`~core.reliability_gate.LicenseDecision`
|
||||
that the gate marked ``licensed`` for ``Action.SERVE`` widens. A ``None``, a bare
|
||||
object, or a forged dict (``{"licensed": True}``) is NOT a ratified license and stays
|
||||
STRICT — the wrong=0 guard is that widening rests on the gate's verdict over a
|
||||
committed ledger, never on a caller's say-so.
|
||||
"""
|
||||
from core.reliability_gate import Action
|
||||
from core.reliability_gate.gate import LicenseDecision
|
||||
|
||||
return (
|
||||
isinstance(license_decision, LicenseDecision)
|
||||
and license_decision.action is Action.SERVE
|
||||
and license_decision.licensed
|
||||
)
|
||||
|
||||
|
||||
def govern_response(
|
||||
*,
|
||||
epistemic_state: EpistemicState | None = None,
|
||||
|
|
@ -170,19 +139,17 @@ def govern_response(
|
|||
) -> ReachPolicy:
|
||||
"""Decide the reach policy for a response.
|
||||
|
||||
Step E (ADR-0206 §5) — the first license-gated widening. Returns
|
||||
:data:`APPROXIMATE_POLICY` IFF ``license_decision`` is a genuine licensed
|
||||
``Action.SERVE`` decision (a predicate-class that earned SERVE on the committed
|
||||
reliability ledger); otherwise :data:`STRICT_POLICY`. Every current serving call
|
||||
site passes no ``license_decision`` → STRICT → byte-identical to the pre-E path.
|
||||
SCAFFOLD (ADR-0206 §3): returns :data:`STRICT_POLICY` unconditionally.
|
||||
The inputs are accepted now so the call site is the final shape, but
|
||||
the stakes-weighing and license-gated widening they will drive is
|
||||
*designed, not built*. Every response is therefore governed at STRICT
|
||||
— commit-only-when-grounded, else the existing refuse/disclose path —
|
||||
which is exactly the pre-bridge behavior.
|
||||
|
||||
The STRICT default remains the load-bearing line for ``wrong == 0``: nothing
|
||||
widens without a ratified license, and even APPROXIMATE only ever surfaces a
|
||||
DISCLOSED estimate (``shape_surface`` adds ``[approximate]``), never a silent
|
||||
commit. ``stakes``-weighing (SITUATE) stays designed-not-built (ADR-0206 §1).
|
||||
This single return value is the load-bearing line for ``wrong == 0``:
|
||||
the live-wiring tests prove that the *only* thing keeping the response
|
||||
path strict is this STRICT return, not the absence of a consumer.
|
||||
"""
|
||||
if _serve_licensed(license_decision):
|
||||
return APPROXIMATE_POLICY
|
||||
return STRICT_POLICY
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,236 +0,0 @@
|
|||
# Roadmap: the autonomous-improvement engine (path to AGI-candidacy)
|
||||
|
||||
**Date:** 2026-06-05 · **Status:** ROADMAP (the hyperfocus design plan) · **Telos:** [[project-core-is-one-continuous-life]] — `listen → comprehend → recall → think → articulate → learn → replay`, as one continuous, ever-improving life.
|
||||
|
||||
## The bar (what we are actually building toward)
|
||||
|
||||
A **serious AGI candidate**: an engine that is at least as **book-smart as an LLM**,
|
||||
**keeps up with the world**, and **forever gets smarter autonomously under human
|
||||
supervision** — by **taking in inputs (literature, told facts, world inputs,
|
||||
experiences), comprehending them, and *realizing* them as structured grounded
|
||||
memory it can recall** — rather than the LLM move of bulk-absorbing the whole
|
||||
corpus indiscriminately and compressing it into weights.
|
||||
|
||||
> **Clarification — "intake" vs "ingestion".** The engine absolutely *ingests*:
|
||||
> it must take in literature, knowledge, and experience to learn anything, and
|
||||
> intake is first-class (Phase 3). The distinction from an LLM is *what is kept
|
||||
> and how*: CORE keeps **selectively-realized, comprehended, provenance- and
|
||||
> status-tagged knowledge + remembered experiences** (the vault + corpus *are* its
|
||||
> memory, with exact recall), and never realizes unverified content as true — vs
|
||||
> indiscriminately swallowing everything (junk included) and lossily averaging it
|
||||
> into weights. "We don't need the world's *data*" means we get smart from
|
||||
> comprehended structure + high-signal told facts, *not* that we don't take input.
|
||||
|
||||
What the bar is **not**: mass indiscriminate absorption, statistical
|
||||
pattern-matching, or confident guessing (the LLM trick — and a different identity
|
||||
config could even make our own models behave that way; it is not the point).
|
||||
Determinism / `wrong=0` / auditability are the **necessary baseline**, not the
|
||||
achievement.
|
||||
|
||||
## The strategic key: grounded honesty is the efficient-learning mechanism
|
||||
|
||||
An LLM must swallow the entire internet — junk, lies, contradictions — and average
|
||||
its way past them. CORE **only realizes what it can ground** (told-and-evidenced,
|
||||
comprehended, or reasoned), so it never absorbs garbage and never has to unlearn
|
||||
it. The junk-filter *is* the learning advantage: we don't need the world's data,
|
||||
we need the world's **true, comprehended structure**, accumulated forever. So
|
||||
grounding is load-bearing for the *capability*, not just for trust. (`wrong=0` is
|
||||
the high-stakes gear of this honesty — see the epistemic foundation below — not a
|
||||
universal law that forces the engine to refuse everything it can't prove.)
|
||||
|
||||
## The loop (the autonomous-improvement engine)
|
||||
|
||||
```
|
||||
open question / discovery / TOLD fact
|
||||
→ COMPREHEND (arbitrary input → structured meaning)
|
||||
→ REALIZE (make it real: integrate into the held self with an epistemic status)
|
||||
→ REASON / GROUND / RECALL
|
||||
→ RESPOND in the honest gear:
|
||||
ASSERT (verified / realized)
|
||||
ESTIMATE (evidence-grounded likelihood — ONLY where taught it is apt)
|
||||
REFUSE (no grounding, or stakes forbid an estimate)
|
||||
→ PROPOSE (idle_tick, proposal-only)
|
||||
→ HITL ratify (reviewed, supervised)
|
||||
→ ACCUMULATE into the one continuous life
|
||||
→ MEASURABLY more capable → repeat, autonomously
|
||||
```
|
||||
|
||||
When this loop demonstrably climbs a **general capability curve** over time, on its
|
||||
own, under supervision — that is the AGI candidate.
|
||||
|
||||
## The epistemic foundation (honesty designed, estimation learned)
|
||||
|
||||
This corrects an earlier over-emphasis on `wrong=0` as a universal law. The right
|
||||
frame has three commitments:
|
||||
|
||||
1. **Honesty is designed in; confabulation is impossible by construction.** The
|
||||
engine's native stance is grounded: ASSERT what it has realized, REFUSE what it
|
||||
has not. It has **no organ that fabricates** — no statistical token-soup, no
|
||||
manufactured confidence. That cannot emerge by accident; it could only be
|
||||
*deliberately built*, and we will not build it. This is the absolute floor, not
|
||||
a policy defended turn by turn.
|
||||
|
||||
2. **Estimation is a LEARNED, ratified competence — never a designed-in default.**
|
||||
There is a season for a calibrated assessment (*"on the evidence, most likely
|
||||
X"*). The engine may acquire the competence to give one — **but only through
|
||||
human ratification and deliberate guidance**, realized as knowledge like any
|
||||
other. We do NOT design a "guess mode" with a risk knob; the engine never
|
||||
self-authorizes a guess. `wrong=0` is therefore **demoted to one gear**
|
||||
(high-stakes / verified assertion), not deleted.
|
||||
|
||||
3. **All confidence is evidence-grounded, so even uncertainty is honest.** A CORE
|
||||
"likelihood" attaches to the deterministic confidence primitives we already
|
||||
have — the calibrated-learning ledger, one-sided Wilson floors, cue-precision
|
||||
reliability counts, the `EpistemicStatus` taxonomy. It means *"seen N times, M
|
||||
coherent → confidence M/N with a hard lower bound"* — a counted fact about the
|
||||
engine's own realized experience, not a vibe. This is the exact inverse of an
|
||||
LLM (softmax over absorbed text) and is **why it can offer graded answers
|
||||
without ever confabulating**.
|
||||
|
||||
**The measured invariant is calibration + grounding, not "never wrong":** every
|
||||
confidence the engine states must trace to counted evidence, and it offers graded
|
||||
answers only where it was taught that is appropriate. Being *honestly uncertain*
|
||||
is success; being *dishonestly confident* is the only failure — and the substrate
|
||||
makes the latter impossible without intentional design.
|
||||
|
||||
> **"Being told" is first-class.** Most knowledge arrives as *told facts* ("these
|
||||
> are facts"); the engine realizes them and earns the why/how (coherence /
|
||||
> evidence) over time. Determination does NOT mean proof-from-first-principles —
|
||||
> intake → realize-with-evidence → build coherence is a primary growth path. The
|
||||
> seed packs are the told bootstrap; the engine comprehends the new by relating it
|
||||
> to what it has already realized, and grows.
|
||||
|
||||
## What is already built (compose, don't rebuild)
|
||||
|
||||
- **The continuous self** — Shape B+ resume ([[milestone-shape-b-plus-persistence]]),
|
||||
L11 identity continuity + the idle learning mechanism
|
||||
([[milestone-l11-identity-and-continuous-learning]]). The life that accumulates.
|
||||
- **Verified reasoning substrate** — sound+complete propositional entailment
|
||||
(`deductive_logic`, wrong=0, independent gold), `generate/proof_chain/`
|
||||
(proof-tree builder/entail/rules), `generate/binding_graph/` (the universal-
|
||||
structure interlingua DAG).
|
||||
- **Determination pieces** — `core/reliability_gate/` (gold-tether, ledger,
|
||||
calibrated propose) determines correctness in the math lane; the wrong=0
|
||||
self-verification gate in `generate/derivation/verify.py`.
|
||||
- **Comprehension front door** — `generate/derivation/` (extract → clauses →
|
||||
compose), the question layer.
|
||||
- **Measurement raw material** — independent-gold lanes (`deductive_logic`,
|
||||
`relational_metric`, `dimensional`, `cold_start_grounding`, `symbolic_logic`)
|
||||
+ the Perplexity-surveyed adoptables (ProntoQA, ProofWriter-CWA, CLUTRR, FOLIO —
|
||||
all with independently-checkable gold + a refuse class).
|
||||
|
||||
## The bottleneck that gates everything
|
||||
|
||||
The flywheel can only **propose what is already determined** — `idle_tick` refuses
|
||||
`undetermined` candidates. The engine can *learn a fact it is handed*; it cannot yet
|
||||
autonomously **figure one out**. The missing organ is **general determination**:
|
||||
comprehend an open question, reason/ground it to a *verified* conclusion (or refuse),
|
||||
and feed *that* to the flywheel. The math lane does a narrow version; nothing does
|
||||
it generally. **Closing comprehend → determine → learn, measured on a general
|
||||
capability curve, is the load-bearing arc.**
|
||||
|
||||
## Phased roadmap (entry → exit gates; wrong=0 is structural throughout)
|
||||
|
||||
| Phase | Build | Exit gate / measurement |
|
||||
|-------|-------|-------------------------|
|
||||
| **0 — the yardstick** | A **general capability index**: compose the independent-gold reasoning lanes (+ adopt ProntoQA/ProofWriter-CWA/CLUTRR/FOLIO) into one report with two axes — **correctness (wrong=0, never fabricate)** and **coverage (determined vs honestly-refused)**. Frozen-gated. | A single reproducible capability number the engine must climb; `wrong=0` enforced; a baseline measured. *You cannot improve what you cannot measure.* |
|
||||
| **1 — the determination organ** | A general `determine(question) → {determined: conclusion ∣ refused}` path composing comprehension (`derivation`/`binding_graph`) + reasoning (`proof_chain`/`deductive`) + the reliability gate. Commits ONLY verified conclusions; refuses the rest. | On the Phase-0 yardstick: coverage rises with **wrong still 0**; every committed conclusion is independently checkable. |
|
||||
| **2 — close the autonomous loop** | Wire `determine` → the `idle_tick` flywheel: take open questions, determine what it can (wrong=0), propose, HITL-ratify, accumulate. | The capability index **rises across loop iterations**, autonomously, under supervision — falsifiably (a frozen replay shows monotonic, junk-free improvement). |
|
||||
| **3 — autonomous curriculum** | The engine drives its own agenda: identifies its determination frontier (what it can't yet determine), proposes what to learn next, under HITL guidance. | "Forever getting smarter autonomously under supervision" — the engine's self-chosen curriculum measurably advances the index. |
|
||||
| **4 — breadth / generality** | Expand comprehension + reasoning across domains so the index is genuinely GENERAL (book-smart breadth), acquired via the loop — intake → comprehend → realize, not bulk indiscriminate absorption. | The capability index spans enough domains to credibly claim general book-smarts — every gain via comprehension+determination over realized knowledge, none via indiscriminate corpus absorption or per-domain matchers. |
|
||||
|
||||
## Invariants (non-negotiable across all phases)
|
||||
|
||||
- **`wrong=0` is structural** — the engine commits only verified conclusions; it
|
||||
refuses rather than fabricates. This is the learning filter, not just a gate.
|
||||
- **Reviewed learning** — ratification stays HITL (`teaching/review`); the loop
|
||||
*proposes*, the human *ratifies*. Autonomy is supervised, not unmoored.
|
||||
- **Determinism / replay** — every capability gain is reproducible; improvement is
|
||||
a replayable curve, not a vibe.
|
||||
- **Identity continuity** — the improving engine stays one continuous self
|
||||
(L11); a smarter CORE is the *same* CORE, grown.
|
||||
|
||||
## Execution order — logical necessity × technical priority
|
||||
|
||||
Not arbitrary phases: each step is gated by what it *logically depends on*, then
|
||||
ordered within that by leverage × risk. The dependency DAG:
|
||||
|
||||
```
|
||||
MEASURE ───────────────────────────────────┐ (gates every "improved" claim)
|
||||
│ │
|
||||
COMPREHEND ──► REALIZE ──► DETERMINE/RESPOND ─┼─► AUTONOMOUS LOOP ──► CURRICULUM
|
||||
(NL → universal (hold (assert / refuse │ (idle_tick + BREADTH
|
||||
interlingua) with over realized) │ climbs the curve,
|
||||
status) │ │ autonomously)
|
||||
└─ LEARNED ESTIMATION ◄── needs MEASURE(calibration)
|
||||
(ratified, evidence-grounded)
|
||||
```
|
||||
|
||||
**Step 1 — MEASURE: the cross-domain capability yardstick.** *Logical necessity:*
|
||||
nothing can be called "more capable" without it; it is prior to all improvement.
|
||||
*Technical priority:* HIGH leverage (north-star instrument + the anti-self-
|
||||
deception guard — a per-domain hack moves one lane and breadth stays flat,
|
||||
exposing it), MODERATE effort (compose the existing independent-gold lanes +
|
||||
adopt ProntoQA/ProofWriter-CWA/CLUTRR/FOLIO). Measures **assert-correctness +
|
||||
grounding + coverage + calibration** under a configurable risk budget. **Build
|
||||
first.**
|
||||
|
||||
**Step 2 — COMPREHEND: NL/prose → the universal interlingua.** *Logical
|
||||
necessity:* it is the wall (GSM8K refuses 92% on comprehension coverage, not
|
||||
arithmetic; prose/exams are ~0); every downstream step needs structure to operate
|
||||
on. *Technical priority:* HIGHEST leverage (unlocks all breadth) AND HIGHEST
|
||||
risk/effort (open-ended; the overfit trap lives here). The discipline: it must
|
||||
emit the **general** binding-graph / universal-structure, never per-domain parses
|
||||
— and the Step-1 yardstick is what proves it generalized rather than gamed. **The
|
||||
make-or-break.**
|
||||
|
||||
**Step 3 — REALIZE: integrate comprehended/told structure into the held self**
|
||||
with an epistemic status (`EpistemicStatus`), persisted via Shape B+. *Necessity:*
|
||||
needs COMPREHEND. *Priority:* MODERATE effort (vault/corpus/persistence exist),
|
||||
HIGH leverage — this is what makes knowledge *accumulate* (told facts become
|
||||
realized; the engine grows). Intake ("being told") lands here.
|
||||
|
||||
**Step 4 — DETERMINE / RESPOND: reason over realized structure → the honest
|
||||
gear** (assert verified / refuse ungrounded). *Necessity:* needs COMPREHEND +
|
||||
REALIZE. *Priority:* MODERATE effort (compose `proof_chain` / `deductive` /
|
||||
binding-graph entail onto comprehension output), HIGH leverage — coverage rises
|
||||
on the yardstick with grounding intact. **No estimation yet — assert/refuse only.**
|
||||
|
||||
**Step 5 — AUTONOMOUS LOOP: wire comprehend→realize→determine→idle_tick→ratify→
|
||||
accumulate.** *Necessity:* needs Steps 1–4. *Priority:* MODERATE effort (idle_tick
|
||||
exists), HIGH leverage — this is the step that makes "forever improving" real and
|
||||
falsifiable (the yardstick curve climbs autonomously, under supervision).
|
||||
|
||||
**Step 6 — LEARNED ESTIMATION: the calibrated likelihood competence.**
|
||||
*Necessity:* needs DETERMINE (the honest floor) + MEASURE-calibration + the
|
||||
teaching loop. *Priority:* MODERATE effort, MODERATE leverage — deliberately
|
||||
LATE: only after the assert/refuse floor and the calibration measurement are
|
||||
solid do we teach (HITL-ratified) when/how to offer evidence-grounded likelihoods.
|
||||
Never a designed-in default.
|
||||
|
||||
**Step 7 — AUTONOMOUS CURRICULUM + BREADTH.** *Necessity:* needs the loop. The
|
||||
engine drives its own determination frontier under supervision; breadth expands
|
||||
across domains via the loop (intake → comprehend → realize), never via
|
||||
indiscriminate corpus absorption or per-domain matchers.
|
||||
|
||||
**Critical-path summary:** `MEASURE → COMPREHEND → REALIZE → DETERMINE → LOOP`,
|
||||
with ESTIMATION grafted after DETERMINE+MEASURE and CURRICULUM after LOOP. The
|
||||
single highest-risk step is **COMPREHEND** (Step 2); the single highest-necessity
|
||||
"do-first" is **MEASURE** (Step 1), because it is the only thing that keeps every
|
||||
later step honest.
|
||||
|
||||
## Cross-cutting invariants (hold at every step)
|
||||
|
||||
The 8 foundation commitments above, plus the standing CLAUDE.md invariants:
|
||||
`versor_condition < 1e-6` (math floor), no forbidden-site repair/normalization,
|
||||
reviewed learning stays HITL, exact CGA recall (no approximation), deterministic
|
||||
replay. Every step is TDD + mutation-verified-to-bite + curated-smoke + CI-lane-SHA
|
||||
gated, the way the L10→L11 spine was built.
|
||||
|
||||
## Honest scope boundary
|
||||
|
||||
This is the multi-phase arc to AGI-candidacy, not one PR. AGI is the destination;
|
||||
this roadmap is the **critical path** and the **measurement** that makes progress
|
||||
toward it real and falsifiable. **Phase 0 (the yardstick) is the first build** —
|
||||
without a general capability curve, "getting smarter" is unfalsifiable, and we'd
|
||||
be doing exactly the unmeasured hand-waving the LLM world runs on.
|
||||
|
|
@ -1,153 +0,0 @@
|
|||
# Step D — CLOSE: idle deductive consolidation of soundly-derived facts
|
||||
|
||||
**Date:** 2026-06-06
|
||||
**Branch:** `feat/loop-learns-determined`
|
||||
**Sequence:** A INSTRUMENT (#598/#599) → B WIRE (#600/#601) → C DEEPEN (#602) → **D CLOSE** → E ESTIMATION (last)
|
||||
**Telos:** [[project-core-is-one-continuous-life]] — the loop *learns from determined facts*, not just the partial discovery chains it emits today.
|
||||
|
||||
## The scoped seam (read, not assumed)
|
||||
|
||||
The autonomous loop today is `extract_discovery_candidates → _pending_candidates →
|
||||
idle_tick(contemplate → propose_from_candidate) → ProposalLog (pending, HITL)`. Two
|
||||
findings from reading the source fix what D actually is:
|
||||
|
||||
1. **Today's loop is a within-corpus generalizer.** `extract_discovery_candidates`
|
||||
emits an *intentionally partial* chain `{subject, intent, connective:None,
|
||||
object:None}`; `contemplate` completes it *only* by enumerating objects the
|
||||
reviewed corpus already used, and `check_eligibility` **Gate 2** requires a
|
||||
`source="corpus"` evidence pointer. So it can only re-propose a shape the corpus
|
||||
already contains. It cannot learn a fact that arose in lived conversation.
|
||||
|
||||
2. **The HITL ProposalLog never reaches serving.** D's falsification — "the capability
|
||||
index climbs across loop iterations" — therefore *cannot* be met by emitting
|
||||
proposals (the index is a static breadth×accuracy eval, hard-gated to 0 on any
|
||||
wrong; proposals are HITL-gated and never feed serving). For the index to climb,
|
||||
a *determined fact must feed back into what the engine can answer.*
|
||||
|
||||
So the inevitable D is **session-memory consolidation**, not proposal emission:
|
||||
|
||||
> When idle, the engine consolidates each soundly-derived determination
|
||||
> (`member∘subset`, `subset∘subset` — the transitive is-a reasoning C built) back
|
||||
> into the realized-knowledge vault as a new realized record, so the *next*
|
||||
> `determine()` reaches it directly and can chain one hop further. The directly
|
||||
> answerable set climbs monotonically across idle ticks to the deductive-closure
|
||||
> fixed point.
|
||||
|
||||
Emitting determined facts as HITL teaching-chain proposals is a *distinct* capability
|
||||
that collides with Gate 2 (the corpus-evidence floor, a teaching-safety / wrong=0
|
||||
surface). It is **deliberately deferred** to its own evidence-floor-touching PR rather
|
||||
than bolted onto D — the falsifiable core stays clean.
|
||||
|
||||
## Mechanism (semi-naive deductive closure, one layer per tick)
|
||||
|
||||
`idle_tick` (gated by `config.consolidate_determinations`, default OFF) runs
|
||||
`consolidate_once(ctx)`:
|
||||
|
||||
1. Recall realized `member` + `subset` facts.
|
||||
2. Compute every **one-hop** extension under the two SOUND is-a rules:
|
||||
- `member(s,b) ∧ subset(b,t) → member(s,t)`
|
||||
- `subset(a,b) ∧ subset(b,t) → subset(a,t)`
|
||||
- **`member ∘ member` is never an edge** — instance-of is not transitive
|
||||
("Socrates is a man" + "man is a species" ⊬ "Socrates is a species"). The
|
||||
reader's member/subset split is exactly what makes the included rules sound.
|
||||
3. For each derived edge not already realized, **verify with the sound+complete
|
||||
proof_chain ROBDD** (reusing C's single verifier `_verify_subsumption` — no
|
||||
duplicate proof logic), then write it via `realize_derived`.
|
||||
|
||||
Each tick adds exactly one hop-layer; the closure saturates after `diameter` rounds;
|
||||
re-running at the fixed point is a no-op. This is textbook semi-naive evaluation, and
|
||||
it is precisely what produces the monotone "climbs across iterations" signal.
|
||||
|
||||
## Invariants held (the wrong=0 / honesty boundary)
|
||||
|
||||
- **Soundness (wrong=0).** Every consolidated fact is the conclusion of a sound rule
|
||||
over realized premises, *confirmed by the sound+complete decider* — defense in depth
|
||||
beyond the one-hop rule. The `member∘member` fallacy is structurally unreachable (no
|
||||
member→member edge). A consolidated `member(s,t)` can only be extended by a *subset*
|
||||
edge, never a member edge, so the fallacy stays unreachable across iterations.
|
||||
- **Epistemic honesty.** A fact derived from SPECULATIVE premises is SPECULATIVE,
|
||||
`basis="as_told"`. The soundness of the *inference* never upgrades the *standing* of
|
||||
the premises. **COHERENT is never minted.**
|
||||
- **Session memory, not reviewed learning.** Consolidation is the immediate session
|
||||
tier (allowed), an extension of the `generate.realize` path — **not** corpus
|
||||
mutation and **not** coupled to proposals. The teaching/review HITL path is
|
||||
untouched; no parallel learning path is created.
|
||||
- **Sanctioned write path.** Writes reuse `_realize_structured` (the INV-21 allowed
|
||||
vault writer). No new normalization; no closure/repair — `algebra/versor.py` keeps
|
||||
closure. The derived record reuses the subject's `probe_ingest` placement, identical
|
||||
to a told fact about the same subject.
|
||||
- **Idempotency / determinism.** Dedup on the span-free `structure_key` (identical to
|
||||
a told fact's, so a later told duplicate collapses). Deterministic order; no clock,
|
||||
no LLM, no metric call. Bounded by the existing `_SUBSUMPTION_SUBSET_FACT_BUDGET`.
|
||||
|
||||
## Provenance — the replayable proof obligation (Fork 2)
|
||||
|
||||
`Determined` already carries `grounds: tuple[RealizedRecord, ...]` — the premise
|
||||
records that entailed it. The consolidated record records, as derived-provenance, the
|
||||
premise `structure_key`s + the `rule` + the `verdict` (always `entailed`). This makes
|
||||
the soundness claim **meaningfully fail** (per the Schema-Defined Proof Obligations
|
||||
rule): a replay re-fetches the premises by `structure_key` and re-runs the rule +
|
||||
proof_chain — if a consolidated fact were ever unsound, re-verification fails loudly.
|
||||
|
||||
## Falsification — `evals/determination_closure/`
|
||||
|
||||
A frozen replay seeds a deep is-a chain (`member(x,c0)`, `subset(c0,c1)…subset(cₙ₋₁,cₙ)`)
|
||||
and runs idle consolidation ticks. Asserts:
|
||||
|
||||
- **Monotone climb:** the directly-realized `member(x, ·)` closure grows by exactly one
|
||||
per tick (each layer), strictly increasing until the fixed point.
|
||||
- **Convergence:** after `n` ticks the closure is saturated; a further tick is a
|
||||
**no-op** (`at_fixed_point=True`, 0 consolidated).
|
||||
- **wrong=0:** no `member(x, y)` is ever consolidated for `y` outside the chain's
|
||||
reachable set (no fabricated fact); the `member∘member` canary derives nothing.
|
||||
- **Provenance replay:** every consolidated record's recorded premises re-verify as
|
||||
`ENTAILED`.
|
||||
|
||||
## Adversarial verification (5 independent skeptics, refute-the-claim)
|
||||
|
||||
A panel re-read the shipped source under five distinct lenses, each tasked to *refute*.
|
||||
- **wrong=0 / soundness** — held. `member ∘ member` is structurally unreachable (member
|
||||
facts are only ever extended by subset edges); every write is proof_chain-`ENTAILED`;
|
||||
no cross-subject leakage. **Acted on its note:** `_verify_subsumption` now *refuses* a
|
||||
mislabeled/wrong-arity path (a member fact smuggled into `subset_path`), converting
|
||||
soundness-by-caller-discipline into soundness-by-construction now that consolidation is
|
||||
a second caller.
|
||||
- **epistemic honesty** — held. `realize_derived` writes SPECULATIVE unconditionally;
|
||||
`_basis` returns `as_told` for SPECULATIVE grounds; `promote_eligible_entries`
|
||||
(SPECULATIVE→COHERENT) requires energy metadata derived facts never carry and is
|
||||
disjoint from `idle_tick`.
|
||||
- **teaching-safety boundary** — held. Single write path (`ctx.vault.store`, `tier=
|
||||
"session"`); zero `teaching/`/proposal/pack/identity coupling; the two `idle_tick`
|
||||
passes are orthogonal (the proposal pass's vault probe filters to COHERENT, excluding
|
||||
these SPECULATIVE facts).
|
||||
- **determinism / replay / persistence** — held. Sorted write order; derived
|
||||
`structure_key` identical to a told fact's; `Derivation` round-trips through the
|
||||
snapshot; the lane's `reverify_derived` meaningfully fails on a non-entailing record.
|
||||
- **normalization / closure invariant** — flagged `high`, assessed a **misattribution**.
|
||||
The cited `vault.store → reproject → null_project` is pre-existing (`aadaf116`,
|
||||
2026-05-13, ~3 weeks before D), triggered *identically* by the already-merged
|
||||
`realize_comprehension` path, operates on vault **content** null-vectors (which
|
||||
CLAUDE.md says to preserve as null vectors — sanctioned), and is a different object
|
||||
from the runtime field `versor_condition(F)<1e-6` invariant the claim referenced. The
|
||||
lane's high `vault_reproject_interval` is the established determine/realize test idiom,
|
||||
not a D-specific sidestep. D adds zero normalization code and reuses the INV-21-allowed
|
||||
writer; the claim ("consolidation *adds* no forbidden normalization") stands.
|
||||
|
||||
## Out of scope (follow-ups)
|
||||
|
||||
- **Promotion firewall (defensive).** If a future change ever called
|
||||
`promote_eligible_entries` inside `idle_tick` or attached energy metadata to derived
|
||||
facts, SPECULATIVE derived facts could promote to COHERENT and bootstrap standing. No
|
||||
live path (the separation is architectural); a structural `derived ⇒ never-promote`
|
||||
marker would harden it.
|
||||
- **Runtime vs. lane proof obligation.** The provenance replay (`reverify_derived`) runs
|
||||
in the eval lane, not per-write at runtime — matching the repo's "wrong=0 proven by
|
||||
lanes, not runtime asserts" pattern (consolidation already verifies each write *before*
|
||||
writing). Noted, not a gap.
|
||||
|
||||
|
||||
- Determined-fact → HITL teaching proposal (touches Gate 2 / evidence floor — its own PR).
|
||||
- Incremental frontier (semi-naive with a delta set) instead of recompute-and-dedup per
|
||||
tick — an O() optimization once the closure substrate is proven.
|
||||
- Order/containment transitivity (`less_than`, `before_event`, `inside_of`) — the C-2
|
||||
predicate widening; consolidation generalizes to them once C-2 lands.
|
||||
|
|
@ -1,83 +0,0 @@
|
|||
# Step E — ESTIMATION: calibrated, disclosed estimation via the ADR-0206 reach bridge
|
||||
|
||||
**Date:** 2026-06-06
|
||||
**Branch:** `feat/learned-estimation`
|
||||
**Sequence:** A INSTRUMENT → B WIRE → C DEEPEN → D CLOSE → **E ESTIMATION** (last)
|
||||
**Executes:** ADR-0206 §5 (cognition-path widening) — the `LICENSE` node ("built — not yet called from serving") finally called from serving.
|
||||
|
||||
## What E is (and is not)
|
||||
|
||||
E lets the engine **commit a disclosed estimate** for a class it has *measured itself
|
||||
reliable on*, instead of always refusing past proof. It is **not** generic guessing,
|
||||
not a probabilistic model, and it does **not** touch the sealed GSM8K serving metric
|
||||
(that is a separate, riskier ADR-0206 §5 PR — `select_self_verified`).
|
||||
|
||||
The whole step is one wire: `govern_response` consults `reliability_gate.license_for(…,
|
||||
Action.SERVE)`; a licensed class reaches `ReachLevel.APPROXIMATE`; `shape_surface`
|
||||
**discloses** the estimate with an `[approximate]` prefix.
|
||||
|
||||
## Why it is wrong=0-safe by construction
|
||||
|
||||
`shape_surface(APPROXIMATE)` never commits an estimate silently — it prefixes
|
||||
`[approximate]`. So an estimate that is wrong is a **disclosed**-wrong, categorically
|
||||
different from the silent/asserted wrong the `wrong=0` invariant forbids. The reliability
|
||||
gate (θ_SERVE=0.99 on a committed `ClassTally`) governs *when* the disclosed estimate is
|
||||
even offered. Two independent guards: disclosure (honesty) + license (calibration).
|
||||
|
||||
## The estimator — a blind converse-guesser
|
||||
|
||||
Given a realized `p(a, b)` and a query `p(b, a)`, the estimator commits the **converse**
|
||||
as a candidate. It is **blind** — it never reads the pack's symmetry metadata. Therefore:
|
||||
|
||||
- on a **symmetric** predicate (`sibling_of`, `spouse_of`, `equal_to`, `distinct_from`,
|
||||
`adjacent_to`, `overlaps_event` — `graph.edge.symmetric`) the converse is **true**;
|
||||
- on a **directed** predicate (`parent_of`, `less_than`, `before_event`, … —
|
||||
`graph.edge.directed`) the converse is **false**.
|
||||
|
||||
The engine does not *know* which is which. It **measures** its converse-guess precision
|
||||
per predicate-class over a gold lane and earns a SERVE license only where the measured
|
||||
floor clears θ. That is calibrated learning (ADR-0175): reliability is commitment
|
||||
precision, earned by volume. The symmetry metadata is the **gold** (the `GoldTether`'s
|
||||
truth), never a serving-time shortcut.
|
||||
|
||||
## The committed ledger (real, sealed, HITL-ratified)
|
||||
|
||||
- `evals/determination_estimation/gold.py` deterministically generates the gold cases:
|
||||
657+ symmetric converse cases for the licensed class (gold=true) and a directed class
|
||||
(gold=false). 657 is the Wilson volume floor: a perfect record clears θ_SERVE=0.99 at
|
||||
`n/(n+z²) ≥ 0.99`, `z=2.576` ⇒ `n ≥ 657`. Reliability is earned by volume, never a
|
||||
lucky streak — so the gold lane is sized to that bar, not the bar to the lane.
|
||||
- `core.learning_arena.run_practice` folds a `DomainSolver` (the converse-guesser) +
|
||||
`GoldTether` (symmetry-as-truth) over the cases → `dict[str, ClassTally]`.
|
||||
- The resulting ledger (per-class committed counts) is frozen as a **ratified artifact**
|
||||
(`evals/determination_estimation/ratified_ledger.json`) with an expected-hash. Committing
|
||||
it via a reviewed PR **is** the HITL ratification. Ceilings stay at safe defaults
|
||||
(θ_SERVE=0.99) — no override, so the engine never raises its own bar (invariant #4).
|
||||
|
||||
## The wire (E-3, the delicate part)
|
||||
|
||||
In `chat/runtime.py`, gated by a new config flag (default OFF): when a turn is a
|
||||
**converse query** (`p(b,a)` asked, `p(a,b)` realized, `p(b,a)` not directly
|
||||
determinable) whose predicate-class holds a committed `license_for(SERVE).licensed`,
|
||||
pass that real `LicenseDecision` into `govern_response` → it emits `APPROXIMATE` →
|
||||
`shape_surface` discloses the converse estimate as `[approximate] …`. Every other turn,
|
||||
and every unlicensed class, stays `STRICT` (byte-identical, wrong=0 untouched). Never a
|
||||
designed-in default: absent a cleared committed tally, refuse.
|
||||
|
||||
## Falsification — `evals/determination_estimation`
|
||||
|
||||
A frozen replay asserts:
|
||||
- **Discriminating gate:** the symmetric class is SERVE-licensed; the directed class is
|
||||
not (its converse-guess reliability is ~0 over the committed lane).
|
||||
- **Disclosed estimate:** a licensed converse query yields an `[approximate]`-prefixed
|
||||
surface; an unlicensed one stays STRICT-refuse, byte-identical to pre-E.
|
||||
- **No silent estimate:** every reach > STRICT carries the disclosure prefix.
|
||||
- **wrong=0 (silent):** zero silently-committed wrong answers — every estimate is disclosed.
|
||||
- **Volume floor:** below 657 committed the symmetric class is NOT licensed (the bar binds).
|
||||
- **Determinism:** the ledger + verdicts reproduce byte-identically (frozen expected hash).
|
||||
|
||||
## Out of scope (separate PRs)
|
||||
- The **math-serving seam** (`select_self_verified`) — ADR-0206 §5, touches the sealed metric.
|
||||
- **SITUATE** (stakes/gravity) and the live **FEED-BACK** loop (serving outcome → ledger) —
|
||||
ADR-0206 §1 "designed, not built". E uses an offline, sealed, ratified ledger.
|
||||
- `EXTRAPOLATE` / `CREATIVE` reach levels (need `VERIFIED` / novelty capabilities).
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue