7cb826a548
The final AGI-spine step (A INSTRUMENT → B WIRE → C DEEPEN → D CLOSE → E ESTIMATION).
The engine may now SERVE a DISCLOSED estimate for a query it would otherwise refuse —
but only for a predicate-class that has measured itself reliable, and never as fact.
This executes the ADR-0206 §5 cognition-path widening: the bridge's LICENSE node
(reliability_gate.license_for), previously "built — not yet called from serving", is now
called. govern_response returns APPROXIMATE iff a genuine licensed Action.SERVE
LicenseDecision is passed (STRICT for every other input — so every existing serving call
site is byte-identical); shape_surface DISCLOSES the estimate as "[approximate] …".
Mechanism:
- generate/determine/estimate.py — a BLIND converse-guesser: told p(a,b), asked p(b,a),
it commits the converse. It never reads the pack's symmetry metadata; whether the guess
is right is MEASURED, not assumed.
- evals/determination_estimation/ — the gold lane: run_practice (sealed, ADR-0199) folds
the converse-guesser over symmetric (sibling_of) vs directed (parent_of) cases, scored
against the pack's graph.edge.symmetric truth (gold independent of the solver). The gate
DISCRIMINATES: sibling_of earns SERVE (660 correct → Wilson floor 0.990046 ≥ θ_SERVE),
parent_of does not (660 wrong → 0.0). The license is earned by VOLUME — 657 perfect
commits is the exact θ_SERVE=0.99 threshold (656 is below).
- generate/determine/data/estimation_ledger.json — the ratified committed ledger,
hash-verified on load (a hand-edited ledger raises RatifiedLedgerError); it IS the
deterministic sealed-practice output (a GSM8K-style --check test pins this).
- chat/runtime.py — when a converse query is refused and the class holds a SERVE license,
the disclosed estimate is surfaced through the bridge (gated by config.estimation_enabled,
default OFF; only meaningful with accrue_realized_knowledge).
Invariants:
- wrong=0 by construction — an estimate is ALWAYS disclosed ([approximate]), never a silent
commit (UNVERIFIED_POSSIBLE is never in APPROXIMATE's admissible set), and only a genuine
ratified license widens (a forged {"licensed":True} dict / a PROPOSE license / an
unlicensed SERVE all stay STRICT). Defense-in-depth: type-gate ∧ admissible-set ∧
hardcoded disclosed state.
- never self-authored — ceilings stay at safe defaults (θ_SERVE=0.99); the engine cannot
raise its own bar. The ledger is sealed practice, hash-verified.
- session/serving only — no corpus/pack/identity/proposal/vault mutation; the HITL teaching
path is untouched. Deterministic; no clock/random.
- byte-identical for every non-E turn (the 2643 govern_response call passes no license).
Out of scope (separate ADR-0206 §5 PRs): the math-serving seam (select_self_verified,
touches the sealed metric), SITUATE (stakes), and the live FEED-BACK loop.
Verified green: smoke (90), architectural invariants (56), response_governance (321,
incl. the new license-gated widening test), the determination-estimation lane (12), and
the B/D/determine regression net. Four-lens adversarial review (disclosure/wrong=0,
calibration integrity, byte-identity, boundary/determinism): all held. Design:
docs/analysis/E-estimation-design-2026-06-06.md.
87 lines
3.2 KiB
Python
87 lines
3.2 KiB
Python
"""E gold-lane runner — committed ledger + license verdicts for the converse-guess.
|
|
|
|
Folds ``run_practice`` (the sealed ADR-0199 engine) over the gold lane and reads the
|
|
resulting per-predicate ``ClassTally`` through the reliability gate. The report is the
|
|
falsifiable artifact: it shows the gate DISCRIMINATING — the symmetric class earns the
|
|
SERVE license, the directed class does not — and carries the committed counts the
|
|
ratified ledger artifact (E-2) freezes.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Any
|
|
|
|
from core.learning_arena.engine import run_practice
|
|
from core.reliability_gate import Action, Ceilings, ClassTally, license_for
|
|
from evals.determination_estimation.gold import (
|
|
LICENSED_PREDICATE,
|
|
REFUSED_PREDICATE,
|
|
ConverseSolver,
|
|
SymmetryGoldTether,
|
|
all_gold_problems,
|
|
generate_gold_problems,
|
|
)
|
|
from generate.determine.estimate import converse_class_name
|
|
|
|
|
|
def build_ledger() -> dict[str, ClassTally]:
|
|
"""Run sealed practice over the gold lane → the committed per-class ledger."""
|
|
report = run_practice(all_gold_problems(), ConverseSolver(), SymmetryGoldTether())
|
|
return dict(report.ledger)
|
|
|
|
|
|
def _tally_dict(tally: ClassTally) -> dict[str, Any]:
|
|
return {
|
|
"class_name": tally.class_name,
|
|
"correct": tally.correct,
|
|
"wrong": tally.wrong,
|
|
"refused": tally.refused,
|
|
"committed": tally.committed,
|
|
"reliability": tally.reliability,
|
|
"coverage": tally.coverage,
|
|
}
|
|
|
|
|
|
def run(ceilings: Ceilings | None = None) -> dict[str, Any]:
|
|
"""Build the ledger and report the SERVE/PROPOSE license verdict per class."""
|
|
ceilings = ceilings if ceilings is not None else Ceilings.default()
|
|
ledger = build_ledger()
|
|
licensed_cls = converse_class_name(LICENSED_PREDICATE)
|
|
refused_cls = converse_class_name(REFUSED_PREDICATE)
|
|
|
|
classes: dict[str, Any] = {}
|
|
for cls, tally in sorted(ledger.items()):
|
|
serve = license_for(tally, Action.SERVE, ceilings)
|
|
propose = license_for(tally, Action.PROPOSE, ceilings)
|
|
classes[cls] = {
|
|
"tally": _tally_dict(tally),
|
|
"serve_licensed": serve.licensed,
|
|
"serve_ratio": serve.ratio,
|
|
"propose_licensed": propose.licensed,
|
|
}
|
|
|
|
licensed_serves = classes.get(licensed_cls, {}).get("serve_licensed", False)
|
|
refused_serves = classes.get(refused_cls, {}).get("serve_licensed", True)
|
|
# The whole point: the gate discriminates — symmetric earns SERVE, directed does not.
|
|
discriminates = bool(licensed_serves) and not bool(refused_serves)
|
|
|
|
return {
|
|
"lane": "determination-estimation",
|
|
"licensed_class": licensed_cls,
|
|
"refused_class": refused_cls,
|
|
"classes": classes,
|
|
"gate_discriminates": discriminates,
|
|
}
|
|
|
|
|
|
def reliability_at(predicate: str, n: int) -> float:
|
|
"""The committed reliability of ``predicate``'s converse-guess over ``n`` cases —
|
|
used to prove the SERVE volume floor binds (below 657, a symmetric class is unlicensed)."""
|
|
report = run_practice(
|
|
generate_gold_problems(predicate, n), ConverseSolver(), SymmetryGoldTether()
|
|
)
|
|
tally = report.ledger[converse_class_name(predicate)]
|
|
return tally.reliability
|
|
|
|
|
|
__all__ = ["build_ledger", "reliability_at", "run"]
|