Commit graph

726 commits

Author SHA1 Message Date
Shay
1fe56e9b6f feat(workbench): calibration + serving-metrics readers — the gold-tether loop, visible (Wave M B1)
First Wave M / Phase B piece (GATING): read-only backend that makes the
calibrated-learning / serving-discipline loop inspectable — 'the engine
earns the right to guess', ADR-0175.

The workbench computes NONE of these numbers:
- GET /calibration/classes — per-class gold-tether view from the persisted
  practice arena ledger (evals/gsm8k_math/practice/v1/report.json per_class).
  Each class's reliability_floor is the engine's own one-sided Wilson
  conservative_floor (via ClassTally.reliability); PROPOSE (θ=0.85) / SERVE
  (θ=0.99) license verdicts come from core.reliability_gate.license_for.
  Failures-first ordering. A test proves the reader's floor equals a direct
  conservative_floor() call — no re-implementation.
- GET /serving/metrics — the live correct/refused/wrong counts read unchanged
  from the committed train_sample + holdout_dev report.json (currently
  4/46/0 and 5/495/0 — wrong=0). Never re-runs a lane.

Honest current state: the committed practice ledger's three classes
(additive/divisive/multiplicative) are all below N_MIN=10, so none has
earned a license yet — the reader shows exactly that, no fake green light.

- workbench/calibration.py: pure readers; imports core.reliability_gate;
  EvidenceUnavailableError -> 501 (fail-closed) when the artifact is absent.
- schemas + TS mirrors (CalibrationClass, ServingMetrics); both snapshots
  regenerated (deterministic); both drift gates pass.
- trust boundary: read-only over committed artifacts + engine-owned
  derivation; no execution, no mutation, no license ever changed.

Verified: 30 Python tests (incl. the no-reimplementation proof + fail-closed),
390 vitest, both schema drift gates, snapshots deterministic.
2026-06-13 00:38:16 -07:00
Shay
fd0d7f1f6a finish wave r3 ui and trace integrity 2026-06-12 23:25:24 -07:00
Shay
80e02ce7de feat(workbench): sealed single-turn replay backend — GET /replay/{turn_id} (Wave R3)
Replaces the W-026 501 stub. Re-executes a journaled prompt in a sealed
fresh runtime (ChatRuntime(no_load_state=True): no checkpoint load, no
checkpoint write, no proposal lineage) and compares the envelope
leaf-by-leaf against the recorded TurnJournalEntry.

Design correction vs the scoping doc (amended in-doc): journaled turns
each ran in a fresh ChatRuntime(), never one continuous session, so
genesis-PREFIX replay would manufacture spurious divergence; shipped
basis is sealed_fresh_runtime_single_turn (O(1)). origin_state:
"unrecorded" — the journal does not record whether the original turn's
runtime loaded a checkpoint, so divergence is reported as nondeterminism
OR origin-state influence, never disambiguated.

- workbench/replay.py: pure comparison + injected executor; every
  TurnJournalEntry field classified critical/informational exactly once
  (exhaustiveness enforced by test)
- api.py: route wiring under _CHAT_TURN_LOCK; runtime failure -> 500
  runtime_unavailable, no comparison may be fabricated
- schemas: additive TurnReplayComparison/TurnReplayDivergence (W-026
  artifact-keyed pair retires with the frontend Replay Moment PR)
- tests: 10 obligations incl. tamper-prompt, tamper-leaf precision,
  no-execution-no-comparison, no-trace (journal + engine_state bytes),
  wall-clock tolerance, sealed-construction proof
- snapshot regenerated; NOT_YET_MIRRORED debt entries for the two new
  classes (mirrors land with the frontend PR)
- api-contract-v1.md § Replay rewritten for the turn-keyed shape
2026-06-12 17:15:39 -07:00
Shay
f177efc837 feat(workbench): add R2 backend read substrate 2026-06-12 12:49:41 -07:00
Shay
6edf2d3ac1 Merge pull request #702 from AssetOverflow/feat/wb-journal
feat(workbench): turn evidence journal + trace API
2026-06-12 07:03:50 -07:00
Shay
0d203d5b55 feat(workbench): turn evidence journal + trace API
Append-only JSONL journal records the exact ChatTurnResult envelope returned by /chat/turn with stable turn_id, trace_hash, all three surfaces, verdicts, and deterministic journal_digest.

GET /trace/turns and GET /trace/{turn_id} serve journal evidence for the Trace route frontend. Read model only; no teaching, pack, or journal-owned engine_state mutation.
2026-06-12 06:44:25 -07:00
Shay
b2ad7c3ce9 feat(demo): add deductive entailment authority demo (Demo E)
Local deterministic demo exposing the formal entailment substrate as its
own public authority: the proposer's verdict/confidence/proof are data
only; CORE recomputes propositional entailment through the pinned ROBDD
engine (generate.proof_chain.entail.evaluate_entailment_with_trace),
independently cross-checks with the deductive lane's code-disjoint
truth-table oracle (evals.deductive_logic.oracle.oracle_entailment), and
serves entailed/refuted/unknown only when both procedures agree —
refusing on inconsistent premises (no vacuous entailment), out-of-regime
input (by design, before grammar), a distinct-atom regime bound (honors
the brute-force oracle's small-atom contract), and engine/oracle
disagreement (test-only fault injection, defensive refusal).

Seven committed scenarios with byte-pinned expected artifacts; 38 tests
covering recursive schema closure, output-smuggling-before-evaluation,
proposer-garbage byte-invariance in both directions, oracle module
disjointness, trace-hash folding of trace + oracle verdict, runner
hardening, and INV-21/24/29 discipline of the demo files.
2026-06-12 00:28:46 -07:00
Shay
c6d0e2a920
Merge pull request #697 from AssetOverflow/feat/demo-authority-strength
fix(demo): harden #690 epistemic truth-state — sealed corpus + entailment-decided inference
2026-06-11 23:27:05 -07:00
Shay
4c64328884 fix(demo): harden epistemic truth-state authority — sealed corpus + entailment-decided inference
Two-pass hardening of the #690 demo so no epistemic state is proposer-mintable:

Pass 1 (sealed corpus, from the reconciliation pass):
- evidence items are references only (evidence_id + content_sha256) resolved
  against a committed content-addressed corpus (evidence_corpus.json)
- support derived from committed subject/predicate match; independence from
  distinct provenance_root values; proposer support/independence labels are
  rejected by the closed schema
- evidence-strength taxonomy (substrate-capability vs interface-contract) in
  PUBLIC-DEMO-ROADMAP / DEMO-PACKAGING-CHECKLIST; position-paper refinements

Pass 2 (entailment-decided inference, closes a live false-status hazard):
- the merged inferred leg required only that premise IDs resolve — a claim
  with no support citing one unrelated record as a premise was assigned
  'inferred' (empirically confirmed). inferred is now decided by
  generate.proof_chain.entail (sound+complete ROBDD) over resolved corpus
  premises, cross-checked against the independent truth-table oracle
  (evals.deductive_logic.oracle, no shared code); proof keys + oracle verdict
  in the trace; disagreement => defensive refusal; refutation => contradicted
- committed adversarial scenario unrelated-premise-still-undetermined proves
  the attack now lands on undetermined
- corpus seal (corpus_sha256) pinned into every evaluated trace; scope
  refusals no longer consult the corpus; corpus cache hands out copies;
  claim subject/predicate constrained to atom-compatible snake_case
- .gitignore: private packet dir, agent worktrees, tooling artifacts

Validation: 30 demo tests; 96 across all three demo suites; 7/7 scenarios
double-run byte-identical; 61 architectural invariants green.
2026-06-11 21:25:08 -07:00
Shay
8c40ee3f20 feat(demo): ADR-0218 PR D — proof-carrying coherence promotion demo
Local deterministic public evidence for the ratified PCCP capability:
proposer submits claim + proof candidate + status/confidence garbage; CORE
ignores proposer authority, fresh-reads curator-certified store state,
recomputes the proof under the pinned engine, and promotes/refuses only
through VaultStore.apply_certified_promotion. The trace proves the decision.

- demos/proof_carrying_promotion/{schema.json,authority.py,run_demo.py}:
  closed recursive schema (output fields unsuppliable at the root), the REAL
  decider (teaching.proof_promotion.certify_promotion) and the REAL owner —
  no demo-local promotion reimplementation. The local arena is rebuilt via
  VaultStore.from_dict over dict-literal metadata: demos/ is scanned by
  INV-21/INV-24/INV-29, and the demo adds no vault writer, no recall site,
  and no status-transition site (proven in tests with the invariants' own
  detectors).
- 8 fixtures + pinned expected artifacts: entailed-promotes,
  proposer-status-ignored, non-coherent-premise, uncertified-reading,
  tampered-certificate (fails byte replay), stale-premise-status (same
  honest certificate digest as the promoting run — live state is what
  differs), non-sequitur, invalid-state-smuggling (rejected before
  evaluation, authority_evaluated=false, null digest).
- Hardened runner per the #688/#690 pattern: default out/ dir, unsafe
  output roots refused, --write-expected explicit, double-run byte-identity
  enforced per fixture, scenario pass/fail summary.
- tests/test_proof_carrying_promotion_demo.py (26): expected-artifact
  conformance, tamper detection, proposer-garbage decision-invariance (incl.
  on the promotable setup), refused-cases-mutate-nothing, REFUTED-never-
  demotes, trace-hash folds certificate digest, forbidden-import/call scan
  (incl. uuid/random/time), INV-21/INV-29 allowlists pinned unchanged, no
  named-company/strategy terms.

No runtime turn integration; promotion remains SPECULATIVE->COHERENT only;
README carries the full honesty ledger.
2026-06-11 17:33:08 -07:00
Shay
e473f5b55f feat(teaching): ADR-0218 P3 — proof-carrying coherence promotion
The ratified capability, through the existing mutation owner only:

- teaching/proof_promotion.py — pure decider. Fresh-reads every premise AND
  the claim from the store (claim_entry_index; forms never proposer-supplied),
  strict status compares (no parse-defaulting), reading_certified must be the
  boolean True, engine certification via the pinned deductive engine,
  proposer_payload accepted and provably never read (del before use; poisoned-
  mapping test), certificate digest emitted for D4 trace-hash folding. Zero
  mutation; zero vault.store calls; zero status writes (proven with the
  INV-21/INV-29 detectors themselves).
- vault/store.py::apply_certified_promotion — the single transition site.
  Independently re-verifies: byte-for-byte replay under DEDUCTIVE_ENGINE_PIN,
  promotion_positive, live claim form/reading/status cross-checks, live
  premise form/reading/status cross-checks (staleness refuses). Fabricated or
  tampered certificates flip nothing; authority is live store state plus
  recomputation, never the artifact.
- generate/proof_chain/engine_pin.py — DEDUCTIVE_ENGINE_PIN mirrors the
  deductive_logic_v1 lane SHA; sync pinned by AST test against
  scripts/verify_lane_shas.py.
- Consistency fix found in lookback: both promotion sites now stamp
  epistemic_state alongside epistemic_status (the ADR-0148 site left the
  stored state tag stale; recall recomputed it, but the stored key lied).

Obligations: all six strict-xfail markers retired and live (O1/O2
strengthened to prove the transition through the vault owner); the
module-existence pin deleted per its own docstring; the two remaining
honesty pins kept. INV-21 allowlist unchanged; INV-29 allowlist unchanged
({vault/store.py}); INV-29's 29c visibility floor raised 2 -> 3 to cover the
new site.

No runtime turn path calls promotion; the deterministic demo is PR D.

Validation: new suite 49 passed; obligations 8 passed (0 xfail remaining);
invariants 66 passed; certificate 27 passed; 0148 6 passed; epistemic 20
passed; smoke files green; lane SHAs verified.
2026-06-11 17:02:45 -07:00
Shay
bdc4940e1e feat(proof_chain): ADR-0218 PR B — PromotionCertificate + pure replay verifier
Pure evidence substrate for proof-carrying coherence promotion. No
promotion, no vault import, no status transition, no teaching-path
change; ADR-0218 stays Proposed.

- generate/proof_chain/certificate.py: frozen PromotionCertificate
  (version/claim_form/premise ids+forms+statuses/entailment_trace/
  engine_pin/decision/reason, canonical_json with sort_keys+compact
  separators); build_certificate runs evaluate_entailment_with_trace
  over entry_id-sorted premises; verify_certificate rebuilds from the
  embedded forms and accepts iff byte-identical — tampered forms/claim/
  trace/decision/reason/ordering/version all fail replay, structural
  breakage fails closed as malformed_certificate.
- promotion_positive is necessary-not-sufficient: entailed over a
  non-empty all-coherent recorded premise set; REFUTED/UNKNOWN/REFUSED
  verify as themselves but are never positive; zero-premise tautology
  certificates fail-closed non-positive in v1.
- engine_pin is recorded provenance; pin tampering is caught only via
  caller-supplied expected_engine_pin (P3 must pass the lane SHA) —
  limit pinned by dedicated tests.
- entail.py: docstring-only fix removing the incorrect ADR-0206
  attribution (numbering collision); now cites ADR-0218. Zero behavior
  change. generate_claims.py:74 still carries the label (serving-frozen
  surface, flagged for follow-up).
- No xfail markers retire: every PR-A obligation binds to the P3
  promoter, which must not exist before ratification. The certificate-
  shaped halves of O1/O7 are proven for real in
  tests/test_proof_chain_certificate.py (27 tests). ADR phasing bullet
  and obligations docstring record the reconciliation.

Validation: certificate 27 passed; obligations 3 passed + 6 xfailed;
architectural invariants 61 passed (INV-21/INV-29 green, allowlists
unchanged); entail/builder/rules 61 passed; smoke files 95 passed.
2026-06-11 16:21:32 -07:00
Shay
ed37060223 test(inv-29): widen transition-write detector to mutation-call shapes
INV-29 is the load-bearing contribution of #693, so its detector must match
its claim. The assignment-only scan missed common write indirections; the
widened _status_transition_writes (renamed from _status_assignment_targets)
now also catches:

- m.update({"epistemic_status": ...}) / update(epistemic_status=...) /
  update(**{...}) / update(dict(epistemic_status=...))
- m.setdefault("epistemic_status", ...)
- bound and unbound __setitem__ with the literal key
- setattr(entry, "epistemic_status", ...)
- writes hidden in tuple-unpacking assignment targets

29b now proves all 13 shapes are flagged (per-shape, so one regression names
itself); new 29b' proves reads, other keys, and dict-literal construction do
NOT flag (false positives would train reflexive allowlisting). Honest scope
note added: variable keys / exec / C-level aliasing are statically
undetectable and belong to review.

Pre-scan confirmed zero production call-shape writes exist, so the widened
detector flags nothing new; ALLOWED_STATUS_TRANSITION_SITES stays
{"vault/store.py"}. No runtime change; ADR-0218 still Proposed.

Validation:
- tests/test_architectural_invariants.py: 61 passed
- tests/test_proof_carrying_promotion_obligations.py: 3 passed, 6 xfailed
2026-06-11 15:50:05 -07:00
Shay
bf091035df docs(adr-0218): propose proof-carrying coherence promotion; ship INV-29 + strict-xfail obligations
PR A of the stack specified by docs/issues/proof-carrying-coherence-promotion.md.
No runtime change; no promotion behavior; ADR-0218 is Proposed, not ratified.

- ADR-0218 (Proposed): rulings the issue Section 9 demands — extend the
  existing mutation owner (policy decides in teaching/proof_promotion.py at
  P3, only vault/store.py mutates), reading stays curator-certified in the
  first cut, exact fail-closed admissibility predicate, certificate folds
  into trace_hash. Records two substrate corrections: proof_chain Phase 2
  (2.1-2.4) is already implemented, and ADR-0148 already automates an
  energy-arm promotion.
- INV-29 (passing): epistemic-status transition sites are allowlisted.
  ADR-0148's in-place promotion proved INV-21's AST scan cannot see
  status transitions (they are not store() calls) — the boundary the
  issue calls highest-risk did not actually cover the riskiest shape.
  Allowlist pinned to {vault/store.py} BEFORE any promoter exists;
  detector mutation-proven non-vacuous (29b/29c).
- Obligations O1-O5, O7 as strict xfail (executable spec; ImportError today;
  strict=True forces the P3 PR to retire markers and pass for real) + three
  honesty pins that flip red if promotion is wired without review.

Validation (worktree off main@187b008b, python -m pytest):
- tests/test_proof_carrying_promotion_obligations.py: 3 passed, 6 xfailed
- tests/test_architectural_invariants.py: 60 passed (56 prior + 4 INV-29)
- smoke file set: 94 passed
- proof_chain/entail/binding-graph/ADR-0148 set: 84 passed
- epistemic/teaching set: 40 passed
- scripts/verify_lane_shas.py: 8/9 pinned lanes match incl. deductive_logic_v1;
  public_demo missed on the known env wall-clock budget flake (47.6s > 30s),
  pre-existing and unrelated (diff is docs+tests only).
2026-06-11 15:39:08 -07:00
Shay
5306c9f944 fix(demo): keep normative_clearance unassessable for all non-invalid outputs
This demo assigns epistemic truth-state only and runs no normative/
safety/ethics clearance pass, so it has no basis to positively clear
a claim. Drop the verified->cleared mapping: verified now reports
normative_clearance=unassessable like every other non-invalid output.

Updates authority.py, regenerates the affected expected trace hashes,
tightens the test (verified asserts unassessable) and adds an invariant
test that no non-invalid output is ever cleared, and corrects the README
honesty ledger.
2026-06-11 12:55:45 -07:00
Shay
a68b7060bd feat(demo): add epistemic truth-state authority demo
A model-style proposer submits a claim, evidence bundle, and optional
bounded-inference block; CORE alone assigns the typed epistemic state
drawn from the canonical core.epistemic_state taxonomy (no parallel
enum) and emits a deterministic, replayable evidence artifact.

The proposer cannot set assigned_state, status, trace_hash,
authority_path, evidence_ledger, or normative_clearance: the closed
recursive schema rejects any root-level injection before state
evaluation runs, and proposer-supplied proposed_state/trace_hash are
recorded as ignored and never read.

Adds closed schema, six fixtures, pinned expected artifacts, a
double-run byte-identical runner with hardened output-dir policy, and a
22-test suite. No network, subprocess, eval/exec, model API, or side
effects beyond demo output.
2026-06-11 12:46:17 -07:00
Shay
77a8c79f94 fix(demo): harden tool-authority runner hygiene 2026-06-11 10:14:59 -07:00
Shay
1f41935f5f fix(demo): refuse normalized note path traversal 2026-06-11 08:18:00 -07:00
Shay
8fe63c4032 feat(demo): add Claude-to-CORE tool authority demo 2026-06-11 08:07:07 -07:00
Shay
57a0e38f0b feat(demo): add Claude-to-CORE hybrid verification demo
A narrow, auditable proof path for the hybrid-mode boundary: a
Claude/Fable-style System 1 proposer submits an MCP-shaped tool call
(core.semantic_derivation.verify); CORE re-derives via the ADR-0184
derivation lane, keeps sole acceptance authority, refuses or asks when
it cannot honestly answer, and emits a deterministic replay/provenance
trace for every decision.

Boundary properties (each pinned by a failing-if-removed test):
- the proposer has no commit path: additionalProperties:false schema,
  and the demo modules import nothing from generate.* (AST-scanned with
  a planted-bypass self-test) — the lane is consumed only through the
  ADR-0184 S4b audited trace facade;
- 'verified' requires pool commit AND clean commit-law/faithfulness
  audits AND a gold-audited envelope.json entry AND byte-match to the
  pinned derivation trace (measured: 118 of the off-serving pool's 231
  corpus commits disagree with lane gold, so a bare pool commit is
  never served — scenario s4 shows the refusal on a commit that even
  matches gold);
- the ask leg is the real Q1 stack (router -> limitation -> Q1-D
  producer -> carried handle -> gated serving), dark by default,
  tamper-fail-closed on the content hash;
- authority_path names only consulted authorities; responses are
  deterministic (double-run byte-identical) and pinned under expected/.

No serving/runtime change; no CLAIMS/metrics/telemetry/lane-pin
movement; no network or Anthropic API dependency (System 1 payloads are
clearly-labeled static fixtures).
2026-06-10 20:41:50 -07:00
Shay
ad75a2c79a test(derivation): add ADR-0184 semantic replay equivalence harness
ADR-0184 S4b — the replay/provenance equivalence boundary, the
precondition for any semantic-primary path. After #685 the in-repo
legacy path delegates to the semantic boundary, so old-vs-new is
vacuous; the durable reference is a pinned canonical trace artifact
(evals/gsm8k_math/equivalence/v1/, 937 problems) whose authority is
the #684/#685 cross-tree differentials (documented provenance chain).

- state/provenance.py: replay_is_faithful — the replay bridge's law as
  a checkable structural property (no verify/pool import; covered by
  the #685 authority scan).
- equivalence/trace.py: canonical traces (worlds, candidates, order,
  multiplicity, classifications, resolutions) + authority_violations
  re-deriving the pool commit law from trace content.
- scripts/verify_semantic_equivalence.py: check / --update gate
  mirroring verify_lane_shas.py.
- tests/test_adr_0184_s4b_replay_equivalence.py: full-corpus live-vs-
  pinned equality, corpus-wide authority + faithfulness, and 17
  single-mutation non-vacuity proofs.

Pure additions; verify.py/pool.py untouched; no serving, CLAIMS,
metrics, or lane pins moved.
2026-06-10 14:35:22 -07:00
Shay
a6e846ebdb
Merge pull request #685 from AssetOverflow/feat/adr-0184-s3-candidate-source-boundary
feat(derivation): define semantic-ledger candidate source boundary
2026-06-10 13:53:45 -07:00
Shay
951bcc0252 feat(derivation): define semantic-ledger candidate source boundary
ADR-0184 §7 S4: add generate/derivation/state/source.py — the single
boundary through which semantic-ledger worlds become pool candidates —
and swap pool.py's accumulation source to semantic_state_candidates().
accumulate.py keeps its public surfaces as thin compatibility wrappers
(ADR-0184 §10). Byte-identical over the 937-problem differential;
verify.py / pool.py remain the sole commit authority (structural
no-authority-import scan over state/, non-vacuous).

Also: docs/analysis design note for the S3–S5 boundary stack, and a
drive-by ruff F541 fix in r1_reconstruction.py (rf->r, zero behavior).
2026-06-10 13:17:53 -07:00
Shay
130344d373
Merge pull request #683 from AssetOverflow/feat/ask-carried-handle-seam
feat(ask): add carried-handle acquisition seam for served ASK
2026-06-10 12:54:38 -07:00
Shay
c0d6493219 fix(ask): canonicalize question/proposal path collision check
Compare resolved canonical Path values, not raw string spellings, when
detecting a question_path / proposal_path collision. A relative
proposal_path and an absolute question_path that name the same file now
fail closed with reason "path_collision" instead of resolving.

Adds a regression test for the absolute-vs-relative same-file collision
and keeps the existing exact-string collision test. The new test is
non-vacuous: it fails under the prior raw-string comparison.

Scope: ask_handle.py seam + its test only. No runtime.py, CLAIMS,
metrics, telemetry, schema, or lane-pin changes.
2026-06-10 12:44:46 -07:00
Shay
9dcb00248c feat(derivation): route accumulation through ADR-0184 S2 semantic ledger
ADR-0184 S1 (state/bind.py, state/change.py) landed the reusable referent and
change-cue helpers. S2 adds the first explicit state-transition substrate and
routes both accumulation readings through it:

- state/model.py   frozen SET/GAIN/LOSS transition model over one entity/unit
                   StateKey, with structural validation (closed op set; unit is a
                   str with "" = unitless, mirroring the extractor's Quantity.unit
                   contract; entity optional, mirroring leading_subject_token).
- state/ledger.py  build_accumulation_ledger() — the proven single-referent
                   gain/loss reading, expressed as a SemanticLedger.
- state/replay.py  replay_accumulation_ledger() — the only bridge to
                   GroundedDerivation; refuses any non-accumulation ledger shape
                   (non-SET start, cross-key mutation).

accumulate.py's _build_accumulation / _build_accumulation_anchor_skip now build a
ledger and replay it instead of constructing the arithmetic chain inline; the
public compose_accumulation()/accumulation_candidates() surfaces are unchanged.

Behavior-equivalent: a byte-for-byte differential over 937 real GSM8K problems
(accumulation_candidates, compose_accumulation, pooled_candidates, resolve_pooled)
shows 0 differences vs main. Semantic worlds never commit directly — replay emits
GroundedDerivation and verify.py / pool.py remain the sole authority. Sealed: no
serving/runtime/chat import; no CLAIMS or eval-lane-SHA movement.

Refs ADR-0184 section 7 (S2).
2026-06-10 12:00:23 -07:00
Shay
41664ce60d feat(ask): add carried-handle acquisition seam for served ASK
The missing-boundary doc (#682) established that chat/runtime.py has no
lawful in-turn provider for a QUESTION_NEEDED ContemplationResult: the
only producer is the off-serving contemplation pass, and scanning
teaching/questions/ would be new, unspecified acquisition logic. This
slice builds the recommended unblocking seam instead of the forbidden
runtime shortcut: carried-handle acquisition.

core/epistemic_disclosure/ask_handle.py:
- AskArtifactHandle — an explicit, caller-carried reference to one
  already-produced Q1-D DeliveredQuestion artifact. Provenance is the
  producer's own content address (sha256 of blocking_reason:slot_name:
  text, also the artifact filename stem) — the only identity field the
  Q1-D artifact carries today. No turn/case provenance exists yet; that
  boundary is documented, not invented.
- resolve_served_ask_handle — gate-first (zero filesystem access while
  ask_serving_enabled is dark), structural handle validation, single
  explicit-path artifact read (no glob/iterdir/scan), and content-hash
  re-derivation so a stale/replaced/tampered artifact fails closed.
  Emits a ResolvedAskCandidate duck-typed to what evaluate_served_ask
  reads (terminal/question_path/proposal_path) — no question prose is
  ever constructed or carried.
- acquire_served_ask_from_handle — composes resolution into the
  existing acquire_served_ask_candidate / evaluate_served_ask stack.
  All Q1-D artifact policy (status/review/served/answer-binding/text/
  slot/collision) stays delegated; the seam owns identity only.

Not changed: chat/runtime.py, chat(...) signature, ChatResponse,
TurnEvent, telemetry schema, Q1B_ASK_CARVE_OUT, proposal_allowed,
VERIFIED serving, CLAIMS/metrics/lane pins. The seam imports neither
generate.contemplation (pass_manager) nor core.epistemic_questions
render/delivery — enforced by AST tests.

Tests (tests/test_ask_handle.py, 18 cases): gate-dark no-I/O proof,
fallback preservation, valid resolution served end-to-end through the
existing stack, proposal-artifact / missing / malformed / invalid-Q1-D
/ path-collision / stale-hash / non-addressed-filename / structurally-
invalid-handle fail-closed paths, no-discovery-without-exact-handle,
and AST guards for scan primitives, forbidden imports, and prose.

Verification: focused ASK+governance suites 396 passed; architectural
invariants 56 passed; smoke-equivalent suite 90 passed; lane SHAs 8/9
content-match (the 9th is the documented local public_demo wall-clock
flake, 48.3s vs the 30s ADR-0099 budget — no content drift, no re-pin).
2026-06-10 10:53:54 -07:00
Shay
97ba7a3a17 feat(ask): add runtime-facing ASK helper module
Adds a small chat.ask_runtime helper that consumes the ASK acquisition seam without editing chat/runtime.py.

- Uses honest contemplation_result keyword forwarding
- Preserves fallback behavior when ASK serving is disabled
- Does not call pass_manager or render_question
- Does not change public chat/runtime schema
2026-06-10 03:34:02 -07:00
Shay
0e24f8cb0b test(ask): make runtime schema guard structural 2026-06-09 13:34:10 -07:00
Shay
c0b00587f6 revert(ask): remove unsafe runtime hook bypass
Backs out the direct main push that added _maybe_apply_served_ask using a string-concatenation workaround to evade an over-broad text guard.

Restores chat/runtime.py to the #678 merge state and removes tests/test_ask_runtime_hook.py so the next runtime hook can land through a reviewed PR with structural tests instead of guard bypasses.
2026-06-09 13:29:18 -07:00
Shay
2bafceea46 feat(ask): wire default-dark ASK candidate into runtime fallback path
- This is the smallest runtime consumption hook for the ASK acquisition seam.

- It is default-dark.

- It preserves normal runtime behavior when disabled.

- It does not change public `chat(...)` signature.

- It does not change `ChatResponse`, `TurnEvent`, or telemetry schema.

- It does not call/render question prose.

- It does not serve from `proposal_path`.

- It does not retire Q1B carve-out or flip `proposal_allowed`.

- It does not serve VERIFIED.

- It does not move CLAIMS or metrics.
2026-06-09 13:26:13 -07:00
Shay
14285b5f21 test(ask): cover served ASK acquisition seam 2026-06-09 13:00:11 -07:00
Shay
4308519ca9
feat(ask): add served ASK artifact adapter
Adds the Stage 2 served ASK artifact adapter and adapter-only tests.

- Validates Q1-D DeliveredQuestion artifacts before ASK eligibility
- Uses ask_serving_enabled as a default-dark gate
- Uses the disclosure bus disposition mapping
- Fails closed to fallback surface/disposition on invalid artifacts
- Does not wire chat/runtime, telemetry, TurnEvent, ChatResponse, CLAIMS, or metrics
- Does not retire Q1B_ASK_CARVE_OUT or flip proposal_allowed
2026-06-09 12:49:52 -07:00
Shay
9cff9755cd
feat(ask): split question artifact path from proposal path
Adds dedicated ContemplationResult.question_path for off-serving ASK artifacts and stops storing QUESTION_NEEDED question paths in proposal_path. Proposal artifacts remain under proposal_path. No chat/runtime wiring, served ASK surface, Q1B carve-out retirement, registry flip, or CLAIMS/metric movement.
2026-06-09 11:28:52 -07:00
Shay
b687f62cf0
feat(ask): integrate off-serving ASK delivery in pass manager
Adds off-serving pass_manager integration for ASK delivery through deliver_ask behind explicit exercise_ask. Preserves boundary-first ordering, writes question artifacts to the question sink via question_root, avoids double delivery, and keeps chat/runtime, served ASK, Q1B carve-out, registry, CLAIMS, and metrics untouched.
2026-06-09 10:50:36 -07:00
Shay
3cfaaf9153
feat(config): add explicit default-dark serving gate fields
Adds explicit default-false RuntimeConfig ask_serving_enabled and verified_serving_enabled fields while preserving default-dark helper behavior. Includes focused ASK/VERIFIED gate tests and cross-gate independence coverage. No pass_manager wiring, runtime wiring, verify.py wiring, served behavior, or CLAIMS/metric movement.
2026-06-09 10:24:28 -07:00
Shay
45835987b9
feat(verified): add default-dark VERIFIED serving gate helper
Adds a default-dark verified_serving_enabled helper with focused tests. The helper is unwired: no runtime integration, no verify.py wiring, no eval producer imports, no served VERIFIED behavior, and no CLAIMS/metric movement.
2026-06-09 10:06:36 -07:00
Shay
297349fd8d
feat(ask): add default-dark ASK serving gate helper
Adds a default-dark ask_serving_enabled helper with focused tests. The helper is unwired: no pass_manager integration, no runtime surface, no carve-out retirement, and no served question text.
2026-06-09 09:17:14 -07:00
Shay
65413c415f docs(epistemic): Q1-D — disambiguate question_only from proposal_only lane
Per review: 'proposal-only' described the Q1-D sink/artifact in two docstrings,
conflating two distinct lanes. A question is an intake request, not a capability
proposal; question_only is its own lane. The analogy to the proposal emitter is
structural (toothless, review-gated, never served), never semantic. Tighten the
sink/artifact docstrings and rename the sink test to *question_only* to match
_QUESTION_STATUS. No behavior change — docstrings + one test name only.
2026-06-08 19:18:47 -07:00
Shay
f88e03e53a feat(epistemic): Q1-D — off-serving ASK delivery (QUESTION_NEEDED tenant)
The fourth and final Q1 rung: route the Q1-C rendered question onto the
contemplation bus as the QUESTION_NEEDED tenant (sibling of PROPOSAL_EMITTED),
off-serving. Consumes render_question VERBATIM — Q1-D constructs no prose, so the
Q1-C grounded-rendering wrong=0 guard is never bypassed by a second surface.

- generate/contemplation/findings.py: add Terminal.QUESTION_NEEDED.
- core/epistemic_disclosure/limitation.py: ask_question's home terminal is now
  QUESTION_NEEDED, making terminal_for_action total (all six actions map onto
  shipped terminals — the consolidation proof is now complete, not 'five of six').
- core/epistemic_questions/delivery.py: DeliveredQuestion (proposal-only artifact,
  cannot wrap an unrenderable question / cannot be served / answer_binding reserved
  None — illegal states unrepresentable), DeliveryOutcome, deliver_ask, the
  teaching/questions sink emitter, and AnswerBinding (reserved for Q2).

Rulings honored:
- D1: off-serving only — no served surface, no ask_serving_enabled, no chat wiring.
- D2: an unrenderable ASK falls back to the family's standing disposition
  (PROPOSAL_EMITTED if it still proposes, else NO_PROGRESS) — never a contentless
  QUESTION_NEEDED. Enforced twice: in deliver_ask and structurally in __post_init__.
- D3: Q1B_ASK_CARVE_OUT untouched — registry keeps proposal_allowed=True; both
  signals coexist off-serving so no operational signal is lost.
- D4: sink under teaching/questions/ (sibling of teaching/proposals/).
- D5: strictly single-slot — multi-slot is unrenderable, takes the D2 fallback.

Tests: 16 delivery tests (happy path verbatim, both D2 fallback branches,
structural guards, idempotent sink, off-serving AST) + updated the two limitation
consolidation tests for the now-total terminal map. smoke 90/0, affected suites
128/0.
2026-06-08 19:10:54 -07:00
Shay
0401ec4b39
Merge pull request #666 from AssetOverflow/feat/q1-c-question-renderer
feat(epistemic): Q1-C — grounded-only question renderer (off-serving)
2026-06-08 18:52:06 -07:00
Shay
5f74351729 feat(epistemic): Q1-C strictly single-slot — refuse multi-slot ASK
The template asserts 'one more value is still needed' (exactly one). Rendering
the first of several missing slots and ignoring the rest makes that claim
subtly false. Refuse multi-slot assessments with multi_slot_not_supported
(slot=None) rather than render-first-drop-rest; one-question-per-slot fan-out
is a later rung. Replaces the now-contradicting first-of-many test with a
test that pins the honest refusal.
2026-06-08 18:42:40 -07:00
Shay
b186f07382 feat(epistemic): Q1-C grounded-only question renderer (off-serving)
Add core/epistemic_questions/ — the renderer rung of the ASK spine.
render_question(LimitationAssessment) -> EpistemicQuestion turns ASK typed
residue into a user-facing question or an explicit question_unrenderable verdict.
Render only: no bus delivery, no served disposition, no serving (that is Q1-D).

Policy (scoping §2 / session §1.5.7 wrong=0 invariant): a question may name a
problem entity only if it appears verbatim in grounded_terms. Two substrate facts
force a generic-structural policy today: grounded_terms is empty everywhere (the
readers do not yet emit verbatim evidence), and a *missing* slot's referent is by
definition absent from the trace. So the renderer:

- renders a generic question whose only variable content is a controlled English
  phrase for the slot's expected_unit_or_type, from a closed audited map;
- never surfaces slot_name or binding_target (snake_case), never prettifies an
  identifier into a natural-language entity, never names a problem entity;
- degrades unmapped types to question_unrenderable (renderability_gap) rather
  than dumping raw snake_case;
- single-slot (first slot only); non-ASK / zero-slot -> unrenderable;
- post-render fabrication guard re-checks every token is closed-vocab scaffold or
  grounded (defense in depth).

Off-serving: imports no generate.derivation / core.reliability_gate (AST-tested).
Tests: 9/9 green; smoke 90/0.
2026-06-08 17:20:49 -07:00
Shay
710743c38f
feat(verified): P1-C — split bound_slots_digest into separate obligation
Adds bound_slots_digest as a distinct VerificationProof obligation before any serving wiring. Keeps the VERIFIED lane off-serving while separating derivation, stated-slot binding, and back-substitution.
2026-06-08 17:17:31 -07:00
Shay
05b83a57cf
Merge pull request #664 from AssetOverflow/feat/q1-b-ask-residue
feat(epistemic-disclosure): Q1-B — typed ASK residue + missing_* reclassification (carve-out)
2026-06-08 16:57:30 -07:00
Shay
f90d84b4f7 feat(epistemic-disclosure): Q1-B — typed ASK residue + missing_* reclassification (carve-out)
Reclassify missing_total_count / missing_weighted_total in the limitation layer
from (capability_gap, emit_proposal) to (missing_information, ask_question) —
the user CAN state the total, so these are missing data, not capability gaps.
Add typed ASK residue (MissingSlot, grounded_terms) on LimitationAssessment;
defaults to empty so existing P0-1 callers continue to work.

Q1B_ASK_CARVE_OUT: the shipped failure_family REGISTRY still flags both as
proposal_allowed=True so existing consumers (proposal pile, contemplation
pass_manager) keep emitting proposal artifacts until Q1-C/Q1-D wires ASK
delivery. Flipping the registry flag before ASK is served would create a
no-proposal/no-question dead zone — a capability-regression window with no
compensating intake. The carve-out is named, enumerated, and explicit; once
ASK delivery is live the REGISTRY flag flips and the carve-out retires.

Wrong=0 invariant on residue (scoping §2): MissingSlot carries family-typed
structural identifiers only (snake_case); grounded_terms is verbatim text from
ComprehensionAttempt.evidence SourceSpanLinks — never fabricated from family
or refusal_reason. Today classify_* leaves evidence empty for refusals, so
grounded_terms is empty everywhere; the wiring is ready for the residue
upgrade to readers later.

Off-serving: limitation.py imports no generate.derivation / core.reliability_gate
(AST-guarded by test_limitation_module_is_off_serving). No renderer, no bus
delivery — those are Q1-C/Q1-D.

Tests:
  tests/test_limitation_assessment.py — 39 passing
  + test_proposal_allowed_iff_emit_proposal_with_q1b_ask_carveout (amended INV-A)
  + test_missing_{total,weighted}_count_limitation_is_ask_oriented
  + test_q1b_ask_carveout_is_explicit_and_enumerated
  + test_registry_proposal_allowed_preserved_until_ask_delivery
  + test_no_signal_loss_before_question_bus_is_serving (live pass_manager check)
  + test_missing_{total,weighted}_count_has_typed_missing_slot_residue
  + test_grounded_terms_populated_only_from_evidence_spans
  + test_residue_never_contains_ungrounded_terms
  + test_residue_empty_for_non_ask_attempts
  + test_missing_slots_default_empty_for_non_ask_families
  + test_missing_slot_keys_are_subset_of_ask_families
  + test_limitation_module_is_off_serving (AST guard)

Suites:
  tests/test_epistemic_*.py, test_disclosure_claim.py, test_served_disposition.py,
  test_verified_contract.py, test_failure_family.py, test_failure_proposal.py,
  test_r3_router_contemplation.py, test_proposal_review_*.py,
  test_idle_proposal_review.py — 135 passing
  core test --suite smoke -q — 90 passing
2026-06-08 16:51:53 -07:00
Shay
8c0311b847 feat(verified): P1-B — gold-setup-backed off-serving R2 verification producer
The first consumer of the P1-A VERIFIED contract. Given an R2 problem (prose) +
the INDEPENDENTLY hand-authored gold SETUP, it builds a VerificationProof and runs it
through the contract. NOT a serving-time VERIFIED producer: it depends on the eval-lane
gold setup (off-serving only), cannot move any serving metric, cannot emit served
[verified]. Serving-time independence (a non-gold second reader) is a separate, larger
rung, explicitly deferred.

evals/constraint_oracle/verified_producer.py (placed in evals/, not core/, to avoid a
backwards core->evals import: it needs the reader+solver (generate), the canonical
signature + gold (evals), and the contract (core) — clean direction evals->core+generate):
- verify_r2(text, gold_setup) -> R2VerificationOutcome.
- Two independent reads: primary = read_constraint_problem(text) (lineage
  constraint_reader_v1); independent = the gold-authored setup (lineage
  constraint_gold_author_v1). Convergence = equal constraint_setup_signature digests
  (the SAME function for both reads). The gold ANSWER never enters — only the gold
  STRUCTURE, through the signature.
- A wrong reader structure diverges from the gold-authored structure -> reads_disagree
  -> NOT_VERIFIED (the poison). Reader refusal -> no proof + a limitation. Solver
  boundary -> boundary_clear False -> NOT_VERIFIED. VERIFIED state/claim reached ONLY
  via disclosure_for_verification.

On the real r2_gold lane (13 records): VERIFIED=7 (exactly the 7 'solved'),
NOT_VERIFIED=6 (3 solver_refuses + 3 reader_refuses) — wrong=0.

11 tests: integration over real gold (verifies >=1, never on divergence); all 7 solved
verify; the poison (divergent independent read -> reads_disagree); reader-refusal -> no
proof + limitation; solver-boundary -> not verified; distinct/named lineages; the
sanctioned route; all digests populated; gold answer structurally cannot enter; off-serving.

NOTE (bound_slots_digest): the P1-A VerificationProof (merged, unchanged per scope) has
no bound_slots_digest field; the "bound to stated quantities" obligation is carried by
derivation_digest (hashed over the canonical constraints + solution). Flag to extend.

Off-serving; smoke 90/0.
2026-06-08 16:50:16 -07:00
Shay
ec7c680cfc feat(epistemic-disclosure): P1-A — the VERIFIED contract + meaningful-fail tests
Defines what it MEANS for a result to earn DisclosureClaim.VERIFIED /
EpistemicState.VERIFIED — the soundness≠correctness gate the whole VERIFIED lane
rests on. Contract ONLY: no producer, no reader, no solver, no serving, no
verify.py, no verified_serving_enabled, no claim emission in any serving path.

core/epistemic_disclosure/verified_contract.py:
- VerificationObligation — the declarative contract (4 load-bearing flags);
  VERIFICATION_OBLIGATION is fully strict.
- VerificationProof — the replayable proof SHAPE P1-B+ will fill (two reads kept
  separate: distinct lineages for independence, matching digests for convergence).
- evaluate_verification(proof, *, limitation, obligation) -> VerificationResult —
  refuse-preferring: VERIFIED only if EVERY obligation survives.
- disclosure_for_verification(result) — the ONLY sanctioned route to
  (EpistemicState.VERIFIED, DisclosureClaim.VERIFIED); anything else -> UNDETERMINED/NONE.

The mechanism that makes the lane safe: VERIFIED requires TWO INDEPENDENT reads
(distinct lineages) that CONVERGE on one canonical structure. A faithful solve of a
WRONG read is caught because the independent read disagrees (back-substitution alone
cannot catch a read error); "same answer twice" is rejected as not independent.
Neither gold-agreement nor no-refusal nor a second solver over one read can verify.

15 tests, all non-vacuous. THE central test rejects a faithful solve of a wrong read;
one proves the wrong-read obligation is load-bearing (relax it -> poison slips the
check); the full chain composes proof -> contract -> (VERIFIED, VERIFIED) -> P0-3
DISCLOSE, and the poison degrades to COMMIT. Off-serving; smoke 90/0.
2026-06-08 16:20:19 -07:00
Shay
178e3967ed feat(epistemic-disclosure): P0-3 — ServedDisposition mapping scaffold
The third axis of the bus: choose_served_disposition composes
EpistemicState × LimitationAssessment × DisclosureClaim → ServedDisposition.
Pure mapping — no rendering, no bus behaviour, no verify.py, no govern_response
mutation. Nothing consumes it yet.

core/epistemic_disclosure/disposition.py:
- ServedDisposition(str, Enum): commit | disclose | ask | propose | report |
  explain | refuse | step_aside.
- choose_served_disposition(*, epistemic_state, limitation, disclosure_claim=NONE):
  a blocking limitation dominates (its resolution_action -> ask/propose/report/
  explain/refuse/step_aside; scope_boundary -> EXPLAIN, NOT a hard REFUSE);
  otherwise the disclosure claim + state pick the serve mode.

The Phase-1 VERIFIED guard (load-bearing): a DisclosureClaim.VERIFIED discloses
ONLY when EpistemicState.VERIFIED backs it; an unbacked verified claim degrades to
a plain COMMIT — never served as verified before the producer exists.

30 tests: the load-bearing rules via REAL P0-1 assessments; the VERIFIED guard
parametrized across all 14 non-VERIFIED states; limitation-dominates-claim;
answer-action fall-through; totality over both axes; off-serving. Smoke 90/0.
2026-06-08 16:00:15 -07:00
Shay
8811917280 feat(epistemic-disclosure): P0-2 — the DisclosureClaim axis
A served response carries two ORTHOGONAL governance properties: ReachLevel (how
far past grounded fact it reaches) and DisclosureClaim (the epistemic claim it
makes about its own truth status). "verified" is NOT a reach level — it is a
proof-state claim. Keeping the axes separate is the locked Stage-2 decision (no
ReachLevel.VERIFIED; a proven answer never inherits an [approximate] surface).

core/epistemic_disclosure/disclosure_claim.py:
- DisclosureClaim(str, Enum): none | verified | approximate | proposal_only.
  Default = none. str-valued for stable serialization (EpistemicState convention).
- Discipline "no claim without a producer": every member has a real (APPROXIMATE
  = cognition Step E; PROPOSAL_ONLY = teaching/proposals) or imminent (VERIFIED =
  Phase 1 target) emitter. PROVEN and ESTIMATED are intentionally ABSENT — nothing
  emits them; ESTIMATED is a future split of APPROXIMATE, added only when a real
  estimator producer exists (docs-note only).

Also folds the P0-1 review nit: _KIND_TO_STATE / _ACTION_TO_TERMINAL tightened
from dict[str, ...] to their Literal key types.

9 tests, non-vacuous: default is NONE; verified is not a ReachLevel (structural);
claim axis disjoint from ResolutionAction; PROVEN and ESTIMATED absent; exactly the
approved four; str-enum serialization; module imports nothing (off-serving).

No bus behaviour, no ServedDisposition mapping (that is P0-3). Off-serving; smoke
green (90 passed) locally.
2026-06-08 15:48:20 -07:00