core-labs/core
core-labs/core
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

feat(deduction-serve): Phase 1 — deduction turn spine (flag-gated, propositional Band v1)

Browse source 
Wires the verified ROBDD entailment engine (generate/proof_chain,
716/716 wrong=0) into core chat serving. A propositional argument
('P1. P2. ... Therefore C.') is now comprehended, projected, and
decided end-to-end -- the first real working basic-logic workflow:
user question -> CORE decides -> articulated deterministic answer.

New: chat/deduction_surface.py (DEDUCTION composer, sibling to the
existing oov/narrative/example composer pattern), generate/proof_chain/
render.py (deterministic EntailmentTrace -> surface templates, no LLM),
tests/test_deduction_surface.py (15 tests incl. full propositional gold
corpus decided end-to-end across a multi-turn session, wrong=0).

Changed: core/config.py (deduction_serving_enabled flag, default off),
generate/intent.py (IntentTag.DEDUCTION, observability-only -- the core
classify_intent rule table is untouched so flag-off stays byte-identical),
chat/runtime.py (one flag-gated block in the existing pack/teaching/
partial/oov dispatcher).

Design notes (full detail in docs/research/deduction-serve-arc-phase1-
turn-spine-2026-07-23.md): the deduction check runs BEFORE generic intent
classification (never touches classify_intent's rule table) and BEFORE
the empty-vault gate (a live multi-turn probe caught turn-12 silently
falling through once the vault warmed -- pack/teaching/oov are cold-start-
only by design, deduction serving isn't). grounding_source='deduction' is
NOT registered in the closed, Workbench-coupled GroundingSource Literal
(cross-stack TS contract, inert for this arc's core-chat-only consumer,
documented as a deferred follow-up). Band v1 stays propositional-only --
categorical/syllogism arguments commit but honestly decline as
out-of-band; evals.syllogism.oracle is never imported (INV-25 intact).

[Verification]: smoke 180 passed; cognition 122 passed/1 skipped;
test_deduction_surface.py 15 passed; test_dispatch_trace.py +
test_oov_surface.py 26 passed; test_intent*.py 145 passed.
This commit is contained in:
Shay 2026-07-23 12:14:28 -07:00
parent 6a54d27a78
commit 82df3c08ae
7 changed files with 538 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

85
chat/deduction_surface.py Normal file
View file

@ -0,0 +1,85 @@
"""chat/deduction_surface.py — Phase 1 DEDUCTION composer (deduction-serve arc).
Wires the verified propositional-entailment engine (``generate.proof_chain``,
716/716 wrong=0 on ``evals/deductive_logic``) into serving. When a prompt
reads as a propositional argument — "P1. P2. ... Therefore C." — comprehend
the text into a ``MeaningGraph``, project into ``(premises, query)`` formula
strings, and decide with the sound+complete ROBDD entailment engine.
Deterministic templates only (``generate.proof_chain.render``) — no LLM, no
synthesis, matching every other composer in this package.
Band v1 scope (docs/research/deduction-serve-arc-phase0-baseline-2026-07-23.md):
propositional arguments with single-token atoms only. A categorical/syllogism
"Therefore" argument is recognized as argument-shaped but declined as
out-of-band — a production categorical decider is Band v1b, deferred.
``evals.syllogism.oracle`` must never be imported here: it is the sealed
independence oracle the comprehension lane scores against, not a serving
decider — importing it would collapse INV-25 (independent gold).
Fail-closed (INV-34): once ``looks_like_deductive_argument`` fires, every
path below returns a committed, honest surface — reader refusal, out-of-band
shape, and all four ``EntailmentTrace`` outcomes (ENTAILED/REFUTED/UNKNOWN/
REFUSED) — never a silent fall-through to a different composer.
"""
from __future__ import annotations
import re
from generate.meaning_graph.projectors import to_deductive_logic
from generate.meaning_graph.reader import Comprehension, comprehend
from generate.proof_chain.entail import evaluate_entailment_with_trace
from generate.proof_chain.render import render_entailment
#: An argument's conclusion clause starts a sentence with "therefore" — the
#: same shape ``generate.meaning_graph.reader`` recognizes per-clause
#: (``toks[0] == "therefore"``). Matching at this commit-gate with the same
#: sentence-initial discipline avoids drift between "looks like an argument"
#: and "is an argument" — the reader remains the sole decider of the latter.
_ARGUMENT_CONCLUSION_RE = re.compile(r"(?:^|[.!?]\s+)therefore\b", re.IGNORECASE)
def looks_like_deductive_argument(text: str) -> bool:
"""True iff *text* has a sentence-initial "therefore" conclusion clause.
A cheap, deterministic COMMIT gate — not a decision. A match only
signals "attempt deduction serving"; the reader and projector below
remain the sole authority on whether the argument is actually
well-formed and in-band.
"""
return bool(_ARGUMENT_CONCLUSION_RE.search(text))
_READER_REFUSAL_SURFACE = (
"That reads as an argument, but I can't parse it precisely enough to "
"decide it yet ({reason})."
)
_OUT_OF_BAND_SURFACE = (
"That reads as an argument, but right now I can only decide plain "
"propositional arguments (not categorical 'all/no/some' ones yet)."
)
def deduction_grounded_surface(text: str) -> str | None:
"""Return a deterministic DEDUCTION-tier surface, or ``None``.
Returns ``None`` only when *text* is not argument-shaped at all
(``looks_like_deductive_argument`` is False) — the caller then falls
through to the pre-existing dispatch, byte-identical to before this
composer existed. Once argument-shaped, every branch below commits to
an honest surface; see the module docstring's fail-closed contract.
"""
if not looks_like_deductive_argument(text):
return None
comp = comprehend(text)
if not isinstance(comp, Comprehension):
return _READER_REFUSAL_SURFACE.format(reason=comp.reason)
projected = to_deductive_logic(comp)
if projected is None:
return _OUT_OF_BAND_SURFACE
premises, query = projected
trace = evaluate_entailment_with_trace(premises, query)
return render_entailment(trace, premises, query)
__all__ = ["deduction_grounded_surface", "looks_like_deductive_argument"]

41
chat/runtime.py
View file

@ -465,6 +465,19 @@ class ChatResponse:
# "teaching" — answer drawn from a reviewed teaching-chain corpus
# (cold-start CAUSE/VERIFICATION — ADR-0052).
# "none" — universal "insufficient grounding" disclosure on stub.
# "deduction" — answer decided by the verified propositional
# entailment engine (deduction-serve arc, Phase 1;
# chat/deduction_surface.py). NOTE: this value is NOT
# yet registered in core.epistemic_state.GroundingSource
# (the Workbench-coupled closed Literal) or the Workbench
# UI badge contract — epistemic_state_for_grounding_source
# falls through to the honest EPISTEMIC_STATE_NEEDED
# default for it. core chat REPL turns do not flow
# through Workbench's CognitivePipelineRecord path, so
# this is inert today; registering "deduction" as a
# first-class GroundingSource + Workbench badge is
# deferred to a follow-up if/when that visibility is
# needed.
# The string is preserved verbatim in TurnEvent for downstream audit.
grounding_source: str = "none"
# ADR-0071 (R4) — pre-decoration surface. ``surface`` is the
@ -1720,16 +1733,34 @@ class ChatRuntime:
to a walk fragment. CAUSE / VERIFICATION still return None
when no teaching chain exists, preserving the discovery signal.
"""
if not allow_warm and gate_source != "empty_vault":
if attempts is not None:
for src in ("pack", "teaching", "partial", "oov"):
attempts.append(DispatchAttempt(source=src, outcome="skipped", reason="warm_path_disabled"))
return None
if self.config.output_language != "en":
if attempts is not None:
for src in ("pack", "teaching", "partial", "oov"):
attempts.append(DispatchAttempt(source=src, outcome="skipped", reason="non_english_output"))
return None
# Deduction-serve arc, Phase 1 — checked BEFORE the empty-vault gate
# below (unlike pack/teaching/partial/oov, deduction serving is not a
# cold-start-only fallback: a user may ask a logic question at any
# point in a conversation, warm or cold vault). A pure function of
# the input text — no vault/field dependency — so it is safe to
# decide unconditionally of ``gate_source``/``allow_warm``.
if self.config.deduction_serving_enabled:
from chat.deduction_surface import deduction_grounded_surface
from generate.intent import DialogueIntent, IntentTag as _IntentTag
deduction_surface = deduction_grounded_surface(text)
if deduction_surface is not None:
self._last_intent = DialogueIntent(tag=_IntentTag.DEDUCTION, subject=text) # W-013
if attempts is not None:
attempts.append(DispatchAttempt(source="deduction", outcome="admitted", reason="deduction_composer_committed"))
return (deduction_surface, "deduction", ())
if attempts is not None:
attempts.append(DispatchAttempt(source="deduction", outcome="skipped", reason="not_argument_shaped"))
if not allow_warm and gate_source != "empty_vault":
if attempts is not None:
for src in ("pack", "teaching", "partial", "oov"):
attempts.append(DispatchAttempt(source=src, outcome="skipped", reason="warm_path_disabled"))
return None
from generate.intent import IntentTag
from generate.intent_bridge import classify_intent_from_input
intent = classify_intent_from_input(text)

13
core/config.py
View file

@ -371,6 +371,19 @@ class RuntimeConfig:
# default); the engine never raises its own ceiling.
estimation_enabled: bool = False
# Deduction-serve arc, Phase 1 — when on, ``chat/deduction_surface.py``
# intercepts propositional-argument-shaped turns ("P1. P2. ... Therefore
# C.") BEFORE generic intent dispatch and decides them with the verified
# ROBDD entailment engine (generate.proof_chain, 716/716 wrong=0) instead
# of the pack-token-gloss fallback. Band v1 scope: single-token
# propositional atoms only (docs/research/deduction-serve-arc-phase0-
# baseline-2026-07-23.md); categorical/syllogism arguments are recognized
# as argument-shaped but honestly declined as out-of-band. OFF by
# default: flag-off is byte-identical to pre-arc dispatch — the new
# composer is never called, so previously-UNKNOWN "therefore" prompts
# keep their existing pack-token-gloss surface exactly.
deduction_serving_enabled: bool = False
# ASK serving gate enable flag. When True, ASK serving is allowed.
# Default False (dark).
ask_serving_enabled: bool = False

152
docs/research/deduction-serve-arc-phase1-turn-spine-2026-07-23.md Normal file
View file

@ -0,0 +1,152 @@
# Deduction-serve arc — Phase 1 (deduction turn spine), 2026-07-23
**Base:** `main` @ `6a54d27a` (same base as Phase 0). **Branch:** `feat/deduction-serve-phase1`.
**Depends on:** Phase 0 baseline (`docs/research/deduction-serve-arc-phase0-baseline-2026-07-23.md`).
## What shipped
`core chat` now decides propositional-argument questions with the verified
ROBDD entailment engine, behind a default-off flag. The workflow the arc set
out to prove is real end-to-end:
```
$ core chat
> If p then q. p. Therefore q.
Given: p implies q; p. Your premises entail: q.
```
Flag off, the exact same input still returns the pre-arc pack-token-gloss
surface, byte-for-byte — nothing changes for any existing deployment until
the flag is explicitly turned on.
## New files
- **`chat/deduction_surface.py`** — the DEDUCTION composer, sibling to
`chat/oov_surface.py` / `narrative_surface.py` / `example_surface.py` (same
established pattern: a pure `<name>_grounded_surface(text) -> str | None`
function `chat/runtime.py` calls into). Owns the commit-gate
(`looks_like_deductive_argument`) and the fail-closed decision tree:
reader refusal → typed decline, out-of-band shape → typed decline,
otherwise decide via the engine and render.
- **`generate/proof_chain/render.py`** — `render_entailment(trace, premises, query)`,
a sibling to `generate/determine/render.py`'s `render_determination` but for
`EntailmentTrace` instead of `Determined`. Four deterministic templates
(ENTAILED / REFUTED / UNKNOWN / REFUSED), no LLM, no synthesis — every
visible token is either fixed prose or a literal formula string the
projector produced from the user's own text.
- **`tests/test_deduction_surface.py`** — 15 tests: pure-function contract,
flag-off byte-identity, flag-on single-turn behavior, and an end-to-end
regression that decides the **full propositional gold corpus through the
real `ChatRuntime.chat()` path** across one multi-turn session (not just
the isolated comprehend→project→oracle lane) — wrong=0.
## Changed files (small, surgical)
- **`core/config.py`** — `RuntimeConfig.deduction_serving_enabled: bool = False`.
- **`generate/intent.py`** — `IntentTag.DEDUCTION` added. Additive only:
`classify_intent` / `classify_compound_intent` never produce it — the
DEDUCTION composer runs *before* generic intent classification is even
invoked (see "Design decisions" below), and only stamps this tag onto
`self._last_intent` for `/explain` observability fidelity on a turn it
actually commits to.
- **`chat/runtime.py`** — one flag-gated block inside
`_maybe_pack_grounded_surface` (the existing pack/teaching/partial/oov
dispatcher) plus a `grounding_source` docstring update. No other lines
touched.
## Design decisions (and why)
**1. A shape-check + direct composer, not `IntentTag`-routed classification.**
The plan called for "IntentTag.DEDUCTION + classification." Tracing
`classify_intent`'s call sites first surfaced a real hazard: if the core
classifier itself started returning `DEDUCTION` for "therefore"-shaped text
*unconditionally* (regardless of the flag), previously-`UNKNOWN` prompts
would silently stop reaching the pack-token-gloss branch even with the flag
off — a byte-identity violation. The fix: `classify_intent`'s own rule table
is untouched; a separate, flag-gated pre-check
(`looks_like_deductive_argument`) intercepts *before* generic classification
runs at all. Flag off, the new code path is never entered — provably
byte-identical (verified: see tests + manual probe below).
**2. `grounding_source="deduction"` is NOT registered in
`core.epistemic_state.GroundingSource`.** That `Literal` is a closed,
cross-stack contract — mirrored in `workbench/schemas.py`, a TypeScript union
in `workbench-ui/src/types/api.ts`, a badge-metadata table, and a build-time
`enumCoverage.test.ts` snapshot test. Extending it properly means touching
Python *and* TypeScript *and* regenerating a UI enum snapshot — real scope,
not a one-line addition. Since `core chat` REPL turns never flow through
Workbench's `CognitivePipelineRecord` path (confirmed: no `TurnEvent` import
anywhere under `workbench/`), the extension is inert for this arc's actual
consumer. `ChatResponse.grounding_source` itself is typed as plain `str` (not
the `Literal`), so nothing breaks at runtime — `epistemic_state_for_grounding_source("deduction")`
falls through to the honest `EPISTEMIC_STATE_NEEDED` default. Registering
"deduction" as a first-class `GroundingSource` + Workbench badge is deferred
to a follow-up if/when that visibility is actually needed — documented
in-line at both the flag and the field.
**3. The dispatch-order bug an early manual probe caught.** First
implementation placed the deduction check *after* the existing
`if not allow_warm and gate_source != "empty_vault": return None` early-out
(matching the literal insertion point I'd scoped). Running the full
propositional gold corpus through one live multi-turn session caught it
immediately: turn 12 fell through silently once the vault was no longer
"empty" (`gate_source` had changed), because that early-out returns *before*
reaching any composer. Pack/teaching/partial/oov are legitimately
cold-start-only fallbacks — but deduction serving isn't: a user should be
able to ask a logic question at any point in a conversation. Fix: moved the
deduction check to run *before* that gate (it depends only on input text, no
vault/field state). Re-running the full corpus confirmed 12/12 correct, 0
wrong, 0 declined, 0 mis-routed, across the same session.
**4. Band v1 stays propositional-only, exactly as scoped in Phase 0.**
Categorical/syllogism "therefore" arguments *commit* (they're
argument-shaped) but are honestly declined as out-of-band
(`"...not categorical 'all/no/some' ones yet"`) rather than silently
mis-served or falling through to the old gloss. `evals.syllogism.oracle` is
never imported anywhere in the new code — the sealed independence oracle
stays sealed (INV-25 intact). A production categorical decider is Band v1b,
deferred per the Phase 0 fork analysis.
## Verification
```
uv run python -m pytest tests/test_deduction_surface.py -q # 15 passed
uv run core test --suite smoke -q # 180 passed
uv run core test --suite cognition -q # 122 passed, 1 skipped
uv run python -m pytest tests/test_dispatch_trace.py tests/test_oov_surface.py -q # 26 passed
uv run python -m pytest tests/test_intent*.py -q # 145 passed
```
Manual end-to-end probe (flag on) — the exact workflow the arc targeted:
```
If p then q. p. Therefore q.
-> Given: p implies q; p. Your premises entail: q.
p or q. Not p. Therefore q.
-> Given: p or q; not p. Your premises entail: q.
p. Not p. Therefore q.
-> Given: p; not p. Those premises are inconsistent — they can't all be
true, so I won't assert anything from them.
p or q. Therefore p.
-> Given: p or q. Your premises don't settle whether p — it holds in
some cases and fails in others.
All mammals are animals. All whales are mammals. Therefore all whales are animals.
-> That reads as an argument, but right now I can only decide plain
propositional arguments (not categorical 'all/no/some' ones yet).
If it rains then the ground is wet. It rains. Therefore the ground is wet.
-> That reads as an argument, but I can't parse it precisely enough to
decide it yet (reserved_word_in_np).
```
Flag off, the same six inputs return byte-identical pre-arc surfaces
(`grounding_source="pack"`, the token-gloss text) — confirmed directly, not
merely assumed.
## Verdict
Phase 1 complete. The `core chat` REPL genuinely decides basic propositional
logic questions end-to-end, deterministically, with wrong=0 held across the
full gold corpus and a multi-turn session. Proceed to Phase 2 (a formal,
SHA-pinned serving-path eval lane, scoring the production decider itself —
not the isolated comprehend→project→oracle chain the Phase 0 lanes already
cover).

7
generate/intent.py
View file

@ -56,6 +56,13 @@ class IntentTag(Enum):
# P3.4 — "Give me an example of X" / "Show an instance of X" —
# reverse-chain composer surfaces chains where X is the object.
EXAMPLE = "example"
# Deduction-serve arc, Phase 1 — a propositional-argument-shaped turn
# ("P1. P2. ... Therefore C.") committed to chat/deduction_surface.py's
# composer. Never produced by classify_intent/classify_compound_intent
# (that routing table is untouched — this tag is observability-only,
# set by the runtime when the deduction composer commits, so /explain
# reflects what actually happened on a deduction turn).
DEDUCTION = "deduction"
UNKNOWN = "unknown"

54
generate/proof_chain/render.py Normal file
View file

@ -0,0 +1,54 @@
"""Deterministic surface rendering for a propositional ``EntailmentTrace``
(deduction-serve arc, Phase 1).
Renders the four ``Entailment`` outcomes into user-facing prose. Deterministic
templates only — no LLM, no synthesis; every visible token is either fixed
prose or a formula string ``to_deductive_logic`` produced directly from the
user's own text. This is the ONLY renderer for propositional-deduction
serving; it must not be confused with ``generate.determine.render`` (which
renders realized-structure DETERMINE answers — a different gear entirely).
"""
from __future__ import annotations
from generate.proof_chain.entail import (
INCONSISTENT_PREMISES,
Entailment,
EntailmentTrace,
)
def render_entailment(
trace: EntailmentTrace, premises: tuple[str, ...], query: str
) -> str:
"""The user-facing surface for a propositional deduction verdict.
Every branch is a fixed template around the literal premise/query
formula strings — no fabricated content. ``trace.outcome`` selects the
template; REFUSED further branches on ``trace.reason`` since an
inconsistent-premises refusal and an out-of-regime refusal warrant
different honest phrasing.
"""
given = "; ".join(premises)
if trace.outcome is Entailment.ENTAILED:
return f"Given: {given}. Your premises entail: {query}."
if trace.outcome is Entailment.REFUTED:
return (
f"Given: {given}. Your premises entail the opposite of "
f"{query} — it cannot hold."
)
if trace.outcome is Entailment.UNKNOWN:
return (
f"Given: {given}. Your premises don't settle whether "
f"{query} — it holds in some cases and fails in others."
)
# REFUSED
if trace.reason == INCONSISTENT_PREMISES:
return (
f"Given: {given}. Those premises are inconsistent — they "
f"can't all be true, so I won't assert anything from them."
)
return f"Given: {given}. I can't evaluate {query} from that as stated."
__all__ = ["render_entailment"]

191
tests/test_deduction_surface.py Normal file
View file

@ -0,0 +1,191 @@
"""Deduction-serve arc, Phase 1 — DEDUCTION composer tests.
The contract these tests pin:
- ``deduction_serving_enabled`` is OFF by default; flag-off ``chat()``
output is byte-identical to pre-arc behavior for argument-shaped text
(the pack-token-gloss fallback, unchanged).
- Flag-on: a sentence-initial "therefore" conclusion clause commits the
turn to ``chat/deduction_surface.py``; every committed turn returns a
surface (never a silent fall-through to a different composer).
- The full propositional comprehension gold corpus (Band v1) decides
correctly end-to-end through the real ``ChatRuntime.chat()`` path,
across multiple turns in one session (wrong=0).
- Out-of-band shapes (categorical/syllogism, multi-word English
propositions) are honestly declined, not silently misrouted.
- Non-argument-shaped text is untouched regardless of the flag.
"""
from __future__ import annotations
from chat.deduction_surface import (
deduction_grounded_surface,
looks_like_deductive_argument,
)
from chat.runtime import ChatRuntime
from core.config import RuntimeConfig
from evals.propositional_logic.runner import _load_cases
# ---------------------------------------------------------------------------
# Pure-function contract — looks_like_deductive_argument
# ---------------------------------------------------------------------------
def test_looks_like_deductive_argument_true_for_therefore_conclusion() -> None:
assert looks_like_deductive_argument("If p then q. p. Therefore q.")
assert looks_like_deductive_argument("p or q. Not p. Therefore q.")
assert looks_like_deductive_argument(
"All mammals are animals. All whales are mammals. "
"Therefore all whales are animals."
)
def test_looks_like_deductive_argument_requires_sentence_initial_therefore() -> None:
""""therefore" mid-clause (not its own conclusion clause) does not commit
the turn — matches the reader's own per-clause discipline."""
assert not looks_like_deductive_argument("What is light therefore darkness")
assert not looks_like_deductive_argument("What is light?")
assert not looks_like_deductive_argument("")
# ---------------------------------------------------------------------------
# Pure-function contract — deduction_grounded_surface
# ---------------------------------------------------------------------------
def test_non_argument_text_returns_none() -> None:
assert deduction_grounded_surface("What is light?") is None
assert deduction_grounded_surface("Why does parent exist?") is None
def test_entailed_case_renders_entailment() -> None:
surface = deduction_grounded_surface("If p then q. p. Therefore q.")
assert surface is not None
assert "entail: q" in surface
def test_refuted_case_renders_refutation() -> None:
surface = deduction_grounded_surface(
"If p then q. Not q. Therefore not p."
)
assert surface is not None
assert "entail" in surface
def test_unknown_case_renders_honest_indeterminacy() -> None:
surface = deduction_grounded_surface("p or q. Therefore p.")
assert surface is not None
assert "don't settle" in surface
def test_inconsistent_premises_renders_typed_decline() -> None:
surface = deduction_grounded_surface("p. Not p. Therefore q.")
assert surface is not None
assert "inconsistent" in surface
def test_categorical_argument_declines_out_of_band() -> None:
"""A syllogism-shaped "therefore" argument commits (argument-shaped)
but is honestly declined — Band v1 is propositional-only; the
production categorical decider is deferred (Band v1b)."""
surface = deduction_grounded_surface(
"All mammals are animals. All whales are mammals. "
"Therefore all whales are animals."
)
assert surface is not None
assert "categorical" in surface
def test_multiword_conditional_declines_reader_refusal() -> None:
"""Natural-English multi-word propositions are out of Band v1's
single-token-atom scope; the reader refuses and the composer
surfaces that honestly rather than silently falling through."""
surface = deduction_grounded_surface(
"If it rains then the ground is wet. It rains. "
"Therefore the ground is wet."
)
assert surface is not None
assert "reserved_word_in_np" in surface
def test_surface_is_deterministic() -> None:
text = "If p then q. p. Therefore q."
assert deduction_grounded_surface(text) == deduction_grounded_surface(text)
# ---------------------------------------------------------------------------
# Live runtime — flag off is byte-identical to pre-arc dispatch
# ---------------------------------------------------------------------------
def test_flag_off_preserves_pack_token_gloss_byte_identity() -> None:
rt = ChatRuntime(
config=RuntimeConfig(deduction_serving_enabled=False), no_load_state=True,
)
resp = rt.chat("If p then q. p. Therefore q.")
assert resp.grounding_source == "pack"
assert "Pack-resident tokens" in resp.surface
def test_flag_is_off_by_default() -> None:
assert RuntimeConfig().deduction_serving_enabled is False
# ---------------------------------------------------------------------------
# Live runtime — flag on, single turn
# ---------------------------------------------------------------------------
def test_runtime_deduction_serves_entailed_answer() -> None:
rt = ChatRuntime(
config=RuntimeConfig(deduction_serving_enabled=True), no_load_state=True,
)
resp = rt.chat("If p then q. p. Therefore q.")
assert resp.grounding_source == "deduction"
assert "entail: q" in resp.surface
def test_runtime_non_argument_prompt_unaffected_by_flag() -> None:
"""The flag only intercepts argument-shaped text; ordinary prompts
still route through the pre-existing dispatch."""
rt = ChatRuntime(
config=RuntimeConfig(deduction_serving_enabled=True), no_load_state=True,
)
resp = rt.chat("What is light?")
assert resp.grounding_source != "deduction"
# ---------------------------------------------------------------------------
# Live runtime — full propositional gold corpus, multi-turn session, wrong=0
# ---------------------------------------------------------------------------
def test_runtime_decides_full_propositional_gold_corpus_wrong_zero() -> None:
"""End-to-end regression: every Band v1 gold case, decided through the
real ``core chat`` serving path across one multi-turn session (not
just the isolated comprehend->project->oracle lane). wrong=0 is the
load-bearing assertion; declines are acceptable, a disagreement with
gold is not."""
rt = ChatRuntime(
config=RuntimeConfig(deduction_serving_enabled=True), no_load_state=True,
)
cases = _load_cases()
assert cases, "gold corpus must not be empty"
wrong: list[str] = []
for case in cases:
resp = rt.chat(case["text"])
assert resp.grounding_source == "deduction", (
f"expected deduction routing for {case['text']!r}, "
f"got grounding_source={resp.grounding_source!r}"
)
surface = resp.surface
gold = case["gold"]
if gold == "entailed" and "Your premises entail:" in surface:
continue
if gold == "refuted" and "entail the opposite" in surface:
continue
if gold == "unknown" and "don't settle" in surface:
continue
wrong.append(f"{case['text']!r} -> gold={gold} surface={surface!r}")
assert not wrong, "\n".join(wrong)