Shay b938ad617f feat(deduction-serve): Band v3-MEM — member-chain band, Socrates syllogism decided (ADR-0258)

Per-individual propositional lowering over minted (individual, class)
atoms: singular membership/predicate facts + all/every/each/no universals,
instantiated at every named individual, decided by the same verified ROBDD
engine. Closed-table number linking (irregular/invariant priority + three
regular suffix rules) is the one semantic identification — attested-pair
linking only, under-link over over-link. Composer tier strictly after
v1/v1b/v2-EN: monotone widening, v1 lane byte-identical.

- 4 new shape-bands (en_member_single/chain/negative/atomic) each earned
  SERVE at the arena: 13 bands x 720 = 9,360 wrong=0; ledger re-sealed.
- New hand-authored lane evals/deduction_serve/v2_member (26/26); ds-en-0022
  promoted declined->entailed (the designed acceptance case); report
  re-pinned; contract.md refreshed to the 4-band cascade.
- Typed refusals: quantifier pronouns, bare plurals, definite descriptions,
  relative clauses, tense, mixed connectives, universal conclusions.
- Flag posture unchanged (deduction_serving_enabled default off).

[Verification]: deductive 127 / smoke 180 / warmed_session 10; arena
all_bands_serve_licensed wrong_is_zero; lanes v1 28/28, v2_en 26/26,
v2_member 26/26.

2026-07-23 19:21:41 -07:00

4.4 KiB

Raw Blame History

Deduction-serve lane contract

What this lane scores

The production serving decider — the exact pipeline chat/deduction_surface.py::deduction_grounded_surface runs on a core chat turn: looks_like_deductive_argument (commit gate) → comprehend (reader) → the band cascade — to_deductive_logic (Band v1) → to_syllogism + the categorical decider (Band v1b, ADR-0256) → read_english_argument (Band v2-EN, ADR-0257) → read_member_argument (Band v3-MEM, ADR-0258) — with evaluate_entailment_with_trace (the ROBDD engine, ADR-0201/ADR-0218) deciding every band. evals/deduction_serve/runner.py::decide calls these functions directly (typed outcome, not rendered prose) — the same production decision the composer makes, without re-deriving the presentation step (generate.proof_chain.render), so this lane's pinned bytes stay stable against wording-only changes.

Splits

v1/ — the original corpus (single-token propositional + categorical).
v2_en/ — hand-authored REAL-English clause arguments (ADR-0257); content disjoint from the synthetic practice lexicon.
v2_member/ — hand-authored membership/universal arguments (ADR-0258), incl. real nouns across every number-link row-type.

This is distinct from two existing lanes that sound similar:

evals/deductive_logic scores the bare entail.py engine against hand-authored formula strings — it never touches the reader.
evals/comprehension/propositional_runner.py scores reader fidelity by running the reader's projection through the independent oracle (evals.deductive_logic.oracle) as the decision procedure.

This lane is the only one that scores the production ROBDD engine (entail.py, not the oracle) end-to-end from raw text — proving the capability core chat actually serves, not just its parts in isolation.

Gold vocabulary

Four classes: entailed, refuted, unknown, declined.

declined covers every honest non-commitment: inconsistent premises (REFUSED), an out-of-band shape (categorical/syllogism, multi-word English propositions, nested negation inside an if/then clause — see "Known Band v1 boundaries" below), or a shape that doesn't even commit the turn (looks_like_deductive_argument false — not exercised by this corpus, since every committed case reads as an argument by design).

wrong=0 discipline

wrong — the pipeline committed to a definite entailed/refuted/ unknown verdict that disagrees with gold. Must stay 0.
declined (mismatch) — the pipeline declined on a case gold expected a definite verdict for. Not a wrong (never a confabulation), but not a pass either — the runner requires correct == n (every case's outcome class matches gold exactly, including declines matching declined gold).
A case that gold marks declined and the pipeline also declines is correct — the lane rewards honest recognition of the boundary, not just committed accuracy.

Band-boundary history this corpus documents

Boundaries are DISCOVERED as declines, then PROMOTED to decided gold when a later band earns the shape (the corpus keeps the case, renamed …_formerly_out_of_band):

Nested negation inside if/then — a shared-reader grammar limit (not is reserved inside if/then slots); ds-v1-0006 declined until Band v2-EN decided it (promoted, ADR-0257).
Multi-word English propositions — ds-v1-0025 declined until Band v2-EN (promoted, ADR-0257).
Categorical/syllogism shapes — ds-v1-0023/0024/0026, decided by Band v1b since ADR-0256.
is a membership — ds-en-0022 declined until Band v3-MEM decided it (promoted, ADR-0258).
Still-open declines (honest, typed): verb-phrase negation (ds-en-0023/0024), ambiguous and/or scope (ds-en-0025), nested conditionals (ds-en-0026), existential quantifiers / bare plurals / definite descriptions / relative clauses / tense (ds-mem-0020…0026).

Reproduce

uv run python -m evals.deduction_serve.runner                              # human-facing
uv run python -m evals.deduction_serve.runner --report evals/deduction_serve/report.json  # pinned artifact

Pinned in scripts/verify_lane_shas.py as lane id deduction_serve_v1. core test --suite deductive runs tests/test_deduction_serve_lane.py, which asserts wrong == 0 and all_cases_correct is True against the committed corpus.

4.4 KiB Raw Blame History