f66f2ee47f
Adds comprehension_propositional — the comprehension organ now reads the classic
propositional ARGUMENT FORMS end-to-end into the flagship deductive_logic ROBDD
oracle (the most robustly independent gold in the repo). The neutral MeaningGraph
now feeds FOUR independent oracles (set-membership, syllogism-validity,
total-ordering, propositional-entailment) from one interlingua — the Option-B
interlingua thesis validated.
reader.py: propositional templates (atoms are chunked NP ids; fits the existing
entities + n-ary relations + negation model — NO interlingua change, propositional
is not arithmetic-quantities):
- "if <P> then <Q>" -> implies(P, Q)
- "not <P>" -> asserted(P, negated=True)
- "<P> or <Q>" -> or(P, Q)
- "<P>" (single token) -> asserted(P) (bare-atom, single-token only to
keep the parse-or-refuse floor)
- "therefore <prop>" -> query of the same predicate
Relations now carry a negated flag end-to-end (asserted negation).
projectors.py: to_deductive_logic serializes propositional relations/query into
formula strings (keyword operators the oracle tokenizer accepts); returns None
(refusal) unless the comprehension is purely propositional, so categorical/ordering
comprehensions never leak into the entailment oracle.
evals: new evals/propositional_logic/v1 (12 cases — modus ponens/tollens,
hypothetical & disjunctive syllogism, the affirming-consequent / denying-antecedent
fallacies which the oracle marks "unknown"; gold = oracle verdict) + gold-only
runner + evals/comprehension/propositional_runner.py. Oracle "refused" (formula
unevaluable) is treated as a decline, never a wrong.
Scores: comprehension_propositional 12/12 wrong=0 (full coverage); no regression on
the 3 existing lanes (8/8, 7/8, 7/8). Capability index breadth 6->7, score
0.917231 -> 0.928622, wrong_total 0, digest 51df7bba…
Tests: reader propositional templates; to_deductive_logic projector tests;
end-to-end full-coverage wrong=0; propositional generative round-trip added to the
wrong=0 property suite (verified to BITE under a reversed-implies mutation);
capability breadth 6->7. 115 targeted + 87 smoke green. Lane SHAs 8/9 (sole miss =
public_demo env wall-clock flake; deductive_logic_v1 unchanged).
32 lines
1.2 KiB
Python
32 lines
1.2 KiB
Python
"""Phase 2b — end-to-end: the comprehension reader scored on propositional_logic.
|
|
|
|
prose -> comprehend -> to_deductive_logic -> INDEPENDENT ROBDD oracle -> verdict vs
|
|
gold. The load-bearing invariant: wrong == 0 (the reader refuses rather than emit a
|
|
formula that yields a wrong verdict). This lane reads the classic propositional
|
|
argument forms (modus ponens/tollens, hypothetical & disjunctive syllogism) and the
|
|
classic fallacies (affirming the consequent, denying the antecedent), which the
|
|
oracle correctly marks ``unknown``.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from evals.comprehension.propositional_runner import run
|
|
|
|
|
|
def test_comprehension_propositional_wrong_is_zero() -> None:
|
|
report = run()
|
|
assert report["wrong"] == 0, report["wrongs"]
|
|
|
|
|
|
def test_comprehension_propositional_has_real_coverage() -> None:
|
|
report = run()
|
|
assert report["correct"] > 0
|
|
assert report["correct"] + report["refused"] == report["total"]
|
|
|
|
|
|
def test_comprehension_propositional_full_coverage() -> None:
|
|
# Every staged case is read end-to-end with zero wrong commits — atoms are
|
|
# single tokens / reserved-free multi-word NPs, so chunking is unambiguous.
|
|
report = run()
|
|
assert report["refused"] == 0
|
|
assert report["correct"] == report["total"]
|