feat(reader): add overlaps_event finite-verb reader surface (B3) (#783)
Add a closed finite-verb relational surface to comprehend_relational, narrowly: declarative <A> overlaps <B>, interrogative Does <A> overlap <B>?, predicate overlaps_event ONLY (closed finite-verb table, default-off). Kept SEPARATE from the copula-connective grammar (byte-unchanged): the other connectives (before/after/during/inside/adjacent/…) still REQUIRE the copula, so "Monday before Friday." stays a refusal — no connective bypass (adversarial hazard #2). Fail-closed slot gate: each finite-verb argument slot must be EXACTLY ONE content token (after article stripping). An enumerated adverb/negation blocklist is unbounded and leaks — the adversarial audit found "Meeting never overlaps lunch." committed as a POSITIVE overlaps_event(meeting_never, lunch), plus almost/sometimes/trailing-qualifier/ second-verb compound fabrications in BOTH slots and the query path. The single-token gate closes the whole COMPOUND-fabrication class (any extra token refuses); the _FINITE_VERB_MODIFIERS and _CONNECTIVE_TOKENS checks add precise reasons for common bare-modifier / second-verb slots. Multi-word entities in the finite-verb surface are deferred (they need a positive content lexicon, which OOV entities preclude). A lone bare token remains an entity (the reader's universal OOV single-token contract — the copula and general readers behave identically), not a compound fabrication. Hazards pinned (adversarial audit hazard #1 + #2): adverb absorption, negation-as- positive ('never'), interrogative double-verb, trailing qualifier, and the copula-bypass firewall. Measurement: migrated ref-009 "Sunrise overlaps dawn." (the documented coverage gap) to a positive rel-018 — comprehension_relational_predicate 17->18, wrong_total 0, breadth still 11 (coverage added, not a new domain), baseline digest re-frozen. Refusal floor rose 9 -> 24 (finite-verb confusers replace the one migrated input). The #775 inference and #781 transitive lane SHA pins are UNTOUCHED. determine.py is untouched (reader-only): no answer=False, no FrameVerdict, INV-30 unaffected. Verified in the worktree: finite-verb + reader + reader-lane + #775 inference lane + #781 transitive lane + #779 OWA floor + capability baseline/index + INV-30/29/21 + full smoke — all green. Two read-only adversarial passes: the first found 4 real slot- fabrication blockers, closed by the single-token gate; a black-box probe of 24 adversarial inputs confirms the compound-fabrication class is fully closed.
This commit is contained in:
Shay
GitHub
parent
01966ef92e
commit
0be18ebdb9
7 changed files with 366 additions and 22 deletions
|
|
@ -1,13 +1,13 @@
|
|||
{
|
||||
"capability_score": 0.953968,
|
||||
"coverage_geomean": 0.953968,
|
||||
"coverage_micro": 0.993983,
|
||||
"coverage_micro": 0.99399,
|
||||
"accuracy_micro": 1.0,
|
||||
"breadth": 11,
|
||||
"min_domain_coverage": 0.833333,
|
||||
"wrong_total": 0,
|
||||
"assert_mode_valid": true,
|
||||
"deterministic_digest": "c064abc9d13780682632423756a7fab58c8f043ad752f6e136edd305f31ab8b1",
|
||||
"deterministic_digest": "85c873626423c6b151a7d61bebb0cd5b7caff7dcc7e897188609e5ce3313feac",
|
||||
"domains": [
|
||||
{
|
||||
"domain": "comprehension_propositional",
|
||||
|
|
@ -35,7 +35,7 @@
|
|||
},
|
||||
{
|
||||
"domain": "comprehension_relational_predicate",
|
||||
"correct": 17,
|
||||
"correct": 18,
|
||||
"wrong": 0,
|
||||
"refused": 0,
|
||||
"coverage": 1.0,
|
||||
|
|
|
|||
|
|
@ -15,3 +15,4 @@
|
|||
{"id": "rel-015", "text": "Recess is during morning.", "predicate": "during_event", "subject": "recess", "object": "morning"}
|
||||
{"id": "rel-016", "text": "Zorptak is the parent of Quxley.", "predicate": "parent_of", "subject": "zorptak", "object": "quxley"}
|
||||
{"id": "rel-017", "text": "Glarn is less than Brimble.", "predicate": "less_than", "subject": "glarn", "object": "brimble"}
|
||||
{"id": "rel-018", "text": "Sunrise overlaps dawn.", "predicate": "overlaps_event", "subject": "sunrise", "object": "dawn"}
|
||||
|
|
|
|||
|
|
@ -6,4 +6,19 @@
|
|||
{"id": "ref-006", "text": "Alice is not the parent of Bob.", "why": "negation is out of grammar — never reads as a positive"}
|
||||
{"id": "ref-007", "text": "Alice greets Bob.", "why": "no relational connective / no copula"}
|
||||
{"id": "ref-008", "text": "The weather is nice today.", "why": "no relational template at all"}
|
||||
{"id": "ref-009", "text": "Sunrise overlaps dawn.", "why": "verb-form relation, no copula — the honest overlaps_event coverage gap"}
|
||||
{"id": "ref-010", "text": "Alice overlaps with the team and Bob.", "why": "finite-verb overlap with coordinated/leftover structure (with … and …) — must not fabricate a compound entity"}
|
||||
{"id": "ref-011", "text": "A overlaps B and C.", "why": "coordinated object slot — 'and' is a parse leak, must refuse"}
|
||||
{"id": "ref-012", "text": "A nearly overlaps B.", "why": "adverb-modified overlap — 'nearly' must not be absorbed into a fabricated entity a_nearly"}
|
||||
{"id": "ref-013", "text": "A completely overlaps B.", "why": "adverb-modified overlap — 'completely' is a finite-verb modifier, must refuse"}
|
||||
{"id": "ref-014", "text": "Does A nearly overlap B?", "why": "adverb-modified overlap in the query form — must refuse"}
|
||||
{"id": "ref-015", "text": "A does not overlap B.", "why": "negation is out of grammar — never reads as a positive overlap"}
|
||||
{"id": "ref-016", "text": "A borders B.", "why": "finite verb not in the closed table — only 'overlaps' is admitted"}
|
||||
{"id": "ref-017", "text": "Monday before Friday.", "why": "connective without copula — admitting the finite verb must NOT let other connectives bypass the copula"}
|
||||
{"id": "ref-018", "text": "The cat inside the house.", "why": "connective without copula — inside_of still requires the copula"}
|
||||
{"id": "ref-019", "text": "The room adjacent the hall.", "why": "connective without copula — adjacent_to still requires the copula"}
|
||||
{"id": "ref-020", "text": "A overlaps B overlaps C.", "why": "a second finite verb — leftover relational structure, must refuse"}
|
||||
{"id": "ref-021", "text": "A overlaps B then C.", "why": "sequencing modifier 'then' must not be absorbed into a fabricated entity b_then_c"}
|
||||
{"id": "ref-022", "text": "Meeting never overlaps lunch.", "why": "negation 'never' (NOT a reserved word) must not be glued into meeting_never and committed as a POSITIVE overlaps_event — the worst wrong=0 breach the audit found"}
|
||||
{"id": "ref-023", "text": "Alice almost overlaps Bob.", "why": "an adverb outside the enumerated modifier set — the single-token slot gate must refuse, not fabricate alice_almost"}
|
||||
{"id": "ref-024", "text": "Meeting overlaps lunch today.", "why": "trailing temporal qualifier must not be glued into lunch_today — a 2-token slot refuses"}
|
||||
{"id": "ref-025", "text": "Does A overlap B overlap C?", "why": "interrogative double-verb — the base form 'overlap' is not in the connective net; the single-token slot gate refuses b_overlap_c"}
|
||||
|
|
|
|||
|
|
@ -25,11 +25,14 @@ Transitive (direct + through-inverse), asymmetric self-converse (`less_than`,
|
|||
`equal_to`), object mismatch, and ungrounded. A `Determined` on any of these is a
|
||||
wrong=0 breach — the lane test fails loudly.
|
||||
|
||||
## Known coverage gap (honest)
|
||||
## Resolved coverage gap (B3)
|
||||
|
||||
`overlaps_event` (symmetric, pack-declared `graph.edge.symmetric`) is **excluded** from
|
||||
the positive gold: the relational reader requires a copula ("X is _connective_ Y"), but
|
||||
`overlaps` is a finite verb — "The meeting overlaps the lunch" refuses
|
||||
(`no_relational_template`) and "is overlaps" is ungrammatical. This is a **reader
|
||||
surface limitation, not a determination gap** (the symmetric rule would fire if a fact
|
||||
were stored). A finite-verb relational surface is brief-pack follow-up work.
|
||||
`overlaps_event` (symmetric, pack-declared `graph.edge.symmetric`) was previously
|
||||
**excluded** from the positive gold: the relational reader required a copula
|
||||
("X is _connective_ Y"), but `overlaps` is a finite verb. The closed finite-verb reader
|
||||
surface (B3 — `<A> overlaps <B>` / `Does <A> overlap <B>?`, see
|
||||
`generate/meaning_graph/relational.py::_read_finite_verb_clause`) now reads it, so a
|
||||
stored overlap and its symmetric converse determine (proven in
|
||||
`tests/test_relational_finite_verb.py`). The reader-surface coverage lives in
|
||||
`evals/relational`; adding a symmetric `overlaps_event` case to THIS inference lane is
|
||||
optional follow-up.
|
||||
|
|
|
|||
|
|
@ -11,13 +11,18 @@ relational set for DIRECT entailment.
|
|||
|
||||
Fail-closed (wrong=0 at the comprehension layer):
|
||||
|
||||
- a clause is read ONLY when it matches ``<A> is [the] <connective> <B>`` with the
|
||||
copula sitting STRUCTURALLY adjacent to the connective, AND the connective's lemma
|
||||
is present in the LOADED pack (``pack_lemmas``), AND neither argument slot carries
|
||||
leftover relational/reserved vocabulary. A non-matching surface, a dangling copula,
|
||||
a negated form, an argument slot holding a connective token (a trailing qualifier
|
||||
like "… of Dan during school" — unparsed structure, NOT an entity), or a lemma
|
||||
absent from the pack all REFUSE — never guess, never field-vote. Deterministic.
|
||||
- a clause is read ONLY by one of two CLOSED surfaces: the copula-connective grammar
|
||||
``<A> is [the] <connective> <B>`` (the copula sitting STRUCTURALLY adjacent to the
|
||||
connective), OR the closed finite-verb surface ``<A> overlaps <B>`` /
|
||||
``Does <A> overlap <B>?`` (no copula; ``overlaps_event`` ONLY — see
|
||||
``_read_finite_verb_clause``). In BOTH the lemma must be present in the LOADED pack
|
||||
(``pack_lemmas``) AND neither argument slot may carry leftover relational/reserved
|
||||
vocabulary or an adverb modifier. A non-matching surface, a dangling copula, a
|
||||
connective WITHOUT its copula ("Monday before Friday."), a negated form, an
|
||||
adverb-modified overlap ("A nearly overlaps B"), an argument slot holding a connective
|
||||
token (a trailing qualifier like "… of Dan during school" — unparsed structure, NOT an
|
||||
entity), or a lemma absent from the pack all REFUSE — never guess, never field-vote.
|
||||
Deterministic.
|
||||
- only the PREDICATE is closed-vocabulary; the two arguments may be OOV (arbitrary
|
||||
identifiers), grounded downstream by the OOV substrate. A clean multi-word entity
|
||||
("north station") canonicalizes per the join contract; an argument that still holds
|
||||
|
|
@ -174,6 +179,32 @@ _CONNECTIVE_TOKENS = frozenset(tok for key in _CONNECTIVE_TO_LEMMA for tok in ke
|
|||
|
||||
_MAX_CONNECTIVE_LEN = max(len(k) for k in _CONNECTIVE_TO_LEMMA)
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Finite-verb surface (B3) — a verb-form relation with NO copula. CLOSED and
|
||||
# default-off, kept SEPARATE from the copula-connective grammar above: the ONLY
|
||||
# finite verb admitted is ``overlaps`` (``overlaps_event``). The other connectives
|
||||
# (before / after / during / inside / adjacent …) are UNCHANGED — they still REQUIRE
|
||||
# the copula, so "Monday before Friday." stays a refusal (no connective bypass).
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
#: Declarative finite-verb token: ``<A> overlaps <B>``.
|
||||
_FINITE_VERB_DECLARATIVE = "overlaps"
|
||||
#: Interrogative finite-verb token (base form, with the ``does`` auxiliary):
|
||||
#: ``Does <A> overlap <B>?``.
|
||||
_FINITE_VERB_INTERROGATIVE = "overlap"
|
||||
#: The pack lemma the finite-verb surface mints. Already in ``RELATIONAL_PREDICATES`` via
|
||||
#: the ``("overlaps",)`` connective entry, so the predicate universe is UNCHANGED.
|
||||
_FINITE_VERB_LEMMA = "overlaps_event"
|
||||
|
||||
#: Adverb / sequencing tokens that must NEVER be absorbed into a finite-verb argument
|
||||
#: slot. A finite-verb GUARD, not a general grammar: an adverb-modified or sequenced
|
||||
#: overlap REFUSES rather than fabricate an entity (``a_nearly``, ``b_then_c``). These are
|
||||
#: NOT in ``reader._RESERVED``, so ``_chunk`` would otherwise silently absorb them — the
|
||||
#: exact wrong=0 hazard the adversarial audit flagged.
|
||||
_FINITE_VERB_MODIFIERS = frozenset(
|
||||
{"nearly", "completely", "partially", "mostly", "barely", "then"}
|
||||
)
|
||||
|
||||
|
||||
def load_relational_pack_lemmas() -> frozenset[str]:
|
||||
"""The lemma set of the loaded relational-predicates pack — the fail-closed
|
||||
|
|
@ -264,6 +295,88 @@ def comprehend_relational(
|
|||
return Comprehension(meaning_graph=graph, queries=tuple(queries))
|
||||
|
||||
|
||||
def _read_finite_verb_clause(
|
||||
toks: list[str],
|
||||
question: bool,
|
||||
pack_lemmas: frozenset[str],
|
||||
span: MeaningSpan,
|
||||
claim,
|
||||
relations: list[tuple[str, tuple[str, ...], MeaningSpan, bool]],
|
||||
queries: list[Query],
|
||||
) -> bool:
|
||||
"""Read the closed finite-verb surface (``overlaps_event`` ONLY), or return ``False``
|
||||
when the clause is NOT a finite-verb form (the caller then tries the copula grammar).
|
||||
|
||||
Two surfaces: declarative ``<A> overlaps <B>`` and interrogative
|
||||
``Does <A> overlap <B>?`` (base form + the ``does`` auxiliary). Once a clause IS
|
||||
recognized as a finite-verb form it EMITS or REFUSES — it never falls through, so a
|
||||
malformed overlap cannot reach the copula grammar. Fail-closed: emits only when the
|
||||
lemma is in the loaded pack AND each argument slot is, after article stripping, EXACTLY
|
||||
ONE content token. That single-token gate is the load-bearing backstop — it closes the
|
||||
UNBOUNDED adverb / negation / trailing-qualifier / second-verb class a blocklist
|
||||
cannot (so ``meeting never overlaps lunch`` does NOT become a positive
|
||||
``overlaps_event(meeting_never, lunch)``). The ``_FINITE_VERB_MODIFIERS`` and
|
||||
``_CONNECTIVE_TOKENS`` checks add precise refusal reasons for the common bare-modifier
|
||||
/ second-verb slots; ``_chunk`` then validates the token. Scoped to ``overlaps`` only,
|
||||
so the other connectives keep requiring the copula.
|
||||
"""
|
||||
if question:
|
||||
# interrogative ``does <A> overlap <B>`` (the '?' terminator is already stripped).
|
||||
if not toks or toks[0] != "does" or _FINITE_VERB_INTERROGATIVE not in toks[1:]:
|
||||
return False # not the finite-verb query form — fall through
|
||||
verb_idx = toks.index(_FINITE_VERB_INTERROGATIVE, 1)
|
||||
left, right = toks[1:verb_idx], toks[verb_idx + 1 :]
|
||||
else:
|
||||
# declarative ``<A> overlaps <B>``.
|
||||
if _FINITE_VERB_DECLARATIVE not in toks:
|
||||
return False # not the finite-verb declarative form — fall through
|
||||
verb_idx = toks.index(_FINITE_VERB_DECLARATIVE)
|
||||
left, right = toks[:verb_idx], toks[verb_idx + 1 :]
|
||||
|
||||
# From here the clause IS a finite-verb form — emit or REFUSE, never fall through.
|
||||
if _FINITE_VERB_LEMMA not in pack_lemmas:
|
||||
raise _Reject("relational_lemma_not_in_pack", _FINITE_VERB_LEMMA)
|
||||
|
||||
subject_toks = _strip_article_edges(left)
|
||||
object_toks = _strip_article_edges(right)
|
||||
if not subject_toks or not object_toks:
|
||||
raise _Reject("incomplete_relation", " ".join(toks))
|
||||
|
||||
slot_toks = (*subject_toks, *object_toks)
|
||||
# adverb / sequencing guard — a precise refusal for the common modifiers when one is a
|
||||
# BARE slot ("Nearly overlaps dawn."); these are NOT in reader._RESERVED.
|
||||
if any(t in _FINITE_VERB_MODIFIERS for t in slot_toks):
|
||||
raise _Reject("finite_verb_modifier", " ".join(toks))
|
||||
# leftover relational structure (a second finite verb / any connective token), reusing
|
||||
# the copula path's fabrication net: "A overlaps B overlaps C", trailing qualifiers.
|
||||
if any(t in _CONNECTIVE_TOKENS for t in slot_toks):
|
||||
raise _Reject("extra_relational_structure", " ".join(toks))
|
||||
|
||||
# POSITIVE fail-closed slot gate — the backstop a blocklist cannot provide. Each
|
||||
# finite-verb slot must be EXACTLY ONE content token (after article stripping). An
|
||||
# enumerated adverb/negation list is UNBOUNDED and leaks ("almost", "sometimes",
|
||||
# "never", a trailing qualifier "lunch today", a second verb "overlap"): the
|
||||
# adversarial audit found each glued silently into a fabricated id (e.g.
|
||||
# ``overlaps_event(meeting_never, lunch)`` — a negated sentence committed as a POSITIVE
|
||||
# belief). Requiring a single token closes the whole class — any extra token leaves a
|
||||
# 2+-token slot and REFUSES. Multi-word entities in the finite-verb surface are
|
||||
# deferred (they need a positive content lexicon, which OOV entities preclude).
|
||||
if len(subject_toks) != 1 or len(object_toks) != 1:
|
||||
raise _Reject("finite_verb_unclean_slot", " ".join(toks))
|
||||
|
||||
# _chunk canonicalizes and validates the single token (rejects a non-identifier).
|
||||
subject = _chunk(subject_toks, " ".join(toks))
|
||||
obj = _chunk(object_toks, " ".join(toks))
|
||||
claim(subject, span)
|
||||
claim(obj, span)
|
||||
|
||||
if question:
|
||||
queries.append(Query(_FINITE_VERB_LEMMA, (subject, obj), span))
|
||||
else:
|
||||
relations.append((_FINITE_VERB_LEMMA, (subject, obj), span, False))
|
||||
return True
|
||||
|
||||
|
||||
def _read_relational_clause(
|
||||
clause: str,
|
||||
question: bool,
|
||||
|
|
@ -273,11 +386,22 @@ def _read_relational_clause(
|
|||
relations: list[tuple[str, tuple[str, ...], MeaningSpan, bool]],
|
||||
queries: list[Query],
|
||||
) -> None:
|
||||
"""Read one ``<A> is [the] <connective> <B>`` clause; mutate accumulators or REFUSE."""
|
||||
"""Read one clause into accumulators or REFUSE. Two SEPARATE branches: the closed
|
||||
finite-verb surface (``<A> overlaps <B>`` / ``Does <A> overlap <B>?``) is tried first;
|
||||
everything else falls to the copula-connective grammar (``<A> is [the] <connective>
|
||||
<B>``), which is unchanged — the other connectives still require the copula."""
|
||||
toks = clause.strip().lower().split()
|
||||
if not toks:
|
||||
raise _Reject("empty_clause", clause)
|
||||
|
||||
# B3 finite-verb branch (closed: overlaps_event only). Handles the no-copula
|
||||
# finite-verb surfaces with strict fail-closed guards, and returns False (fall
|
||||
# through to the copula grammar below) ONLY when the clause is not a finite-verb form.
|
||||
if _read_finite_verb_clause(
|
||||
toks, question, pack_lemmas, span, claim, relations, queries
|
||||
):
|
||||
return
|
||||
|
||||
split = _split_around_connective(toks)
|
||||
if split is None:
|
||||
raise _Reject("no_relational_template", clause)
|
||||
|
|
|
|||
196
tests/test_relational_finite_verb.py
Normal file
196
tests/test_relational_finite_verb.py
Normal file
|
|
@ -0,0 +1,196 @@
|
|||
"""Relational reader — the closed finite-verb surface (B3): ``overlaps_event``.
|
||||
|
||||
The reader admits ONE finite verb with NO copula: ``<A> overlaps <B>`` (declarative) and
|
||||
``Does <A> overlap <B>?`` (query). Everything else still requires the copula. Fail-closed:
|
||||
an adverb-modified overlap, a coordinated/leftover slot, a second verb, a negation, a
|
||||
non-table verb, or a bare connective-without-copula all REFUSE.
|
||||
|
||||
Two adversarial-audit hazards are pinned here:
|
||||
1. adverb absorption — ``A nearly overlaps B`` must NOT fabricate the entity ``a_nearly``;
|
||||
2. copula bypass — admitting the finite verb must NOT let the OTHER connectives drop the
|
||||
copula: ``Monday before Friday.`` must still refuse.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from chat.runtime import ChatRuntime
|
||||
from generate.determine import Determined, Undetermined, determine
|
||||
from generate.meaning_graph.reader import Comprehension, Refusal
|
||||
from generate.meaning_graph.relational import (
|
||||
comprehend_relational,
|
||||
load_relational_pack_lemmas,
|
||||
)
|
||||
from generate.realize import NotRealized, realize_comprehension
|
||||
from session.context import SessionContext
|
||||
|
||||
_HIGH = 10**9
|
||||
_PACK = load_relational_pack_lemmas()
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def vocab_persona():
|
||||
rt = ChatRuntime(no_load_state=True)
|
||||
return rt._context.vocab, rt._context.persona
|
||||
|
||||
|
||||
def _ctx(vocab_persona) -> SessionContext:
|
||||
vocab, persona = vocab_persona
|
||||
return SessionContext(vocab=vocab, persona=persona, vault_reproject_interval=_HIGH)
|
||||
|
||||
|
||||
def _read(text: str):
|
||||
return comprehend_relational(text, _PACK)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Positives — the finite-verb declarative + interrogative read overlaps_event
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def test_declarative_overlaps_reads() -> None:
|
||||
comp = _read("Sunrise overlaps dawn.")
|
||||
assert isinstance(comp, Comprehension), comp
|
||||
assert not comp.queries and len(comp.meaning_graph.relations) == 1
|
||||
rel = comp.meaning_graph.relations[0]
|
||||
assert rel.predicate == "overlaps_event"
|
||||
assert rel.arguments == ("sunrise", "dawn")
|
||||
assert rel.negated is False
|
||||
|
||||
|
||||
def test_interrogative_does_overlap_reads_a_query() -> None:
|
||||
comp = _read("Does Sunrise overlap dawn?")
|
||||
assert isinstance(comp, Comprehension), comp
|
||||
assert not comp.meaning_graph.relations and len(comp.queries) == 1
|
||||
q = comp.queries[0]
|
||||
assert q.predicate == "overlaps_event" and q.arguments == ("sunrise", "dawn")
|
||||
assert not q.negated
|
||||
|
||||
|
||||
def test_oov_arguments_in_finite_verb_surface() -> None:
|
||||
rel = _read("Zorptak overlaps Quxley.").meaning_graph.relations[0]
|
||||
assert rel.predicate == "overlaps_event" and rel.arguments == ("zorptak", "quxley")
|
||||
|
||||
|
||||
def test_read_then_determine_direct_and_symmetric(vocab_persona) -> None:
|
||||
# the finite-verb read integrates with determine: direct, and the symmetric converse
|
||||
# (overlaps_event is pack-declared symmetric) determines via the one-hop rule.
|
||||
ctx = _ctx(vocab_persona)
|
||||
realize_comprehension(_read("Sunrise overlaps dawn."), ctx)
|
||||
direct = determine(_read("Does Sunrise overlap dawn?"), ctx)
|
||||
assert isinstance(direct, Determined) and direct.answer is True and direct.rule == "direct"
|
||||
converse = determine(_read("Does dawn overlap Sunrise?"), ctx)
|
||||
assert isinstance(converse, Determined) and converse.answer is True
|
||||
assert converse.rule == "symmetric"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Hazard 1 (pinned) — adverb absorption must NOT fabricate an entity
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def test_adverb_modified_overlap_refuses_no_fabrication(vocab_persona) -> None:
|
||||
res = _read("Alice nearly overlaps Bob.")
|
||||
assert isinstance(res, Refusal) and res.reason == "finite_verb_modifier"
|
||||
# and nothing fabricated is realized — no `alice_nearly` enters memory.
|
||||
ctx = _ctx(vocab_persona)
|
||||
told = realize_comprehension(_read("Alice nearly overlaps Bob."), ctx)
|
||||
assert isinstance(told, NotRealized)
|
||||
assert isinstance(determine(_read("Does alice_nearly overlap Bob?"), ctx), Undetermined)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Hazard 2 (pinned) — the finite verb must NOT let other connectives bypass copula
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text",
|
||||
[
|
||||
"Monday before Friday.",
|
||||
"The cat inside the house.",
|
||||
"The room adjacent the hall.",
|
||||
],
|
||||
)
|
||||
def test_connective_without_copula_still_refuses(text) -> None:
|
||||
res = _read(text)
|
||||
assert isinstance(res, Refusal) and res.reason == "no_relational_template"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Mandatory confusers — every malformed finite-verb / non-finite-verb surface refuses
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text",
|
||||
[
|
||||
"Alice overlaps with the team and Bob.", # coordinated/leftover structure
|
||||
"A overlaps B and C.", # coordination
|
||||
"A nearly overlaps B.", # adverb
|
||||
"A completely overlaps B.", # adverb
|
||||
"Does A nearly overlap B?", # adverb in the query
|
||||
"A does not overlap B.", # negation
|
||||
"A borders B.", # finite verb not in the closed table
|
||||
"A overlaps B overlaps C.", # a second finite verb
|
||||
"A overlaps B then C.", # sequencing modifier
|
||||
],
|
||||
)
|
||||
def test_finite_verb_confusers_refuse(text) -> None:
|
||||
res = _read(text)
|
||||
assert isinstance(res, Refusal), f"{text!r} should refuse but committed {res!r}"
|
||||
|
||||
|
||||
def test_no_finite_verb_confuser_fabricates_a_relation() -> None:
|
||||
# stronger than "is a Refusal": not a single relation/query is produced for any of them.
|
||||
for text in (
|
||||
"Alice overlaps with the team and Bob.",
|
||||
"A nearly overlaps B.",
|
||||
"A overlaps B overlaps C.",
|
||||
"A overlaps B then C.",
|
||||
"A does not overlap B.",
|
||||
):
|
||||
comp = _read(text)
|
||||
assert isinstance(comp, Refusal), text
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Slot-fabrication hazards (adversarial audit) — the single-token slot gate closes
|
||||
# the UNBOUNDED adverb / negation / qualifier / second-verb class a blocklist cannot.
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"text",
|
||||
[
|
||||
"Alice almost overlaps Bob.", # adverb NOT in the enumerated modifier set
|
||||
"Meeting sometimes overlaps lunch.", # another non-enumerated adverb
|
||||
"Sprint overlaps almost lunch.", # right-slot adverb
|
||||
"Sprint overlaps lunch almost.", # trailing right-slot adverb
|
||||
"Meeting overlaps lunch today.", # trailing temporal qualifier glued into the id
|
||||
"North station overlaps south station.", # multi-word entities deferred (narrow v1)
|
||||
"Does A overlap B overlap C?", # interrogative double-verb (base form not in net)
|
||||
"Does Sprint overlap lunch now?", # interrogative trailing adverb
|
||||
],
|
||||
)
|
||||
def test_unclean_finite_verb_slots_refuse(text) -> None:
|
||||
# any slot that is not exactly one content token refuses rather than glue extra tokens
|
||||
# into a fabricated id — this is the backstop the enumerated blocklists cannot provide.
|
||||
res = _read(text)
|
||||
assert isinstance(res, Refusal), f"{text!r} should refuse but committed {res!r}"
|
||||
|
||||
|
||||
def test_never_negation_is_not_committed_as_positive(vocab_persona) -> None:
|
||||
# THE worst class the audit found: 'never' is not a reserved word, so without the
|
||||
# single-token gate "Meeting never overlaps lunch." was committed as a POSITIVE
|
||||
# overlaps_event(meeting_never, lunch). It must refuse and hold no positive belief.
|
||||
res = _read("Meeting never overlaps lunch.")
|
||||
assert isinstance(res, Refusal)
|
||||
ctx = _ctx(vocab_persona)
|
||||
told = realize_comprehension(_read("Meeting never overlaps lunch."), ctx)
|
||||
assert isinstance(told, NotRealized)
|
||||
assert isinstance(determine(_read("Does meeting overlap lunch?"), ctx), Undetermined)
|
||||
assert isinstance(
|
||||
determine(_read("Does meeting_never overlap lunch?"), ctx), Undetermined
|
||||
)
|
||||
|
|
@ -45,14 +45,14 @@ def test_positive_lane_is_green() -> None:
|
|||
report = run()
|
||||
assert report["wrong"] == 0
|
||||
assert report["refused"] == 0 # every clean case reads
|
||||
assert report["correct"] == report["total"] == 17
|
||||
assert report["correct"] == report["total"] == 18 # +1: overlaps_event finite-verb (B3)
|
||||
|
||||
|
||||
def test_lane_is_composed_into_capability_index() -> None:
|
||||
results = {d.domain: d for d in collect_domain_results().results}
|
||||
dom = results.get("comprehension_relational_predicate")
|
||||
assert dom is not None, "relational-predicate domain missing from the index"
|
||||
assert dom.wrong == 0 and dom.correct == 17 and dom.coverage == 1.0
|
||||
assert dom.wrong == 0 and dom.correct == 18 and dom.coverage == 1.0
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
|
@ -63,7 +63,12 @@ def test_lane_is_composed_into_capability_index() -> None:
|
|||
def test_adversarial_inputs_all_refuse() -> None:
|
||||
pack = load_relational_pack_lemmas()
|
||||
cases = _refusal_cases()
|
||||
assert len(cases) >= 9 # the #596 hazard family + negation + non-template + the verb gap
|
||||
# the #596 fabrication family + negation + non-template + the B3 finite-verb confuser
|
||||
# family (adverb absorption incl. non-enumerated 'almost'/'never', coordination,
|
||||
# second-verb, trailing qualifier, connective-without-copula). The overlaps verb-gap
|
||||
# (old ref-009) MIGRATED to a positive, replaced by finite-verb confusers — the refusal
|
||||
# floor RISES, never drops.
|
||||
assert len(cases) >= 24
|
||||
for case in cases:
|
||||
comp = comprehend_relational(case["text"], pack)
|
||||
assert isinstance(comp, Refusal), (
|
||||
|
|
|
|||
Loading…
Reference in a new issue