0be18ebdb9
Add a closed finite-verb relational surface to comprehend_relational, narrowly: declarative <A> overlaps <B>, interrogative Does <A> overlap <B>?, predicate overlaps_event ONLY (closed finite-verb table, default-off). Kept SEPARATE from the copula-connective grammar (byte-unchanged): the other connectives (before/after/during/inside/adjacent/…) still REQUIRE the copula, so "Monday before Friday." stays a refusal — no connective bypass (adversarial hazard #2). Fail-closed slot gate: each finite-verb argument slot must be EXACTLY ONE content token (after article stripping). An enumerated adverb/negation blocklist is unbounded and leaks — the adversarial audit found "Meeting never overlaps lunch." committed as a POSITIVE overlaps_event(meeting_never, lunch), plus almost/sometimes/trailing-qualifier/ second-verb compound fabrications in BOTH slots and the query path. The single-token gate closes the whole COMPOUND-fabrication class (any extra token refuses); the _FINITE_VERB_MODIFIERS and _CONNECTIVE_TOKENS checks add precise reasons for common bare-modifier / second-verb slots. Multi-word entities in the finite-verb surface are deferred (they need a positive content lexicon, which OOV entities preclude). A lone bare token remains an entity (the reader's universal OOV single-token contract — the copula and general readers behave identically), not a compound fabrication. Hazards pinned (adversarial audit hazard #1 + #2): adverb absorption, negation-as- positive ('never'), interrogative double-verb, trailing qualifier, and the copula-bypass firewall. Measurement: migrated ref-009 "Sunrise overlaps dawn." (the documented coverage gap) to a positive rel-018 — comprehension_relational_predicate 17->18, wrong_total 0, breadth still 11 (coverage added, not a new domain), baseline digest re-frozen. Refusal floor rose 9 -> 24 (finite-verb confusers replace the one migrated input). The #775 inference and #781 transitive lane SHA pins are UNTOUCHED. determine.py is untouched (reader-only): no answer=False, no FrameVerdict, INV-30 unaffected. Verified in the worktree: finite-verb + reader + reader-lane + #775 inference lane + #781 transitive lane + #779 OWA floor + capability baseline/index + INV-30/29/21 + full smoke — all green. Two read-only adversarial passes: the first found 4 real slot- fabrication blockers, closed by the single-token gate; a black-box probe of 24 adversarial inputs confirms the compound-fabrication class is fully closed.
102 lines
2.1 KiB
JSON
102 lines
2.1 KiB
JSON
{
|
|
"capability_score": 0.953968,
|
|
"coverage_geomean": 0.953968,
|
|
"coverage_micro": 0.99399,
|
|
"accuracy_micro": 1.0,
|
|
"breadth": 11,
|
|
"min_domain_coverage": 0.833333,
|
|
"wrong_total": 0,
|
|
"assert_mode_valid": true,
|
|
"deterministic_digest": "85c873626423c6b151a7d61bebb0cd5b7caff7dcc7e897188609e5ce3313feac",
|
|
"domains": [
|
|
{
|
|
"domain": "comprehension_propositional",
|
|
"correct": 12,
|
|
"wrong": 0,
|
|
"refused": 0,
|
|
"coverage": 1.0,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "comprehension_relational_inference",
|
|
"correct": 13,
|
|
"wrong": 0,
|
|
"refused": 0,
|
|
"coverage": 1.0,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "comprehension_relational_metric",
|
|
"correct": 15,
|
|
"wrong": 0,
|
|
"refused": 0,
|
|
"coverage": 1.0,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "comprehension_relational_predicate",
|
|
"correct": 18,
|
|
"wrong": 0,
|
|
"refused": 0,
|
|
"coverage": 1.0,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "comprehension_relational_transitive",
|
|
"correct": 7,
|
|
"wrong": 0,
|
|
"refused": 0,
|
|
"coverage": 1.0,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "comprehension_set_membership",
|
|
"correct": 8,
|
|
"wrong": 0,
|
|
"refused": 0,
|
|
"coverage": 1.0,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "comprehension_syllogism",
|
|
"correct": 7,
|
|
"wrong": 0,
|
|
"refused": 1,
|
|
"coverage": 0.875,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "comprehension_total_ordering",
|
|
"correct": 7,
|
|
"wrong": 0,
|
|
"refused": 1,
|
|
"coverage": 0.875,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "deductive_logic",
|
|
"correct": 716,
|
|
"wrong": 0,
|
|
"refused": 0,
|
|
"coverage": 1.0,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "dimensional",
|
|
"correct": 10,
|
|
"wrong": 0,
|
|
"refused": 2,
|
|
"coverage": 0.833333,
|
|
"accuracy": 1.0
|
|
},
|
|
{
|
|
"domain": "relational_metric",
|
|
"correct": 14,
|
|
"wrong": 0,
|
|
"refused": 1,
|
|
"coverage": 0.933333,
|
|
"accuracy": 1.0
|
|
}
|
|
],
|
|
"not_covered": []
|
|
}
|