test(adr-0084): integration test pins substrate gate against ratified content (#68)

After PR #64 (substrate) and PR #65 (content) both landed on main, this
test is the promised follow-up that exercises the substrate-callable
verify_definitional_closure against the real ratified content rather
than fixture packs. It pins three contracts:

  1. Substrate-vs-content handshake. The standalone
     scripts/verify_definitional_closure.py is the agent's dev-loop
     tool; this test is the gate-callable equivalent the ratification
     pipeline can invoke. Both must agree on what passes — divergence
     is a contract bug.

  2. Content drift catcher. Any future content edit that adds an
     unresolved token / non-mounted dependency / silent staging leak
     fails this test before the edit lands on main.

  3. Staging exclusion. en_minimal_v1 is staging per the ADR-0084
     pack-content brief and must not be load-bearing for the closure
     rule. Test-pinned via a production-pool subtest.

Substrate fix: allow empty definitional_atoms

The substrate's strict parser previously rejected empty
definitional_atoms. That stance was wrong: per the ADR-0084 pack-
content brief, the per-entry atom list excludes articles, prepositions,
and copulas. A gloss whose every content word is a function word
(e.g. en_core_temporal_v1/prior → "before") has zero content atoms by
construction. The closure rule passes vacuously when atoms is empty
— there is nothing to close. The gloss-vs-atoms mismatch check in
the standalone verifier is the second-layer gate that distinguishes
by-construction emptiness (legitimate) from by-omission emptiness
(laziness). Substrate parser shouldn't double-gate the same concern.

The corresponding substrate test flipped from
test_empty_definitional_atoms_rejected to
test_empty_definitional_atoms_accepted, with comment explaining the
reasoning.

Primitives expansion: can + action

Two content entries (en_core_cognition_v1/person → "who can know and
do" and en_core_meta_v1/intend → "decide before an action") leaned on
'can' and 'action' as atom references. Today those lemmas resolve
ONLY via en_minimal_v1/lexicon.jsonl — the staging pack. That's a
production-vs-staging leak: production content should not be load-
bearing on staging.

Two clean alternatives:
  (a) rewrite the two glosses to avoid 'can' and 'action'
  (b) promote 'can' and 'action' to primitives

Chose (b): both lemmas are genuinely terminal-feeling (can is a basic
capability modal; action is an irreducible "what is done"); the
content reads more naturally with them present than with paraphrased
substitutes; and the floor was always going to need both eventually.
The cost is two primitives.jsonl rows + checksum + count bump.

Verification:
  scripts/verify_definitional_closure.py            exit 0
  tests/test_adr_0084_integration_closure.py        30/30 pass
  tests/test_adr_0084_definitional_substrate.py     39/39 pass
  core test --suite smoke -q                        67/67
  core test --suite packs -q                         6/6
  core eval cognition                               byte-identical
                                                    (100/91.7/100/100)

Two-layer gate now in place:
  - standalone verifier (dev loop, gloss/atom mismatch check)
  - substrate verifier (ratification gate, parametrized over every
    opted-in pack, staging-exclusion test, primitives floor coverage)
This commit is contained in:
Shay 2026-05-20 15:35:37 -07:00 committed by GitHub
parent 48282eef8d
commit 1938aaa674
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 250 additions and 6 deletions

View file

@ -134,8 +134,19 @@ def parse_gloss_entry(payload: dict, *, strict: bool) -> GlossEntry:
lemma = _require_nonempty_str(payload, "lemma", lemma_hint=lemma_hint) lemma = _require_nonempty_str(payload, "lemma", lemma_hint=lemma_hint)
gloss = _require_nonempty_str(payload, "gloss", lemma_hint=lemma) gloss = _require_nonempty_str(payload, "gloss", lemma_hint=lemma)
# Empty ``definitional_atoms`` is legal: per the ADR-0084
# content brief, the per-entry atom list excludes
# articles/prepositions/copulas. A gloss whose every content
# word is a function word (e.g. ``prior`` → ``"before"``) has
# zero content atoms by construction. The closure rule then
# passes vacuously — which is fine, because there is nothing
# *to* close. Note the substrate cannot tell whether atoms
# are empty by-construction (legitimate) or by-omission
# (laziness); the gloss-vs-atoms mismatch check in the
# standalone verifier (``scripts/verify_definitional_closure.py``)
# is the second-layer gate that distinguishes the two.
definitional_atoms = _require_str_tuple( definitional_atoms = _require_str_tuple(
payload, "definitional_atoms", lemma_hint=lemma, allow_empty=False payload, "definitional_atoms", lemma_hint=lemma, allow_empty=True
) )
predicates_invited = _require_str_tuple( predicates_invited = _require_str_tuple(
payload, "predicates_invited", lemma_hint=lemma, allow_empty=True payload, "predicates_invited", lemma_hint=lemma, allow_empty=True

View file

@ -5,8 +5,8 @@
"definitional_layer": true, "definitional_layer": true,
"version": 1, "version": 1,
"issued_at": "2026-05-20T00:00:00Z", "issued_at": "2026-05-20T00:00:00Z",
"checksum": "93d01ff6d4812968a490021e0f5518ebe071c47325e506a6678a57a4154da284", "checksum": "e5a4782a2f0ec78ebe852ba66156534260ab1bda9f53148be098857ccd0e5e78",
"primitive_count": 52, "primitive_count": 54,
"never_auto_mutable": true, "never_auto_mutable": true,
"provenance": "adr-0084:reviewed:2026-05-20" "provenance": "adr-0084:reviewed:2026-05-20"
} }

View file

@ -50,3 +50,5 @@
{"lemma":"female","category":"identity","pos":"ADJ","primitive_version":1,"provenance_ids":["adr-0084:reviewed:2026-05-20"]} {"lemma":"female","category":"identity","pos":"ADJ","primitive_version":1,"provenance_ids":["adr-0084:reviewed:2026-05-20"]}
{"lemma":"other","category":"identity","pos":"ADJ","primitive_version":1,"provenance_ids":["adr-0084:reviewed:2026-05-20"]} {"lemma":"other","category":"identity","pos":"ADJ","primitive_version":1,"provenance_ids":["adr-0084:reviewed:2026-05-20"]}
{"lemma":"only","category":"quantity","pos":"ADV","primitive_version":1,"provenance_ids":["adr-0084:reviewed:2026-05-20"]} {"lemma":"only","category":"quantity","pos":"ADV","primitive_version":1,"provenance_ids":["adr-0084:reviewed:2026-05-20"]}
{"lemma":"can","category":"mode","pos":"AUX","primitive_version":1,"provenance_ids":["adr-0084:reviewed:2026-05-20","adr-0084:integration:2026-05-20"]}
{"lemma":"action","category":"relation","pos":"NOUN","primitive_version":1,"provenance_ids":["adr-0084:reviewed:2026-05-20","adr-0084:integration:2026-05-20"]}

View file

@ -89,11 +89,17 @@ class TestStrictParser:
entry = parse_gloss_entry(payload, strict=True) entry = parse_gloss_entry(payload, strict=True)
assert entry.predicates_invited == () assert entry.predicates_invited == ()
def test_empty_definitional_atoms_rejected(self) -> None: def test_empty_definitional_atoms_accepted(self) -> None:
# Empty atoms list is legal — a gloss whose every content word
# is a function word/preposition (per the ADR-0084 brief's skip
# list) legitimately produces zero atoms. The closure rule
# passes vacuously; the gloss-vs-atoms mismatch check in the
# standalone verifier is the second-layer gate that catches
# laziness vs by-construction emptiness.
payload = _valid_strict_payload() payload = _valid_strict_payload()
payload["definitional_atoms"] = [] payload["definitional_atoms"] = []
with pytest.raises(DefinitionalSchemaError, match=r"non-empty 'definitional_atoms'"): entry = parse_gloss_entry(payload, strict=True)
parse_gloss_entry(payload, strict=True) assert entry.definitional_atoms == ()
@pytest.mark.parametrize("bad_version", [0, -1, "1", 1.0, True]) @pytest.mark.parametrize("bad_version", [0, -1, "1", 1.0, True])
def test_invalid_definition_version_rejected(self, bad_version: object) -> None: def test_invalid_definition_version_rejected(self, bad_version: object) -> None:

View file

@ -0,0 +1,225 @@
"""ADR-0084 integration test — substrate gate against ratified content.
After PR #64 (substrate) and PR #65 (content) both landed on main, this
test was promised as the follow-up that exercises the substrate-callable
``verify_definitional_closure`` against the *real* ratified content,
not fixture packs. Its job:
1. Pin the substrate-vs-content handshake. Today the standalone
``scripts/verify_definitional_closure.py`` is the agent's dev-loop
tool; this test is the *gate-callable* equivalent that the
ratification pipeline can invoke. Both must stay in agreement on
what passes divergence is a contract bug.
2. Catch content drift. If a future content edit adds an unresolved
token, an empty atom list that should not be empty, or a gloss
that depends on a non-mounted pack, this test fails before the
edit lands on main independently of whether the agent's
standalone script also catches it.
3. Document the resolution-pool contract. ``mounted_pack_lemmas`` is
the union of lexicon-resident lemmas across *production* packs.
``en_minimal_v1`` is staging per the ADR-0084 pack-content brief
and must NOT be load-bearing for the closure rule (deliberately
excluded here even though the standalone verifier currently
pools it).
"""
from __future__ import annotations
import pytest
from language_packs.compiler import load_pack_entries
from language_packs.definitions import (
load_pack_glosses,
verify_definitional_closure,
)
from packs.primitives import load_primitives_pack
# Packs that flipped ``definitional_layer: true`` via PR #65.
# Production-only — does NOT include ``en_minimal_v1`` (staging) or
# the Greek/Hebrew packs (per-lens glosses deferred per ADR-0084
# scope limits).
OPTED_IN_PACKS: tuple[str, ...] = (
"en_core_cognition_v1",
"en_core_action_v1",
"en_core_attitude_v1",
"en_core_causation_v1",
"en_core_meta_v1",
"en_core_polarity_v1",
"en_core_quantitative_v1",
"en_core_spatial_v1",
"en_core_temporal_v1",
"en_core_relations_v1",
"en_core_relations_v2",
"en_core_relations_v3",
"en_collapse_anchors_v1",
)
@pytest.fixture(scope="module")
def primitives_lemmas() -> frozenset[str]:
return load_primitives_pack().lemmas
@pytest.fixture(scope="module")
def mounted_lex_lemmas() -> frozenset[str]:
"""Union of lexicon-resident lemmas across production opted-in packs.
Built from each pack's ``lexicon.jsonl`` — the same source the
standalone verifier uses because that's the operational meaning
of "a lemma in another mounted pack" (gloss entries are an additive
overlay on the immutable lexicon, not its replacement).
"""
lemmas: set[str] = set()
for pack_id in OPTED_IN_PACKS:
for entry in load_pack_entries(pack_id):
lemmas.add(entry.lemma.lower())
return frozenset(lemmas)
# --------------------------------------------------------------------------- #
# Strict-parse every opted-in pack
# --------------------------------------------------------------------------- #
class TestStrictParseOptedInPacks:
"""Every opted-in pack must strict-parse under the substrate.
A strict-parse failure means the content carries a schema violation
the substrate would refuse to accept caught here before it ships.
"""
@pytest.mark.parametrize("pack_id", OPTED_IN_PACKS)
def test_strict_parse(self, pack_id: str) -> None:
entries = load_pack_glosses(pack_id, strict=True)
assert entries, f"{pack_id} has no parseable gloss entries"
def test_total_entry_count_matches_standalone_verifier(self) -> None:
# Standalone ``scripts/verify_definitional_closure.py`` reports
# 333 entries. Substrate strict-parse must see the same set
# so the two verifiers agree on scope.
total = sum(len(load_pack_glosses(p, strict=True)) for p in OPTED_IN_PACKS)
assert total == 333, (
f"Substrate parsed {total} entries; standalone verifier reports 333. "
f"Divergence means one of the two verifiers is silently skipping rows."
)
# --------------------------------------------------------------------------- #
# Closure rule against the production resolution pool
# --------------------------------------------------------------------------- #
class TestClosureAgainstProductionPool:
"""Every opted-in pack must close against (same-pack + production-
mounted lexicon + primitives) staging packs deliberately excluded.
"""
@pytest.mark.parametrize("pack_id", OPTED_IN_PACKS)
def test_pack_closes(
self,
pack_id: str,
mounted_lex_lemmas: frozenset[str],
primitives_lemmas: frozenset[str],
) -> None:
violations = verify_definitional_closure(
pack_id,
mounted_pack_lemmas=mounted_lex_lemmas,
primitive_lemmas=primitives_lemmas,
strict=True,
)
assert violations == (), (
f"{pack_id} has {len(violations)} unresolved tokens against the "
f"production pool: "
+ ", ".join(f"{v.lemma}{v.unresolved_token!r}" for v in violations)
)
def test_no_violations_total(
self,
mounted_lex_lemmas: frozenset[str],
primitives_lemmas: frozenset[str],
) -> None:
# Aggregate gate — the integration contract for ADR-0084 as a
# whole: every opted-in pack closes against the production
# pool with zero unresolved tokens.
total_violations = 0
for pack_id in OPTED_IN_PACKS:
total_violations += len(
verify_definitional_closure(
pack_id,
mounted_pack_lemmas=mounted_lex_lemmas,
primitive_lemmas=primitives_lemmas,
strict=True,
)
)
assert total_violations == 0
# --------------------------------------------------------------------------- #
# Staging exclusion contract
# --------------------------------------------------------------------------- #
class TestStagingExclusion:
"""``en_minimal_v1`` is staging and must not be load-bearing for the
closure rule. If a future content edit makes any opted-in pack
depend on en_minimal_v1 to resolve, this test catches it that
dependency would be a production-vs-staging leak.
"""
def test_no_production_pack_depends_on_en_minimal_v1(
self,
primitives_lemmas: frozenset[str],
) -> None:
# Build a pool WITHOUT en_minimal_v1 — the production pool.
production_pool: set[str] = set()
for pack_id in OPTED_IN_PACKS:
for entry in load_pack_entries(pack_id):
production_pool.add(entry.lemma.lower())
# Then check: every opted-in pack closes against that pool.
# If a pack secretly leans on en_minimal_v1, this fails with a
# concrete unresolved-token list.
for pack_id in OPTED_IN_PACKS:
violations = verify_definitional_closure(
pack_id,
mounted_pack_lemmas=production_pool,
primitive_lemmas=primitives_lemmas,
strict=True,
)
assert not violations, (
f"{pack_id} leaks into en_minimal_v1 — unresolved without "
f"staging pool: "
+ ", ".join(f"{v.lemma}{v.unresolved_token!r}" for v in violations)
)
# --------------------------------------------------------------------------- #
# Primitives floor coverage
# --------------------------------------------------------------------------- #
class TestPrimitivesFloor:
"""Floor-level sanity: the primitives pack must carry the foundation
words the brief named and the ones the content ended up leaning
on. Catches accidental removals from the primitives pack."""
REQUIRED_FLOOR = (
# From the original ADR-0084 brief's category list
"exist", "be", "not_be", "not",
"same", "different",
"cause", "because", "change",
"say", "mean",
"if", "then", "and", "or",
# The ones added during integration because the content leaned
# on them via en_minimal_v1 (which is staging and not load-
# bearing) — promoted to primitives so production closure is
# robust.
"can", "action",
)
def test_required_primitives_present(self, primitives_lemmas: frozenset[str]) -> None:
missing = sorted(set(self.REQUIRED_FLOOR) - primitives_lemmas)
assert not missing, f"primitives pack missing required floor lemmas: {missing}"