97b0ee0e13
## Summary
Two test failures on origin/main both trace to PR #315 (ADR-0163.D.2 —
discrete_count_statement recognizer + admissibility-intent chain). Earlier
runs treated them as "pre-existing unrelated" — they are not unrelated.
The first is a real wrong>0 hazard.
## Failure 1: silent admission via recognized-but-uninjectable statement
The ratified `discrete_count_statement` recognizer over-matches: ANY
sentence containing a number + noun resolves it, irrespective of the verb.
When `inject_from_match` returns `()` (the round-2 default for v1
categories without an injector), the old code path used `continue` to
silently drop the statement — and the solver then answered from whatever
initial state remained.
Reproduction:
parse_and_solve("Sam has 5 apples. Sam contemplates 3 apples. "
"How many apples does Sam have?")
→ is_admitted=True, answer=5.0 (silent admission of partial graph)
This is exactly the case-0050-class hazard wearing a different hat
(silently admitting an incomplete graph at the problem level).
ADR-0167 / Brief 11 §"correct-count greed" established the principle on
the reader path; this commit extends it to the recognizer path.
Fix: when a recognizer matches but produces no injection, REFUSE.
generate/math_candidate_graph.py:
- Replaced the skip-only `continue` with a CandidateGraphResult
refusal carrying the recognizer category in the reason.
tests/test_math_candidate_graph.py:
- test_unparseable_statement now accepts either the legacy
"no admissible candidate" reason or the new
"recognizer matched but produced no injection" reason.
Both legitimately refuse; what matters is is_admitted=False.
tests/test_recognizer_skip_wrong_zero.py (NEW):
- 5 regression tests pinning the wrong=0 invariant:
* 3 parametrized verbs unknown to both regex parser and reader
(contemplates / ponders / memorises) — must all refuse
* Nonsense token — must refuse
* Anti-regression: known initial + known operation still admits
## Failure 2: cognition audit drop-reason taxonomy
The audit test hardcoded `dropped.reason.startswith("superseded_by:")`
as the only valid drop-reason prefix. Commit da70919 (ADR-0163.D.2)
ratified an admissibility-intent chain that the audit categorizes with
reason `unsupported_intent:admissibility`, which fails this assertion.
Fix: tests/test_teaching_audit.py — expand the allowed-prefix set to
include `unsupported_intent:` with a written rationale. Future drop
classes extend the allowlist deliberately rather than silently
broadening the assertion to any non-empty reason.
## Surfaced regression: partition-test allowlist (ADR-0167 FOLLOWUPS §2)
This PR modifies three test files that the
test_existing_cognition_tests_untouched assertion would reject under
its named-allowlist scheme. Added the three test paths to the allowlist
as the tactical fix; the architectural fix (retire / move to CI / move
to CODEOWNERS) is queued in docs/handoff/ADR-0167-FOLLOWUPS.md §2.
## Test plan
uv run pytest tests/test_recognizer_skip_wrong_zero.py \
tests/test_math_candidate_graph.py \
tests/test_teaching_audit.py \
tests/test_candidate_domain_partition.py \
tests/test_math_evidence_e2e.py \
tests/test_math_evidence_schema.py \
tests/test_math_contemplation_adapter.py \
tests/test_math_claim_signature.py \
tests/test_math_lexical_ratification.py \
tests/test_brief_11b_audit_artifact.py \
tests/test_brief_11b_step2_lexicon.py \
tests/test_brief_11_audit.py
→ 152 passed
## Hard invariants
- wrong == 0 — restored on the recognizer path (was silently violated on main)
- ADR-0166 — no new eval lanes
- No teaching-store mutation, no pack mutation
- The reader path was already correct (it refused these cases); this fix
brings the regex/recognizer path back in line
104 lines
3.6 KiB
Python
104 lines
3.6 KiB
Python
"""Tests for W2-C — candidate domain partition."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import subprocess
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
from core.protocol import canonical_bytes
|
|
from teaching.discovery import DiscoveryCandidate
|
|
|
|
|
|
def _candidate(*, domain: str | None = None) -> DiscoveryCandidate:
|
|
kwargs: dict[str, Any] = {
|
|
"candidate_id": "cand_1",
|
|
"proposed_chain": {
|
|
"subject": "wisdom",
|
|
"intent": "cause",
|
|
"connective": None,
|
|
"object": None,
|
|
},
|
|
"trigger": "would_have_grounded",
|
|
"source_turn_trace": "t1",
|
|
"pack_consistent": True,
|
|
"boundary_clean": True,
|
|
}
|
|
if domain is not None:
|
|
kwargs["domain"] = domain
|
|
return DiscoveryCandidate(**kwargs)
|
|
|
|
|
|
def test_default_domain_is_cognition():
|
|
"""Bare DiscoveryCandidate has domain == 'cognition'."""
|
|
cand = _candidate()
|
|
assert cand.domain == "cognition"
|
|
|
|
|
|
def test_math_domain_can_be_set():
|
|
"""Explicit domain='math' constructs successfully."""
|
|
cand = _candidate(domain="math")
|
|
assert cand.domain == "math"
|
|
|
|
|
|
def test_round_trip_preserves_domain():
|
|
"""Serialization/deserialization preserves the domain field."""
|
|
# Cognition domain
|
|
cand_cog = _candidate(domain="cognition")
|
|
dict_cog = cand_cog.as_dict()
|
|
assert "domain" not in dict_cog # Omitted to preserve legacy keys
|
|
roundtrip_cog = DiscoveryCandidate.from_dict(dict_cog)
|
|
assert roundtrip_cog.domain == "cognition"
|
|
|
|
# Math domain
|
|
cand_math = _candidate(domain="math")
|
|
dict_math = cand_math.as_dict()
|
|
assert dict_math["domain"] == "math" # Included for math
|
|
roundtrip_math = DiscoveryCandidate.from_dict(dict_math)
|
|
assert roundtrip_math.domain == "math"
|
|
|
|
|
|
def test_canonical_bytes_includes_domain_deterministically():
|
|
"""Same domain value -> same canonical bytes contribution."""
|
|
cand_math_1 = _candidate(domain="math")
|
|
cand_math_2 = _candidate(domain="math")
|
|
bytes_1 = canonical_bytes(cand_math_1)
|
|
bytes_2 = canonical_bytes(cand_math_2)
|
|
assert bytes_1 == bytes_2
|
|
|
|
cand_cog = _candidate(domain="cognition")
|
|
bytes_cog = canonical_bytes(cand_cog)
|
|
assert bytes_cog != bytes_1
|
|
assert b'"domain":"math"' in bytes_1
|
|
assert b'"domain":"cognition"' in bytes_cog
|
|
|
|
|
|
def test_existing_cognition_tests_untouched():
|
|
"""Assert zero modifications to pre-existing cognition test files."""
|
|
# Find modified or untracked test files in git status
|
|
result = subprocess.run(
|
|
["git", "status", "--porcelain", "tests/"],
|
|
capture_output=True,
|
|
text=True,
|
|
check=True,
|
|
)
|
|
lines = [line.strip() for line in result.stdout.splitlines() if line.strip()]
|
|
# Allowlist of new/modified test files contributed by ADR-0167 PRs.
|
|
# Future ADR-0167 PRs append their own file here so this regression net
|
|
# stays meaningful as the wave progresses.
|
|
allowed = {
|
|
"test_candidate_domain_partition.py", # W2-C
|
|
"test_math_evidence_e2e.py", # W3-A
|
|
# Fix PR — wrong=0 hazard regression (recognizer skip-only fallback).
|
|
# Modifies test_math_candidate_graph.py and test_teaching_audit.py;
|
|
# adds test_recognizer_skip_wrong_zero.py. See ADR-0167-FOLLOWUPS §2
|
|
# for the architectural fix that would retire this allowlist.
|
|
"test_math_candidate_graph.py",
|
|
"test_teaching_audit.py",
|
|
"test_recognizer_skip_wrong_zero.py",
|
|
}
|
|
for line in lines:
|
|
path = line.split()[-1]
|
|
assert Path(path).name in allowed, (
|
|
f"unexpected new/modified test file: {path}"
|
|
)
|