6a75f5203a
CMB (r4_combined_rate) now participates in route_setup and bounded contemplation. R1/R2/R3 readers/ solvers, the CMB reader/solver/oracle, and serving are all UNCHANGED — pure shared-infra integration, no new comprehension capability (that was CMB-c). - model.py: Organ += r4_combined_rate. - classify.py: classify_cmb + _cmb_signature (organ-tagged, cross-organ-distinct) + cmb_reason — namespaces CMB reasons cmb_* before the reason-string-keyed registry sees them, so a CMB boundary never inherits R2/R3's family for the SAME bare string (R3 rate_unit_mismatch is a GROWTH surface; R2/R3 non_integer_solution carry other owners). not_combined_rate_shaped/empty stay bare -> the cross input_shape family (router hygiene). - failure_family.py: Owner += r4; input_shape += not_combined_rate_shaped; 8 CMB families (must_remain_refused: cmb_unit_mismatch [until a dimension registry exists — NOT forever], cmb_combine_ambiguous, cmb_underdetermined, cmb_non_positive_net, cmb_non_integer; proposal_allowed -> cmb_gold_fixture: cmb_unsupported_rate_count/_reciprocal/_clock_interval). - pass_manager.py: _solve_and_verify_cmb (solver Refusal -> REFUSED_KNOWN_BOUNDARY, never a proposal). CMB-over-R3 domain-precedence rule (cmb_is_authoritative): when CMB POSITIVELY recognizes combined-rate shape (a setup, or a substantive cmb_* refusal — NOT the input_shape step-aside), R3's broader single-rate over-read of the SAME text is inadmissible — for BOTH routing (veto R3 setup_correct) and family attribution (suppress R3 so CMB's sharper diagnosis owns the terminal/proposal). This came from a pre-build cross-organ pressure-test: R3 reads 'Anna and Ben paint together; Anna paints 3 rooms/hour. How many do THEY paint in 4 hours?' as a single rate and would confidently answer a wrong 12. The rule fixes that (-> REFUSED) without touching R3. It does NOT mean 'CMB always wins': on input_shape CMB cedes (a plain single-rate car problem routes to R3 and solves 180). Narrow CMB<->R3 instance of a future general domain-specificity adjudication. Tests: new test_cmb_router_contemplation (full terminal matrix; cmb-11 veto-no-wrong-answer; cmb-15 cede-to-R3; solver-refusal-never-proposes; CMB-owned proposal-only artifacts mounted:false; no gold ambiguous; off-serving AST). Extended router-organ-hygiene to CMB x R1/R2/R3 (0 breaches). test_setup_router/test_r3_router_contemplation attempt-count 3->4; test_failure_family ALL_REASONS + growth set. 97 CMB-d tests; 597 broad-net pass (1 unrelated pre-existing red); CMB organ lanes green.
74 lines
3.4 KiB
Python
74 lines
3.4 KiB
Python
"""Tests for wiring R3 into the contemplation router + pass manager (R3.1).
|
|
|
|
Pins the R3.1c matrix: supported R3 → SOLVED_VERIFIED; rate_unit_mismatch/combined → proposal-only
|
|
unsupported_rate_duration; temporal_state/missing/non-integer → REFUSED_KNOWN_BOUNDARY (no proposal);
|
|
R1/R2 unaffected; non-rate text never blocks an R2 proposal.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
|
|
from core.comprehension_attempt import classify_r3, route_setup
|
|
from evals.constraint_oracle.runner import _load_r2_gold
|
|
from evals.rate_oracle.runner import _load_rate_gold
|
|
from evals.setup_oracle.runner import _load_r1_gold
|
|
from generate.contemplation import Terminal, contemplate
|
|
|
|
|
|
def test_classify_r3_matches_gold_expect() -> None:
|
|
for fx in _load_rate_gold():
|
|
att = classify_r3(fx["text"], case_id=fx["id"])
|
|
assert att.organ == "r3_rate"
|
|
if fx["expect"] in ("solved", "solver_refuses"):
|
|
assert att.outcome == "setup_correct" and att.setup_signature is not None
|
|
else:
|
|
assert att.outcome == "setup_refused" and att.refusal_reason == fx["reader_reason"]
|
|
|
|
|
|
def test_router_routes_rate_to_r3_and_stays_exclusive() -> None:
|
|
routed = 0
|
|
for fx in _load_rate_gold():
|
|
r = route_setup(fx["text"])
|
|
assert len(r.attempts) == 4 and r.status != "ambiguous"
|
|
if r.selected is not None:
|
|
assert r.selected.organ == "r3_rate"
|
|
routed += 1
|
|
assert routed == 9 # 7 solved (incl. convertible r3-09) + 2 solver_refuses
|
|
# adding R3 does not make any R1/R2 problem ambiguous, nor route it to r3
|
|
for fx in _load_r1_gold() + _load_r2_gold():
|
|
r = route_setup(fx["text"])
|
|
assert r.status != "ambiguous"
|
|
if r.selected is not None:
|
|
assert r.selected.organ != "r3_rate"
|
|
|
|
|
|
def _expected_r3_terminal(fx: dict) -> Terminal:
|
|
if fx["expect"] == "solved":
|
|
return Terminal.SOLVED_VERIFIED
|
|
if fx["expect"] == "solver_refuses":
|
|
return Terminal.REFUSED_KNOWN_BOUNDARY
|
|
if fx["reader_reason"] in ("rate_unit_mismatch", "combined_rates"):
|
|
return Terminal.PROPOSAL_EMITTED
|
|
return Terminal.REFUSED_KNOWN_BOUNDARY # missing_time / temporal_state
|
|
|
|
|
|
def test_contemplation_r3_terminals_and_only_rate_like_propose(tmp_path: Path) -> None:
|
|
for fx in _load_rate_gold():
|
|
kwargs = {"options": fx["options"], "answer_key": fx["answer"]} if fx["expect"] == "solved" else {}
|
|
result = contemplate(fx["text"], proposal_root=tmp_path, case_id=fx["id"], **kwargs)
|
|
assert result.terminal == _expected_r3_terminal(fx), f"{fx['id']}: {result.terminal}"
|
|
if fx["expect"] == "solved":
|
|
assert result.answer == fx["gold"]
|
|
if result.terminal == Terminal.PROPOSAL_EMITTED:
|
|
assert result.family == "unsupported_rate_duration"
|
|
# ONLY the two rate-like unsupported features proposed; temporal/missing/non-integer did not.
|
|
assert len(list(tmp_path.glob("*.json"))) == 2
|
|
|
|
|
|
def test_non_rate_text_does_not_block_r2_proposal(tmp_path: Path) -> None:
|
|
# r2-011 (missing_total_count) must still PROPOSAL_EMITTED — R3's not_rate_shaped refusal on
|
|
# this non-rate text is input_shape (not-my-domain), never a substantive boundary.
|
|
fx = next(f for f in _load_r2_gold() if f["id"] == "r2-011-missing-total-count")
|
|
result = contemplate(fx["text"], proposal_root=tmp_path)
|
|
assert result.terminal == Terminal.PROPOSAL_EMITTED and result.family == "missing_total_count"
|