6a75f5203a
CMB (r4_combined_rate) now participates in route_setup and bounded contemplation. R1/R2/R3 readers/ solvers, the CMB reader/solver/oracle, and serving are all UNCHANGED — pure shared-infra integration, no new comprehension capability (that was CMB-c). - model.py: Organ += r4_combined_rate. - classify.py: classify_cmb + _cmb_signature (organ-tagged, cross-organ-distinct) + cmb_reason — namespaces CMB reasons cmb_* before the reason-string-keyed registry sees them, so a CMB boundary never inherits R2/R3's family for the SAME bare string (R3 rate_unit_mismatch is a GROWTH surface; R2/R3 non_integer_solution carry other owners). not_combined_rate_shaped/empty stay bare -> the cross input_shape family (router hygiene). - failure_family.py: Owner += r4; input_shape += not_combined_rate_shaped; 8 CMB families (must_remain_refused: cmb_unit_mismatch [until a dimension registry exists — NOT forever], cmb_combine_ambiguous, cmb_underdetermined, cmb_non_positive_net, cmb_non_integer; proposal_allowed -> cmb_gold_fixture: cmb_unsupported_rate_count/_reciprocal/_clock_interval). - pass_manager.py: _solve_and_verify_cmb (solver Refusal -> REFUSED_KNOWN_BOUNDARY, never a proposal). CMB-over-R3 domain-precedence rule (cmb_is_authoritative): when CMB POSITIVELY recognizes combined-rate shape (a setup, or a substantive cmb_* refusal — NOT the input_shape step-aside), R3's broader single-rate over-read of the SAME text is inadmissible — for BOTH routing (veto R3 setup_correct) and family attribution (suppress R3 so CMB's sharper diagnosis owns the terminal/proposal). This came from a pre-build cross-organ pressure-test: R3 reads 'Anna and Ben paint together; Anna paints 3 rooms/hour. How many do THEY paint in 4 hours?' as a single rate and would confidently answer a wrong 12. The rule fixes that (-> REFUSED) without touching R3. It does NOT mean 'CMB always wins': on input_shape CMB cedes (a plain single-rate car problem routes to R3 and solves 180). Narrow CMB<->R3 instance of a future general domain-specificity adjudication. Tests: new test_cmb_router_contemplation (full terminal matrix; cmb-11 veto-no-wrong-answer; cmb-15 cede-to-R3; solver-refusal-never-proposes; CMB-owned proposal-only artifacts mounted:false; no gold ambiguous; off-serving AST). Extended router-organ-hygiene to CMB x R1/R2/R3 (0 breaches). test_setup_router/test_r3_router_contemplation attempt-count 3->4; test_failure_family ALL_REASONS + growth set. 97 CMB-d tests; 597 broad-net pass (1 unrelated pre-existing red); CMB organ lanes green.
52 lines
2.3 KiB
Python
52 lines
2.3 KiB
Python
"""Tests for the deterministic multi-organ setup router (N3).
|
|
|
|
Pins that each problem routes to exactly the organ that owns it (R1 relational, R2 constraint),
|
|
that unreadable problems refuse, that no gold problem is read by both organs (no ambiguity), and
|
|
— the wrong=0 invariant — that every routed R2 setup matches the independent gold signature.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from core.comprehension_attempt import route_setup
|
|
from core.comprehension_attempt.classify import _r2_signature
|
|
from evals.constraint_oracle.runner import _load_r2_gold, gold_to_problem
|
|
from evals.setup_oracle.runner import _load_r1_gold
|
|
|
|
|
|
def test_r2_well_formed_routes_to_r2_and_matches_gold_signature() -> None:
|
|
for fx in _load_r2_gold():
|
|
result = route_setup(fx["text"], case_id=fx["id"])
|
|
if fx["expect"] in ("solved", "solver_refuses"):
|
|
assert result.status == "routed", f"{fx['id']}: {result.status}"
|
|
assert result.selected is not None and result.selected.organ == "r2_constraints"
|
|
# wrong=0: the routed setup is byte-equal to the independent gold setup.
|
|
assert result.selected.setup_signature == _r2_signature(gold_to_problem(fx)), fx["id"]
|
|
else: # reader_refuses
|
|
assert result.status == "all_refused" and result.selected is None
|
|
|
|
|
|
def test_r1_admitted_routes_to_r1_refused_all_refused() -> None:
|
|
routed = refused = 0
|
|
for fx in _load_r1_gold():
|
|
result = route_setup(fx["text"], case_id=fx["id"])
|
|
if result.status == "routed":
|
|
routed += 1
|
|
assert result.selected.organ == "r1_quantitative", fx["id"]
|
|
else:
|
|
refused += 1
|
|
assert result.status == "all_refused"
|
|
assert routed == 7 and refused == 3
|
|
|
|
|
|
def test_no_gold_problem_is_ambiguous() -> None:
|
|
# Each organ is exclusive on the gold corpora — no text is admitted by both.
|
|
for fx in _load_r2_gold() + _load_r1_gold():
|
|
assert route_setup(fx["text"]).status != "ambiguous", fx.get("id")
|
|
|
|
|
|
def test_router_never_selects_a_refusal() -> None:
|
|
for fx in _load_r2_gold() + _load_r1_gold():
|
|
result = route_setup(fx["text"])
|
|
if result.selected is not None:
|
|
assert result.selected.outcome == "setup_correct"
|
|
assert len(result.attempts) == 4 # always one R1 + R2 + R3 + R4(combined-rate) attempt
|