feat(gsm8k): capability paradigm sprint 12 singleton contract lift

Add Gate A2r nested_fraction_remainder_total (0004) and Gate A2s
loose_crayon_box_capacity (0007) under explicit ClusterContracts with
hard 0026/0047 neighbor confuser refusal. Serving lifts 24/26/0 to
26/24/0 with wrong=0 preserved; report.json and sealed artifacts
untouched.
This commit is contained in:
Shay 2026-06-18 14:56:10 -07:00
parent 50e8de03fe
commit ae974ce48d
8 changed files with 1078 additions and 5 deletions

View file

@ -0,0 +1,265 @@
# GSM8K Capability Paradigm Sprint 12 — Lookback (2026-06-17)
## 1. Starting baseline
| Metric | Value |
|--------|------:|
| correct | **24** |
| refused | **26** |
| wrong | **0** |
After #824 (Sprint 11 calendar ClusterContract). Prerequisite satisfied.
## 2. Experience Flywheel output summary
`scripts/gsm8k_experience_flywheel.py --limit 50 --out /tmp/gsm8k-experience-sprint12.json`
| Field | Value |
|-------|------:|
| retained_record_count | 50 |
| lift_refused_to_correct | **0** |
| scout serving | 24 / 26 / 0 |
Top singleton signals:
- **0004**`joint_refusal`, `fraction_surface`, `not_promotable`; no sealed-wrong on surface.
- **0007**`joint_refusal`, `unbound_target`, `not_promotable`; no sealed-wrong on surface.
- Blocked families unchanged: `sealed_elimination` (0011, 0026), `relation_hypothesis` (0032, 0047), currency.
- No second calendar sibling under Sprint 11 contract.
## 3. Scout/frontier/microscope output summary
| Regime | correct | refused | wrong |
|--------|--------:|--------:|------:|
| Serving | 24 | 26 | 0 |
| Sealed (resolve_pooled) | 3 | 39 | 8 |
- `lift_refused_to_correct`: **0**
- Dominant bottleneck: joint refusal (20/50)
- Sealed-wrong unchanged: 0011, 0018, 0019, 0025, 0026, 0028, 0032, 0047
Microscope: 0004 tagged `fraction_operand`; 0007 tagged `conditional_question`. Adjacent sealed-wrong neighbors 0026 and 0047 on distinct surfaces.
## 4. Candidate families ranked
| Rank | Candidate | Lift | Wrong-risk neighbor | Verdict |
|------|-----------|------|---------------------|---------|
| 1 | `nested_fraction_remainder_total` (0004) | +1 | 0026 sealed_elimination | **Selected (A2r)** |
| 2 | `loose_crayon_box_capacity` (0007) | +1 | 0047 DCS/divisive | **Selected (A2s)** |
| 3 | calendar contract extension | +0 | — | **Rejected** (no second sibling) |
| — | DCS / currency / sealed_elimination wholesale | — | blocked | **Deferred** |
0004 and 0007 rank equally on flywheel; both admitted with hard neighbor confuser matrices.
## 5. ClusterContracts
### nested_fraction_remainder_contract (Gate A2r)
```yaml
ClusterContract:
family_id: nested_fraction_remainder_total
proposed_organs:
- nested_fraction_remainder_total
included_cases:
- gsm8k-train-sample-v1-0004
explicitly_excluded_cases:
- gsm8k-train-sample-v1-0026 # sealed_elimination / currency / scoops
- gsm8k-train-sample-v1-0011 # sealed_elimination
- asks subgroup only (not root altogether)
- multiple fraction relations
- 3/4 bill-of-money surfaces
positive_anchors:
- "Half of the {population}"
- "1/4 of the {population} going to {camp} camp"
- "{camp} camp in the morning" + afternoon complement count
- question asks root {population} altogether
negative_anchors:
- $ currency / each saved / scoops / bill / ice cream
- 3/4 of total money
- asks soccer-camp subgroup only
- multiple fractions
actor_binding_rule: shared population token across half/quarter/afternoon clauses
target_binding_rule: question must ask root population altogether
unit_binding_rule: afternoon count unit matches bound population
quantity_obligations:
- text-grounded afternoon complement count
- exactly one 1/4 inner partition
allowed_external_grounding: none
blocked_sibling_families:
- sealed_elimination
- currency_amount
- percent_partition
sealed_wrong_neighbors:
- 0026 (blocked by currency/each/scoops/3/4 bill hazards)
serving_admission_rule: resolve_promotable_nested_fraction_remainder_total()
implementation_allowed: true
reason: Typed morning/afternoon complement chain + 0026 confuser matrix clears wrong-risk.
```
### loose_crayon_box_capacity_contract (Gate A2s)
```yaml
ClusterContract:
family_id: loose_crayon_box_capacity
proposed_organs:
- loose_crayon_box_capacity
included_cases:
- gsm8k-train-sample-v1-0007
explicitly_excluded_cases:
- gsm8k-train-sample-v1-0047 # equal-distribution weight / eaten bag
- asks per-box capacity (not more boxes)
- equal number in bags
- weight/ounce surfaces
positive_anchors:
- "{actor} has N full boxes of {item}"
- "M loose {item}" before friend clause
- "friend has K loose"
- conditional "total of T" in question
- "how many more boxes"
negative_anchors:
- ounces / macaroons / equal number / eats / Steve
- asks per-box not more-boxes
- DCS divisive packing
actor_binding_rule: primary actor owns full boxes + loose; friend contributes loose only
target_binding_rule: question asks additional boxes for all loose items
unit_binding_rule: answer unit is boxes
quantity_obligations:
- total, loose_primary, loose_friend, num_full_boxes all text-grounded
allowed_external_grounding: none
blocked_sibling_families:
- multiplicative_aggregate / DCS
- currency_amount
sealed_wrong_neighbors:
- 0047 (blocked by weight/equal-number/eaten-bag hazards)
serving_admission_rule: resolve_promotable_loose_crayon_box_capacity()
implementation_allowed: true
reason: Capacity derivation from boxed portion + loose aggregation; 0047 confuser matrix holds.
```
## 6. Blocked families and why
| Family | Blocked because |
|--------|-----------------|
| calendar extension | No second train_sample sibling under #824 policy |
| DCS 0032/0047 | sealed-wrong; relation_hypothesis blocked |
| currency 0019/0028 | sealed-wrong |
| sealed_elimination 0011/0026 | sealed-wrong |
| wholesale multiplicative_aggregate | 0025/0047 sealed-wrong |
## 7. Research gate decision
**Proceed** with two independent singleton ClusterContracts (A2r + A2s). Calendar extension rejected for lack of sibling evidence.
## 8. Selected paradigm
Dual singleton organs under explicit contracts:
1. `generate/derivation/nested_fraction_remainder_total.py` — Gate A2r
2. `generate/derivation/loose_crayon_box_capacity.py` — Gate A2s
Wired in `generate/math_candidate_graph.py` after Gate A2q.
## 9. Typed chains for target cases
**0004 (gold 2000):**
- afternoon = 750 kids (3/4 of soccer-camp subgroup)
- subgroup = 750 / 0.75 = 1000
- total = 1000 × 2 = 2000 (half go to soccer camp)
**0007 (gold 2):**
- boxed = 85 5 = 80
- per_box = 80 / 5 = 16
- loose_total = 5 + 27 = 32
- boxes_needed = 32 / 16 = 2
## 10. Implemented organ(s)
| Module | Gate | Promotion entry |
|--------|------|-----------------|
| `nested_fraction_remainder_total.py` | A2r | `resolve_promotable_nested_fraction_remainder_total()` |
| `loose_crayon_box_capacity.py` | A2s | `resolve_promotable_loose_crayon_box_capacity()` |
## 11. Before/after score
| Metric | Before | After | Δ |
|--------|-------:|------:|--:|
| correct | 24 | **26** | +2 |
| refused | 26 | **24** | 2 |
| wrong | 0 | **0** | 0 |
**State A achieved:** 26/24/0.
## 12. Newly solved IDs
**0004**, **0007**
## 13. Preserved solved IDs
0001, 0002, 0003, 0005, 0006, 0008, 0009, 0010, 0013, 0014, 0015, 0017, 0018, 0021, 0024, 0025, 0029, 0030, 0035, 0037, 0038, 0042, 0045, 0046
## 14. Wrong=0 proof
- Full train_sample replay: 26 / 24 / 0
- `tests/test_math_candidate_graph_sprint12_singleton_contract_lift.py`
- Holdout_dev: wrong=0
- Regression lane: sprint612 + flywheel + scout + microscope
## 15. Confuser matrix
| Confuser | Organ | Result |
|----------|-------|--------|
| 0026 sealed_elimination | A2r | refuse |
| asks subgroup only | A2r | refuse |
| multiple fractions | A2r | refuse |
| 0047 equal-distribution weight | A2s | refuse |
| asks per-box capacity | A2s | refuse |
| equal number in bags | A2s | refuse |
## 16. Sibling generality proof
| Sibling | Expected | Result |
|---------|----------|--------|
| music camp / 600 afternoon | 1600 | pass |
| four marker boxes / 67 total | 6 boxes | pass |
## 17. Sealed-wrong negative evidence used
- 0026: currency + each + scoops + 3/4 bill → A2r hazard refusal
- 0047: ounces + equal number + eaten bag → A2s hazard refusal
## 18. Composition-validation pin changes
None.
## 19. report.json untouched
Confirmed — no changes to committed `report.json`.
## 20. sealed artifacts untouched
Confirmed — no sealed lane artifact movement.
## 21. no case-id logic
Confirmed — admission via structural anchors and bindings only.
## 22. no hardcoded answer
Confirmed — arithmetic recomputed from grounded quantities.
## 23. no direct-answer shortcut
Confirmed — self-verified derivation chains required.
## 24. ClusterContract method enabled bigger lift
Yes — bundling two narrow singleton contracts under explicit neighbor exclusions delivered +2 with wrong=0, exceeding single-organ sprint cadence.
## 25. Sprint 13 recommendation
1. **Practice-lane mini-family clustering** — flywheel joint_refusal clusters (20 cases) may yield a 2-case mini-family safer than next singleton.
2. **Deferred blocked surfaces** — DCS 0032/0047 and currency 0019/0028 remain blocked until dedicated confuser ADRs land.
3. **Calendar contract** — revisit only if practice lane surfaces a genuine second piecewise sibling without policy expansion.
4. Target: 27+/23-/0 via one mini-family or two verified singletons with lowest sealed-wrong adjacency (candidates from flywheel: 0004/0007 neighbors exhausted; next scout frontier TBD).

View file

@ -0,0 +1,285 @@
"""Gate A2s — loose crayon box capacity remainder (ClusterContract organ).
Sprint 12: train_sample **0007** actor has ``N full boxes`` plus loose items,
friend contributes loose items, conditional total count, derive per-box capacity
from boxed portion, question asks ``how many more boxes`` for all loose items.
Chain (0007-class):
boxed = total loose_primary
per_box = boxed / num_full_boxes
loose_total = loose_primary + loose_friend
boxes_needed = loose_total / per_box
Algebraically (single fold):
boxes_needed = (loose_primary + loose_friend) × num_full_boxes / (total loose_primary)
Narrow organ not equal-distribution/DCS (0047), not currency, not weight surfaces.
Promotion requires ClusterContract positive anchors and 0047-class hazard refusal.
Deterministic; sealed module (no ``chat/`` import).
"""
from __future__ import annotations
import re
from collections import Counter
from typing import Final
from generate.derivation.extract import extract_quantities
from generate.derivation.model import GroundedDerivation, Quantity, Step
from generate.derivation.target import _question_clause
from generate.derivation.verify import Resolution, SelfVerification
from generate.math_roundtrip import WORD_NUMBERS, _token_in, _tokens, _value_grounds
_FULL_BOXES_RE: Final[re.Pattern[str]] = re.compile(
r"(?i)(\w+)\s+has\s+(\w+)\s+full\s+boxes\s+of\s+(\w+)"
)
_LOOSE_PRIMARY_RE: Final[re.Pattern[str]] = re.compile(
r"(?i)(\d+)\s+loose\s+(\w+)"
)
_FRIEND_LOOSE_RE: Final[re.Pattern[str]] = re.compile(
r"(?i)(?:\w+\s+)?friend\s+has\s+(\d+)\s+loose"
)
_TOTAL_CONDITIONAL_RE: Final[re.Pattern[str]] = re.compile(
r"(?i)total\s+of\s+(\d+)"
)
_TEXT_BLOCKERS: Final[frozenset[str]] = frozenset(
{
"bags",
"bakes",
"brother",
"eats",
"equal",
"macaroons",
"ounces",
"packs",
"steve",
"weighing",
"weight",
}
)
_QUESTION_BLOCKERS: Final[frozenset[str]] = frozenset(
{"ounces", "weight", "per", "macaroons"}
)
def _box_count_token(problem_text: str, num_boxes: float) -> str:
match = _FULL_BOXES_RE.search(problem_text)
if match is not None:
return match.group(2).lower()
return str(int(num_boxes))
def _resolve_count(token: str) -> float | None:
lowered = token.lower()
if lowered in WORD_NUMBERS:
return float(WORD_NUMBERS[lowered])
try:
return float(token)
except ValueError:
return None
def _asks_more_boxes(question_clause: str) -> bool:
tokens = _tokens(question_clause)
return (
"how" in tokens
and "many" in tokens
and "more" in tokens
and "boxes" in tokens
)
def _asks_per_box(question_clause: str) -> bool:
tokens = _tokens(question_clause)
return "each" in tokens or "per" in tokens
def _has_hazard_surface(problem_text: str, question_clause: str) -> bool:
if "$" in problem_text:
return True
text_tokens = _tokens(problem_text)
question_tokens = _tokens(question_clause)
if text_tokens & _TEXT_BLOCKERS:
return True
if question_tokens & _QUESTION_BLOCKERS:
return True
if _asks_per_box(question_clause):
return True
if "equal number" in problem_text.lower():
return True
if not _FULL_BOXES_RE.search(problem_text):
return True
if not _FRIEND_LOOSE_RE.search(problem_text):
return True
if not _TOTAL_CONDITIONAL_RE.search(question_clause):
return True
return False
def _parse_bindings(problem_text: str) -> dict[str, float] | None:
boxes_match = _FULL_BOXES_RE.search(problem_text)
if boxes_match is None:
return None
num_boxes = _resolve_count(boxes_match.group(2))
if num_boxes is None or num_boxes <= 0:
return None
friend_match = _FRIEND_LOOSE_RE.search(problem_text)
if friend_match is None:
return None
friend_pos = friend_match.start()
loose_primary_matches = [
match
for match in _LOOSE_PRIMARY_RE.finditer(problem_text)
if match.start() < friend_pos
]
if len(loose_primary_matches) != 1:
return None
loose_primary = float(loose_primary_matches[0].group(1))
loose_friend = float(friend_match.group(1))
total_match = _TOTAL_CONDITIONAL_RE.search(_question_clause(problem_text))
if total_match is None:
return None
total = float(total_match.group(1))
item_unit = loose_primary_matches[0].group(2).lower()
return {
"num_boxes": num_boxes,
"loose_primary": loose_primary,
"loose_friend": loose_friend,
"total": total,
"item_unit": item_unit,
}
def _recompute_boxes_needed(bindings: dict[str, float]) -> float:
boxed = bindings["total"] - bindings["loose_primary"]
per_box = boxed / bindings["num_boxes"]
loose_total = bindings["loose_primary"] + bindings["loose_friend"]
return loose_total / per_box
def build_loose_crayon_box_capacity(
problem_text: str,
) -> tuple[GroundedDerivation, float] | None:
"""Construct the ungated box-capacity chain, or ``None``."""
question_clause = _question_clause(problem_text)
if not _asks_more_boxes(question_clause):
return None
if _has_hazard_surface(problem_text, question_clause):
return None
bindings = _parse_bindings(problem_text)
if bindings is None:
return None
boxed_total = bindings["total"] - bindings["loose_primary"]
if boxed_total <= 0:
return None
loose_total = bindings["loose_primary"] + bindings["loose_friend"]
answer = _recompute_boxes_needed(bindings)
item_unit = bindings["item_unit"]
derivation = GroundedDerivation(
start=Quantity(
value=bindings["loose_primary"],
unit=item_unit,
source_token=str(int(bindings["loose_primary"])),
),
steps=(
Step(
op="add",
operand=Quantity(
value=bindings["loose_friend"],
unit=item_unit,
source_token=str(int(bindings["loose_friend"])),
),
cue="friend",
),
Step(
op="multiply",
operand=Quantity(
value=bindings["num_boxes"],
unit="boxes",
source_token=_box_count_token(problem_text, bindings["num_boxes"]),
),
cue=_box_count_token(problem_text, bindings["num_boxes"]),
),
Step(
op="divide",
operand=Quantity(
value=boxed_total,
unit=item_unit,
source_token=str(int(bindings["total"])),
),
cue="total",
comparative=True,
),
),
)
return derivation, answer
def _self_verifies_box_capacity(
derivation: GroundedDerivation, problem_text: str, expected: float
) -> SelfVerification:
tokens = _tokens(problem_text)
reasons: list[str] = []
for q in [derivation.start, *(s.operand for s in derivation.steps if not s.comparative)]:
if not _value_grounds(q.source_token, tokens):
reasons.append(f"operand {q.source_token!r} not grounded in text")
for step in derivation.steps:
if not _token_in(step.cue, tokens):
reasons.append(f"operation cue {step.cue!r} not grounded in text")
bindings = _parse_bindings(problem_text)
if bindings is None:
reasons.append("missing box-capacity bindings")
elif abs(_recompute_boxes_needed(bindings) - expected) > 1e-9:
reasons.append("arithmetic mismatch on boxes needed")
if abs(derivation.answer - expected) > 1e-9:
reasons.append("derivation fold mismatch")
obligation = Counter(q.source_token for q in extract_quantities(problem_text))
used = Counter(
[
derivation.start.source_token,
*(step.operand.source_token for step in derivation.steps),
]
)
unused = obligation - used
if unused:
reasons.append(f"incomplete: unused quantities {sorted(unused.keys())}")
return SelfVerification(verified=not reasons, reasons=tuple(reasons))
def compose_loose_crayon_box_capacity(problem_text: str) -> Resolution | None:
"""Gate the typed box-capacity chain through self-verification."""
built = build_loose_crayon_box_capacity(problem_text)
if built is None:
return None
derivation, answer = built
if not _self_verifies_box_capacity(derivation, problem_text, answer).verified:
return None
return Resolution(
answer=answer,
answer_unit="boxes",
derivation=derivation,
)
def resolve_promotable_loose_crayon_box_capacity(
problem_text: str,
) -> Resolution | None:
"""Serving promotion bridge (Gate A2s, ClusterContract loose_crayon_box)."""
return compose_loose_crayon_box_capacity(problem_text)

View file

@ -0,0 +1,260 @@
"""Gate A2r — nested fraction complement remainder total (ClusterContract organ).
Sprint 12: train_sample **0004** outer ``Half of`` partition, inner ``1/4 of``
subgroup split (morning vs afternoon), known afternoon complement count, question
asks root population ``altogether``.
Chain (0004-class):
subgroup = afternoon / (1 inner_fraction)
total = subgroup / outer_half_scalar = subgroup × 2
Narrow organ not generic fraction parser, not sealed_elimination, not currency.
Promotion requires ClusterContract positive anchors, explicit target binding, and
0026-class hazard refusal.
Deterministic; sealed module (no ``chat/`` import).
"""
from __future__ import annotations
import re
from collections import Counter
from dataclasses import dataclass
from typing import Final
from generate.derivation.extract import extract_quantities
from generate.derivation.model import GroundedDerivation, Quantity, Step
from generate.derivation.target import _question_clause
from generate.derivation.verify import Resolution, SelfVerification
from generate.math_roundtrip import _token_in, _tokens, _value_grounds
_OUTER_HALF_RE: Final[re.Pattern[str]] = re.compile(
r"(?i)\bhalf\s+of\s+the\s+(\w+)\b"
)
_INNER_QUARTER_RE: Final[re.Pattern[str]] = re.compile(
r"(?i)\b1/4\s+of\s+the\s+(\w+)\s+going\s+to\s+(\w+)\s+camp\b"
)
_MORNING_RE: Final[re.Pattern[str]] = re.compile(
r"(?i)\bgoing\s+to\s+(\w+)\s+camp\s+in\s+the\s+morning\b"
)
_AFTERNOON_COUNT_RE: Final[re.Pattern[str]] = re.compile(
r"(?i)(\d+)\s+(\w+)\s+are\s+going\s+to\s+(\w+)\s+camp\s+in\s+the\s+afternoon"
)
_EXTRA_FRACTION_RE: Final[re.Pattern[str]] = re.compile(r"\d+\s*/\s*\d+")
_TEXT_BLOCKERS: Final[frozenset[str]] = frozenset(
{
"bill",
"dinner",
"dollars",
"each",
"ice",
"money",
"profit",
"saved",
"scoops",
}
)
_QUESTION_BLOCKERS: Final[frozenset[str]] = frozenset(
{"afternoon", "morning", "scoops"}
)
@dataclass(frozen=True, slots=True)
class _CampBinding:
population: str
camp: str
afternoon_count: float
afternoon_token: str
def _asks_root_altogether(question_clause: str, population: str) -> bool:
tokens = _tokens(question_clause)
return (
"how" in tokens
and "many" in tokens
and population in tokens
and ("altogether" in tokens or "there" in tokens)
)
def _asks_subgroup_only(question_clause: str) -> bool:
tokens = _tokens(question_clause)
return "camp" in tokens and "altogether" not in tokens
def _parse_camp_binding(problem_text: str) -> _CampBinding | None:
outer = _OUTER_HALF_RE.search(problem_text)
inner = _INNER_QUARTER_RE.search(problem_text)
morning = _MORNING_RE.search(problem_text)
afternoon = _AFTERNOON_COUNT_RE.search(problem_text)
if outer is None or inner is None or morning is None or afternoon is None:
return None
population = outer.group(1).lower()
inner_pop = inner.group(1).lower()
inner_camp = inner.group(2).lower()
morning_camp = morning.group(1).lower()
afternoon_count = float(afternoon.group(1))
afternoon_pop = afternoon.group(2).lower()
afternoon_camp = afternoon.group(3).lower()
if not {population, inner_pop, afternoon_pop} == {population}:
return None
if not {inner_camp, morning_camp, afternoon_camp} == {inner_camp}:
return None
return _CampBinding(
population=population,
camp=inner_camp,
afternoon_count=afternoon_count,
afternoon_token=afternoon.group(1),
)
def _has_hazard_surface(problem_text: str, question_clause: str) -> bool:
if "$" in problem_text:
return True
text_tokens = _tokens(problem_text)
question_tokens = _tokens(question_clause)
if text_tokens & _TEXT_BLOCKERS:
return True
if question_tokens & _QUESTION_BLOCKERS:
return True
if _asks_subgroup_only(question_clause):
return True
if "3/4" in problem_text.replace(" ", ""):
return True
if "saved" in text_tokens and "each" in text_tokens:
return True
fractions = _EXTRA_FRACTION_RE.findall(problem_text)
if len(fractions) != 1 or fractions[0].replace(" ", "") != "1/4":
return True
return False
def _inner_complement_scalar() -> float:
return 1.0 - (1.0 / 4.0)
def _outer_inverse_half_scalar() -> float:
return 2.0
def _recompute_total(afternoon_count: float) -> float:
subgroup = afternoon_count / _inner_complement_scalar()
return subgroup * _outer_inverse_half_scalar()
def build_nested_fraction_remainder_total(
problem_text: str,
) -> tuple[GroundedDerivation, float, _CampBinding] | None:
"""Construct the ungated nested-fraction chain, or ``None``."""
binding = _parse_camp_binding(problem_text)
if binding is None:
return None
question_clause = _question_clause(problem_text)
if not _asks_root_altogether(question_clause, binding.population):
return None
if _has_hazard_surface(problem_text, question_clause):
return None
afternoon = Quantity(
value=binding.afternoon_count,
unit=binding.population,
source_token=binding.afternoon_token,
)
total = _recompute_total(binding.afternoon_count)
derivation = GroundedDerivation(
start=afternoon,
steps=(
Step(
op="divide",
operand=Quantity(
value=_inner_complement_scalar(),
unit="",
source_token="1/4",
),
cue="4",
comparative=True,
),
Step(
op="multiply",
operand=Quantity(
value=_outer_inverse_half_scalar(),
unit="",
source_token="half",
),
cue="half",
comparative=True,
),
),
)
return derivation, total, binding
def _self_verifies_nested_fraction(
derivation: GroundedDerivation,
problem_text: str,
expected_total: float,
binding: _CampBinding,
) -> SelfVerification:
tokens = _tokens(problem_text)
reasons: list[str] = []
if not _operand_grounds(derivation.start, tokens):
reasons.append(f"operand {derivation.start.source_token!r} not grounded in text")
for step in derivation.steps:
if not step.comparative and not _operand_grounds(step.operand, tokens):
reasons.append(f"operand {step.operand.source_token!r} not grounded in text")
if not _token_in(step.cue, tokens):
reasons.append(f"operation cue {step.cue!r} not grounded in text")
if _parse_camp_binding(problem_text) is None:
reasons.append("missing camp partition binding")
elif abs(_recompute_total(binding.afternoon_count) - expected_total) > 1e-9:
reasons.append("arithmetic mismatch on nested fraction total")
if abs(derivation.answer - expected_total) > 1e-9:
reasons.append("derivation fold mismatch")
obligation = Counter(
q.source_token
for q in extract_quantities(problem_text)
if q.unit == binding.population and q.source_token.isdigit()
)
used = Counter([derivation.start.source_token])
unused = obligation - used
if unused:
reasons.append(
f"incomplete: unused {binding.population} quantities {sorted(unused.keys())}"
)
return SelfVerification(verified=not reasons, reasons=tuple(reasons))
def _operand_grounds(q: Quantity, tokens: frozenset[str]) -> bool:
return _value_grounds(q.source_token, tokens)
def compose_nested_fraction_remainder_total(problem_text: str) -> Resolution | None:
"""Gate the typed nested-fraction chain through self-verification."""
built = build_nested_fraction_remainder_total(problem_text)
if built is None:
return None
derivation, total, binding = built
if not _self_verifies_nested_fraction(derivation, problem_text, total, binding).verified:
return None
return Resolution(
answer=total,
answer_unit=binding.population,
derivation=derivation,
)
def resolve_promotable_nested_fraction_remainder_total(
problem_text: str,
) -> Resolution | None:
"""Serving promotion bridge (Gate A2r, ClusterContract nested_fraction)."""
return compose_nested_fraction_remainder_total(problem_text)

View file

@ -898,6 +898,38 @@ def parse_and_solve(text: str, *, sealed: bool = False) -> CandidateGraphResult:
branches_admissible=1, branches_admissible=1,
) )
# Gate A2r — nested fraction complement remainder total (Sprint 12 singleton).
from generate.derivation.nested_fraction_remainder_total import (
resolve_promotable_nested_fraction_remainder_total,
)
nested_fraction_resolution = resolve_promotable_nested_fraction_remainder_total(
text
)
if nested_fraction_resolution is not None:
return CandidateGraphResult(
answer=nested_fraction_resolution.answer,
selected_graph=None,
refusal_reason=None,
branches_enumerated=1,
branches_admissible=1,
)
# Gate A2s — loose crayon box capacity (Sprint 12 singleton).
from generate.derivation.loose_crayon_box_capacity import (
resolve_promotable_loose_crayon_box_capacity,
)
box_capacity_resolution = resolve_promotable_loose_crayon_box_capacity(text)
if box_capacity_resolution is not None:
return CandidateGraphResult(
answer=box_capacity_resolution.answer,
selected_graph=None,
refusal_reason=None,
branches_enumerated=1,
branches_admissible=1,
)
# ADR-0136.S.1 — Rate/event short-circuit paths (before Cartesian product). # ADR-0136.S.1 — Rate/event short-circuit paths (before Cartesian product).
# Capacity path: single statement with one CandidateCapacity + matching question. # Capacity path: single statement with one CandidateCapacity + matching question.
if len(statement_sentences) == 1: if len(statement_sentences) == 1:

View file

@ -104,7 +104,7 @@ def test_markdown_render_surfaces_partition_candidate():
summary = build_microscope_report(_load_cases()) summary = build_microscope_report(_load_cases())
md = render_markdown(summary) md = render_markdown(summary)
assert "partition_chunking" in md assert "partition_chunking" in md
assert "correct: 24" in md assert "correct: 26" in md
assert "Gate A2a unit_partition" in md assert "Gate A2a unit_partition" in md

View file

@ -218,8 +218,8 @@ class TestTrainSampleScore:
report = build_report(_load_cases(_CASES_PATH)) report = build_report(_load_cases(_CASES_PATH))
counts = report["counts"] counts = report["counts"]
assert counts["wrong"] == 0 assert counts["wrong"] == 0
assert counts["correct"] >= 24 assert counts["correct"] >= 26
assert counts["refused"] <= 26 assert counts["refused"] <= 24
def test_newly_solved_ids(self) -> None: def test_newly_solved_ids(self) -> None:
from evals.gsm8k_math.train_sample.v1.runner import build_report, _load_cases from evals.gsm8k_math.train_sample.v1.runner import build_report, _load_cases

View file

@ -232,8 +232,8 @@ class TestTrainSampleScore:
report = build_report(_load_cases(_CASES_PATH)) report = build_report(_load_cases(_CASES_PATH))
counts = report["counts"] counts = report["counts"]
assert counts["wrong"] == 0 assert counts["wrong"] == 0
assert counts["correct"] >= 24 assert counts["correct"] >= 26
assert counts["refused"] <= 26 assert counts["refused"] <= 24
def test_newly_solved_0013(self) -> None: def test_newly_solved_0013(self) -> None:
from evals.gsm8k_math.train_sample.v1.runner import build_report, _load_cases from evals.gsm8k_math.train_sample.v1.runner import build_report, _load_cases

View file

@ -0,0 +1,231 @@
"""Gate A2r/A2s — Capability Paradigm Sprint 12 singleton ClusterContract lift."""
from __future__ import annotations
import json
from pathlib import Path
import pytest
from generate.math_candidate_graph import parse_and_solve
from generate.derivation.nested_fraction_remainder_total import (
compose_nested_fraction_remainder_total,
resolve_promotable_nested_fraction_remainder_total,
)
from generate.derivation.loose_crayon_box_capacity import (
compose_loose_crayon_box_capacity,
resolve_promotable_loose_crayon_box_capacity,
)
_CASES_PATH = (
Path(__file__).resolve().parents[1]
/ "evals"
/ "gsm8k_math"
/ "train_sample"
/ "v1"
/ "cases.jsonl"
)
CASE_0004 = (
"There are some kids in camp. Half of the kids are going to soccer camp, "
"and 1/4 of the kids going to soccer camp are going to soccer camp in the morning. "
"750 kids are going to soccer camp in the afternoon. "
"How many kids there in camp altogether?"
)
CASE_0007 = (
"Francine has five full boxes of crayons and 5 loose crayons, and her friend "
"has 27 loose crayons. They need to put all of their loose crayons in a box. "
"How many more boxes do they need if Francine has a total of 85 crayons?"
)
SIBLING_CAMP = (
"There are some students in camp. Half of the students are going to music camp, "
"and 1/4 of the students going to music camp are going to music camp in the morning. "
"600 students are going to music camp in the afternoon. "
"How many students there in camp altogether?"
)
SIBLING_BOXES = (
"Jordan has four full boxes of markers and 3 loose markers, and her friend "
"has 21 loose markers. They need to put all of their loose markers in a box. "
"How many more boxes do they need if Jordan has a total of 19 markers?"
)
SEALED_ELIMINATION_0026 = (
"Aaron and his brother Carson each saved up $40 to go to dinner. "
"The bill is 3/4 of their total money. After, they go out for ice cream. "
"Each scoop costs $1.5 and they get the same amount as each other. "
"If they leave with $1 in change each, how many scoops did they each buy?"
)
SEALED_WRONG_0047_SHAPE = (
"John bakes 12 coconut macaroons, each weighing 5 ounces. "
"He then packs an equal number of the macaroons in 4 different brown bags, "
"ready for delivery. When he briefly leaves the kitchen to pick the phone, "
"his little brother Steve eats the entire contents of one of the brown bags. "
"What is the total weight, in ounces, of the remaining coconut macaroons?"
)
ASKS_SUBGROUP_ONLY_CONFUSER = (
"There are some kids in camp. Half of the kids are going to soccer camp, "
"and 1/4 of the kids going to soccer camp are going to soccer camp in the morning. "
"750 kids are going to soccer camp in the afternoon. "
"How many kids are going to soccer camp?"
)
ASKS_PER_BOX_CONFUSER = (
"Francine has five full boxes of crayons and 5 loose crayons, and her friend "
"has 27 loose crayons. They need to put all of their loose crayons in a box. "
"How many crayons are in each full box if Francine has a total of 85 crayons?"
)
MULTIPLE_FRACTION_CONFUSER = (
"Half of the kids are going to soccer camp, and 1/4 of the kids going to soccer camp "
"are going in the morning. 1/2 of the afternoon group brought lunch. "
"750 kids are going to soccer camp in the afternoon. "
"How many kids there in camp altogether?"
)
PRESERVED_SOLVED = (
"0001",
"0002",
"0003",
"0005",
"0006",
"0008",
"0009",
"0010",
"0013",
"0014",
"0015",
"0017",
"0018",
"0021",
"0024",
"0025",
"0029",
"0030",
"0035",
"0037",
"0038",
"0042",
"0045",
"0046",
)
def _load_case(case_suffix: str) -> str:
for line in _CASES_PATH.read_text().splitlines():
row = json.loads(line)
if row["case_id"].endswith(case_suffix):
return row["question"]
raise KeyError(case_suffix)
class TestClusterContractPolicy:
def test_nested_fraction_contract_admits_0004_shape(self) -> None:
assert compose_nested_fraction_remainder_total(CASE_0004) is not None
def test_box_capacity_contract_admits_0007_shape(self) -> None:
assert compose_loose_crayon_box_capacity(CASE_0007) is not None
class TestTargetCases:
def test_0004_solves(self) -> None:
result = parse_and_solve(CASE_0004)
assert result.answer == 2000.0
assert result.refusal_reason is None
def test_0007_solves(self) -> None:
result = parse_and_solve(CASE_0007)
assert result.answer == 2.0
assert result.refusal_reason is None
class TestSiblingGeneralization:
def test_music_camp_sibling(self) -> None:
resolution = compose_nested_fraction_remainder_total(SIBLING_CAMP)
assert resolution is not None
assert resolution.answer == 1600.0
def test_marker_box_sibling(self) -> None:
resolution = compose_loose_crayon_box_capacity(SIBLING_BOXES)
assert resolution is not None
# (3+21)*4/(67-3) = 96/16 = 6
assert resolution.answer == 6.0
class TestNeighborConfuserRefusals:
def test_sealed_elimination_0026_refuses(self) -> None:
assert compose_nested_fraction_remainder_total(SEALED_ELIMINATION_0026) is None
def test_dcs_0047_refuses(self) -> None:
assert compose_loose_crayon_box_capacity(SEALED_WRONG_0047_SHAPE) is None
def test_asks_subgroup_only_refuses(self) -> None:
assert compose_nested_fraction_remainder_total(ASKS_SUBGROUP_ONLY_CONFUSER) is None
def test_asks_per_box_refuses(self) -> None:
assert compose_loose_crayon_box_capacity(ASKS_PER_BOX_CONFUSER) is None
def test_multiple_fraction_refuses(self) -> None:
assert compose_nested_fraction_remainder_total(MULTIPLE_FRACTION_CONFUSER) is None
class TestPromotionBridges:
def test_resolve_promotable_nested_fraction(self) -> None:
assert resolve_promotable_nested_fraction_remainder_total(CASE_0004) is not None
def test_resolve_promotable_box_capacity(self) -> None:
assert resolve_promotable_loose_crayon_box_capacity(CASE_0007) is not None
class TestTrainSampleScore:
def test_wrong_zero_and_state_a(self) -> None:
from evals.gsm8k_math.train_sample.v1.runner import build_report, _load_cases
report = build_report(_load_cases(_CASES_PATH))
counts = report["counts"]
assert counts["wrong"] == 0
assert counts["correct"] >= 26
assert counts["refused"] <= 24
def test_newly_solved_ids(self) -> None:
from evals.gsm8k_math.train_sample.v1.runner import build_report, _load_cases
report = build_report(_load_cases(_CASES_PATH))
correct = {
row["case_id"].split("-")[-1]
for row in report["per_case"]
if row["verdict"] == "correct"
}
assert "0004" in correct
assert "0007" in correct
def test_preserved_solved_ids(self) -> None:
from evals.gsm8k_math.train_sample.v1.runner import build_report, _load_cases
report = build_report(_load_cases(_CASES_PATH))
correct = {
row["case_id"].split("-")[-1]
for row in report["per_case"]
if row["verdict"] == "correct"
}
for case_id in PRESERVED_SOLVED:
assert case_id in correct
class TestHoldoutInert:
def test_holdout_dev_wrong_zero(self) -> None:
from evals.gsm8k_math.holdout_dev.v1.runner import build_report
report = build_report()
assert report["counts"]["wrong"] == 0
class TestPriorOrganRegressions:
def test_piecewise_0013_preserved(self) -> None:
result = parse_and_solve(_load_case("0013"))
assert result.answer == 450.0
assert result.refusal_reason is None