e9b7eb0b1f
* chore(ADR-0163.C): land three Phase C pending proposals in live log Phase C (#301) shipped the CLI but its PR dry-run wrote to a tmp log path. This commit moves the three Phase C proposals into the live teaching/proposals/proposals.jsonl so the Phase B→C audit trail is visible in the proposal log and the proposals are ready for the operator to ratify after Phase D ships. Proposals (all state=pending, kind="exemplar_corpus"): - 59223f13722f906a1cf9b65d9b01c990 — descriptive_setup_no_quantity - 46ce297f797ff16da12db5de422ca3c9 — rate_with_currency - a3b892546977c5f0f64c578d6052adbd — temporal_aggregation Produced by `core teaching propose-from-exemplars --all` against the live Phase B corpora. No ratification (ADR-0161 §5 — only the repo owner ratifies). The Phase D admissibility-replay gate confirmed replay_equivalent=true, wrong_count_delta=0 for all three. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * feat(ADR-0163.D): wire ratified RecognizerSpecs into math_candidate_graph admissibility surface Phase D is the first PR to extend the math admission surface. The audit (#294) said the gap was admission, not operators, algebra, substrate, or packs. Phase A measured the refusal taxonomy. Phase B authored seeds. Phase C synthesized recognizers. Phase D wires those recognizers into generate/math_candidate_graph.py. Modules - generate/recognizer_registry.py — pure projection over the proposal log. Only proposals with source.kind="exemplar_corpus" AND review_state="accepted" enter the tuple. Sorted by (review_date, proposal_id). In-process cache keyed on log (mtime, sha256) — no filesystem cache (ADR-0161 §1). Malformed accepted specs raise RegistryLoadError citing the offending proposal_id; silent drops are forbidden. - generate/recognizer_match.py — per-category rules-only matchers (no LLM, no embedding, no learned classifier). Honors the Phase C synthesizer's narrowness rule: out-of-corpus currency symbols, window units, and per-unit values do NOT match. Three matchers: _match_descriptive_setup_no_quantity (zero-quantity surface), _match_temporal_aggregation (event_count_per_window with observed_window_units/quantifiers honored), _match_rate_with_currency (currency_per_unit_rate with observed currency/per-unit/amount-kind honored). - generate/math_candidate_graph.py — narrowest-edit guard at the per-statement choice loop. Before the existing "no admissible candidate for statement" refusal, consult the ratified registry. Recognized statements are dropped from per_sentence_choices (zero math state) so the Cartesian product is identical to "this statement was never there." Empty registry is a no-op — backward compatibility preserved byte-identically. Downstream consumption of parsed_anchors (turning recognized rate/temporal surfaces into solver state that produces concrete answers) is Phase E follow-up. Tests (32 new) - tests/_phase_d_fixture.py — synthetic in-memory ratified registry built from the three Phase C pending proposals' content. Per ADR-0161 §5 the agent does NOT ratify the live log; the synthetic registry round-trips the real RecognizerSpec bytes the operator will ratify after Phase D ships. - tests/test_recognizer_registry.py (9) — empty/pending/wrong-kind filtering, sort order, malformed-spec rejection, cache hit + invalidation, live-log Phase C audit check. - tests/test_recognizer_match.py (14) — per-category positive cases, narrowness (out-of-corpus surface forms rejected), no-LLM import check. - tests/test_candidate_graph_recognizer_wiring.py (7) — empty registry preserves existing refusal; synthetic registry: recognized statements no longer trigger per-statement refusal; wrong_count_delta == 0 on GSM8K train_sample; capability axes G1.. G5+S1 wrong=0 unchanged; per-category admission counts on the refused-set; unrecognized statements still refuse with the existing reason. - tests/test_phase_d_replay_evidence.py (2) — full admissibility replay gate under synthetic registry: replay_equivalent=true, wrong_count_delta=0, every capability axis wrong=0; each ratified recognizer admits >= 1 train_sample statement (wiring is consequential). Per-category fixture-based admission counts (synthetic registry vs GSM8K train_sample refused-set sentences): - descriptive_setup_no_quantity: 40 - rate_with_currency: 2 - temporal_aggregation: 7 Narrowness-invariant negative case results (matcher correctly returns None on out-of-corpus / load-bearing-math surfaces): - rate_with_currency: "She paid $5 for the book." (no per-unit) - temporal_aggregation: "On Saturday she went to the store." (single day token) - descriptive_setup_no_quantity: "There are some kids in camp." (indefinite quantifier) Candidates for Phase B round 2 (3 of 20 temporal seeds match the spec's structural commitment but not my surface regex — author_notes explicitly flagged these as schema-gap edge cases): - ta-v1-0004 "Mark does a gig every other day for 2 weeks." - ta-v1-0012 "Robin walks 4 dogs every other day around the park." - ta-v1-0019 "The pump fills the tank with 80 gallons over 6 hours." Three landed wirings DO NOT shift the GSM8K train_sample baseline counts under fixture (correct=3, wrong=0, refused=47 unchanged) — Phase D's narrow wiring is wrong=0 safe by construction; lift to "correct" requires Phase E's downstream parser-side consumption of parsed_anchors. Capability axes G1..G5+S1 wrong=0 unchanged. Cross-refs: ADR-0163 (Phase D), ADR-0057 (proposal review), ADR-0151 (auto-proposal), ADR-0161 §5 (ratification boundary), Phase A PR #297, Phase B PR #298, Phase C PR #301. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
79 lines
3 KiB
Python
79 lines
3 KiB
Python
"""Phase D test fixture — synthetic ratified registry from live log.
|
|
|
|
Per ADR-0161 §5, the agent does NOT ratify the live proposal log. The
|
|
agent's tests build a SYNTHETIC RATIFIED REGISTRY in memory from the
|
|
three live PENDING Phase C proposals, populating ``review_date`` with
|
|
a fixed synthetic date. This exercises the per-category match
|
|
functions + the candidate-graph wiring against the EXACT
|
|
RecognizerSpec content the operator will later ratify, with zero
|
|
modification of the live proposal log.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
from typing import Mapping
|
|
|
|
from evals.refusal_taxonomy.shape_categories import ShapeCategory
|
|
from generate.recognizer_registry import RatifiedRecognizer
|
|
from teaching.proposals import ProposalLog
|
|
|
|
|
|
PHASE_C_PROPOSAL_IDS: tuple[str, ...] = (
|
|
"59223f13722f906a1cf9b65d9b01c990", # descriptive_setup_no_quantity
|
|
"46ce297f797ff16da12db5de422ca3c9", # rate_with_currency
|
|
"a3b892546977c5f0f64c578d6052adbd", # temporal_aggregation
|
|
)
|
|
|
|
SYNTHETIC_REVIEW_DATE: str = "2026-05-27"
|
|
|
|
|
|
def build_synthetic_registry(
|
|
log_path: Path | None = None,
|
|
) -> tuple[RatifiedRecognizer, ...]:
|
|
"""Build the in-memory ratified registry from live pending Phase C proposals.
|
|
|
|
Reads ``teaching/proposals/proposals.jsonl`` (or *log_path*), pulls
|
|
the three Phase C proposals by id, and converts their pending
|
|
``proposed_chain.recognizer_spec`` payloads into
|
|
:class:`RatifiedRecognizer` records.
|
|
|
|
Raises :class:`AssertionError` if any of the three Phase C
|
|
proposal_ids cannot be located in the log. This is intentional —
|
|
Phase D's tests assume Phase C's exemplar-corpus proposals exist;
|
|
if they don't, the operator should re-run
|
|
``core teaching propose-from-exemplars --all`` first.
|
|
"""
|
|
log = ProposalLog(log_path)
|
|
state = log.current_state()
|
|
recognizers: list[RatifiedRecognizer] = []
|
|
for pid in PHASE_C_PROPOSAL_IDS:
|
|
record = state.get(pid)
|
|
assert record is not None, (
|
|
f"Phase D fixture: proposal {pid!r} not in {log.path}; "
|
|
"run `core teaching propose-from-exemplars --all` first"
|
|
)
|
|
chain = record["proposal"]["proposed_chain"]
|
|
spec: Mapping[str, object] = chain["recognizer_spec"] # type: ignore[assignment]
|
|
shape_category = ShapeCategory(spec["shape_category"]) # type: ignore[arg-type]
|
|
recognizers.append(
|
|
RatifiedRecognizer(
|
|
proposal_id=pid,
|
|
shape_category=shape_category,
|
|
canonical_pattern=spec["canonical_pattern"], # type: ignore[arg-type]
|
|
spec_digest=str(chain["object"]),
|
|
review_date=SYNTHETIC_REVIEW_DATE,
|
|
ratified_at_revision=str(
|
|
record["proposal"]["source"]["emitted_at_revision"]
|
|
),
|
|
)
|
|
)
|
|
recognizers.sort(key=lambda r: (r.review_date, r.proposal_id))
|
|
return tuple(recognizers)
|
|
|
|
|
|
__all__ = [
|
|
"PHASE_C_PROPOSAL_IDS",
|
|
"SYNTHETIC_REVIEW_DATE",
|
|
"build_synthetic_registry",
|
|
]
|