core/docs/recognizer-registry.md
Shay 5f6e9f5e08
feat(derivation): Workstream A inc 2 — frontier report + rate_with_currency apply_rate injection (#797)
* feat(derivation): Workstream A inc 2 — frontier report + rate_with_currency apply_rate injection

- scripts/gsm8k_frontier_report.py + test (stable buckets; rate_with_currency surfaced)
- docs/recognizer-registry.md + math_candidate_graph.py comments repaired (current refusal doctrine; old skip-only marked historical)
- generate/math_roundtrip.py: add 'a','an' to RATE_ANCHORS (with doc update)
- generate/recognizer_anchor_inject.py: inject_rate_with_currency (narrow ProperName actor, Rate/apply_rate CandidateOperation, rejects unsafe); registered in _INJECTORS; module docs updated
- tests/test_*_rate_injection*.py + frontier test (8+ unit cases, confusers, synthetic wiring, real-report frontier pin)
- ratification doc (pre-code)
- lookback (post-impl, truthful)

All required local commands exercised (pytest green for new + prior extract/invariants; frontier script shows rate bucket; runner per brief; shas captured).

wrong=0 held. No sealed movement. Proxy still expected !passed (correct_min=10).

See ratification and lookback for scope, hazards, exact outputs.

* fix(derivation): dollar grounding, roundtrip proofs, stale comment, lookback commit, a/an containment, duplicate import

- math_roundtrip: add explicit 'dollar'/'dollars' + '$' branch in _unit_grounds (symmetric to cent) + doc alignment.
- tests: assert roundtrip_admissible True for the emitted rate cands; add a/an containment/confuser test (dollar unit only grounds with $ present); add lower-level _apply_rate solver reach test; remove conditional happy-paths that tolerated refusal; clean duplicate __future__.
- math_candidate_graph: update active stale comment (injector-empty now leads to explicit refusal, not drop/preserve-zero).
- Commit the Inc2 lookback (was local-only).

All per lead block list. New head will be pushed.

* fix(derivation): address remaining Inc2 blockers (locate_rate_verb via matcher token, dispatch test, lookback accuracy, narrow symbols, tests)

- recognizer_match: updated _CURRENCY_AMOUNT_RE + parsing to populate 'rate_anchor_token' localized to the rate span (prevents whole-sentence 'a' hazard from 'a lemonade stand').
- recognizer_anchor_inject: use 'rate_anchor_token' from anchor (with allowed-set check); fallback only if absent. Added Alexa-style confuser coverage.
- injector: narrowed _CURRENCY_SYMBOL_TO_UNIT to $ only for Inc2.
- tests: strengthened dispatch to require live-registry non-empty + roundtrip_admissible; added rate_anchor_token confuser test proving 'per' (not earlier 'a') is used; roundtrip asserts already present.
- lookback: updated to 10 files, 10+6 test counts, current head reality, shas/runner status exact, no rebaseline.
- math_candidate_graph comment already corrected in prior.

All Semantic Rigor / wrong=0 invariants preserved. New tip after push.

* fix(inc2): remove rate fallback to _locate_rate_verb; mandatory rate_anchor_token; hard 'for one cup' confuser; strengthen per-span test; update lookback

- injector: for currency_per_unit_rate, rate_anchor_token is now mandatory. Absent or invalid → immediate return ().
  No call to _locate_rate_verb for these anchors.
- tests: _rate_anchor helper now passes rate_anchor_token.
- Added hard confuser test for 'Alexa ...  for one cup.' — live registry + inject_from_match must emit ().
- Strengthened distracting-a ' per' test to unconditional: m not None, RATE_WITH_CURRENCY, len==1, matched_verb=='per', roundtrip_admissible True.
- lookback updated for 11 files, include matcher.py, rate_anchor_token centrality, exact counts after this.

* docs: pin exact final SHA in Inc2 lookback

* test(inc2): strengthen for_one_cup confuser to unconditional asserts on m and category

* docs(inc2): update lookback with exact test counts 18/6, 11 files, remove 9 files line, note final SHA after push

* docs(inc2): pin exact final SHA c4e83399... in lookback

* docs(inc2): pin exact final SHA 1129c502e6e8... in lookback

* docs(inc2): note no PR/CI yet in lookback

* docs(inc2): pin ultimate final SHA 7df0cac4... in lookback

* docs(inc2): final pin of SHA f0766b3b65... in lookback

* docs(inc2): ultimate pin of head deaa1a1e3fe0...

* docs(inc2): final pin of head ec1b5f32...

* docs(inc2): pin head 011e1e17... (the fixes tip)
2026-06-17 06:38:45 -07:00

5.2 KiB

Ratified Recognizer Registry (ADR-0163 Phase D)

The recognizer registry projects accepted exemplar-corpus proposals from the append-only proposal log into a tuple the math candidate-graph consults before refusing on an empty per-statement choice list. It is the connective tissue between Phase C's operator review surface and Phase D/E's admission expansion.

Projection rule

A proposal enters the registry iff all of:

  • source.kind == "exemplar_corpus" (Phase C's source kind)
  • review_state == "accepted" (operator ratification — never agent-side)
  • proposed_chain.recognizer_spec parses as a teaching.recognizer_synthesis.RecognizerSpec

Pending, rejected, withdrawn, and non-exemplar proposals are invisible. Malformed accepted proposals raise RegistryLoadError with the offending proposal_id — silent drops are forbidden.

Registry order is (review_date, proposal_id) ascending — stable across runs.

Match contract

generate.recognizer_match.match(statement, registry) returns at most one RecognizerMatch per call (first-match-wins over registry order). Each per-category matcher is rules-only — no LLM, no embedding, no learned classifier. A module-import test pins the no-ML constraint.

parsed_anchors carry numeric tokens extracted from the statement text, not from the spec. For descriptive_setup_no_quantity, parsed_anchors is the empty tuple by design — the recognizer admits the statement as setup context, contributing no math state.

Narrowness invariant

Per ADR-0163 §Phase C The Synthesis Rule property (b), the recognizer is the narrowest commitment that subsumes the seeds. The matcher inherits that narrowness verbatim:

  • A currency symbol outside the spec's observed_currency_symbols does not match rate_with_currency.
  • A window unit outside observed_window_units does not match temporal_aggregation.
  • A statement with any digit, number word, or indefinite quantifier does not match descriptive_setup_no_quantity.

Widening happens through the corridor — wider exemplar corpus → Phase C synthesis on wider seeds → operator ratifies the wider proposal — never by editing the matcher's permissiveness.

Wiring point (current doctrine — post wrong=0 correction)

generate/math_candidate_graph.py:parse_and_solve consults the registry at the per-statement choice loop, before the existing no admissible candidate for statement refusal.

When the registry recognizes the statement:

  • the per-category injector (generate/recognizer_anchor_inject.inject_from_match) is consulted;
  • if the injector emits one or more CandidateInitial / CandidateOperation that survive admissibility, those candidates are added to the per-sentence choice space exactly as parser output would be;
  • if the injector emits nothing (or all candidates are dropped by later pronoun/lookback guards), the graph refuses explicitly with the reason "recognizer matched but produced no injection for statement: ... (category=...)".

Never silently drop a recognized math statement as "zero state". The historical skip-only rule ("drop it, Cartesian product unchanged") was retired because it admitted incomplete graphs at the problem level (the solver could answer from the remaining statements and produce a number that was not the answer to the full input). The current code and this document treat "recognized + no typed emission" as a refusal case. Old skip-only language appears only in historical notes below.

Empty registry → the guard is a no-op and the pre-registry refusal behavior is preserved byte-identically.

Ratification boundary (ADR-0161 §5)

The agent does not ratify the live proposal log. Phase D tests build a synthetic in-memory RatifiedRecognizer tuple from the Phase C pending proposals' content (tests/_phase_d_fixture.py). The matcher and candidate-graph wiring exercise the same RecognizerSpec bytes the operator will later ratify, with zero modification to teaching/proposals/proposals.jsonl.

The operator's ratification path is the existing core teaching review <proposal_id> --accept --review-date <YYYY-MM-DD> — no new CLI surface lands with Phase D.

Phase E / D.2 follow-up (historical note)

Early Phase D wiring (the registry + skip-only guard) was intentionally "skip-only by construction" so that adding recognizer categories could not regress wrong=0. That doctrine was corrected once the "recognized but uninjected → incomplete graph" hazard was understood (see the explicit refusal branch and comments in math_candidate_graph.py and the ADR-0167 / Brief 11 lineage).

Current follow-up work (Workstream A Inc 2 and successors) adds the per-category injectors that turn parsed_anchors for rate_with_currency (and later categories) into grounded CandidateOperation / Rate / apply_rate primitives that the existing solver already knows how to execute. When an injector is present and emits, the statement contributes real solver state; when it cannot, refusal (not silent drop) is the outcome.

Historical skip-only descriptions are preserved only as "rejected behavior" markers in this document and in code comments. Grep for the old phrases on the active source surfaces should surface only such markers.