# ADR-0164.1 — Lexical Primitive Set Scope (seed registry for `en_core_math_v1`) **Status:** Proposed **Date:** 2026-05-26 **Author:** Shay **Anchor:** [[thesis-decoding-not-generating]] **Parent:** [ADR-0164 — Incremental Comprehension Reader](./ADR-0164-incremental-comprehension-reader.md) **Companion:** [ADR-0165 — Regex Scope Rule](./ADR-0165-regex-scope-rule.md) **Resolves:** ADR-0164 §Open question #1 ("Lexical primitive set scope") --- ## Context — why this sub-ADR exists ADR-0164 specifies the reader and leaves the *exact bootstrap primitive set* open (§Open question #1). ADR-0165 specifies the *rule* that bounds what may become a primitive (lexeme-level, closed orthographic shape) but does not enumerate the set. This ADR closes the gap. It enumerates the seed primitive registry that ships with the ADR-0164 Phase 1 PR, fixes the registry record schema, documents overlap precedence (the only place primitives can interact at recognition time), and records the temptations that are explicitly **not** admitted, so future authors don't relitigate them. The reader's Phase 1 acceptance gate (see ADR-0164 §Phasing) depends on this set being closed before scan-time. Adding a primitive after Phase 1 follows the population corridor in ADR-0165 §Population (contemplation → proposal → review). --- ## Decision The seed lexeme-primitive registry for `en_core_math_v1` contains the **eight** primitives below. Each is a closed orthographic shape per the ADR-0165 three-question test (§Code-review test). Each carries the schema fields in §Registry record schema. Overlap precedence between primitives is fixed by §Overlap precedence and is the only behavior that may differ between "primitive A fires" and "primitive A fires given primitive B already matched the same span." Population beyond this seed set rides the ADR-0165 corridor and is out of scope for this ADR. --- ## Registry record schema Each primitive is a frozen record with the following fields. Field order is canonical (used by the canonical-bytes serialization that feeds `trace_hash` per CLAUDE.md §Runtime Surface Contract). | Field | Type | Meaning | |---|---|---| | `name` | kebab-case string, unique | Registry key. Stable across rounds. Forms the trace-evidence label when the primitive fires. | | `pattern` | regex source string (Python `re` flavor) | The orthographic recognizer. Must be anchorable to a single token or contiguous token-class run (ADR-0165 §Rule). No `.*` across word combinations. No `\s+VERB\s+` constructions. | | `emits` | enum: `QUANTITY`, `ORDINAL`, `UNIT_CATEGORY_TOKEN` | The reader category emitted on match. The category is what the reader's composition rules consume; the surface form is discarded after extraction. | | `extracted_fields` | typed tuple `(name: type, ...)` | The structured payload produced. Types are `int`, `Decimal`, `str` from a closed enum, or `tuple[int, int]` for compound shapes (e.g. fractions). Reader composition is allowed to read these fields and only these fields. | | `provenance` | string `"ADR-"` or `"teaching-ratified #"` | Audit trail. Seed primitives in this ADR are stamped `"ADR-0164.1"`. New primitives stamped by the HITL queue at acceptance time (ADR-0161). | | `priority` | integer ≥ 0 (lower wins) | Tiebreaker when two primitives match the same span. The overlap-precedence table (§Overlap precedence) fixes seed priorities; new primitives declare priority at proposal time and the operator ratifies it. | The registry is a tuple of records, ordered by `priority` ascending then `name` ascending (stable ordering for replay equivalence). --- ## Seed primitive set (n = 8) Each entry below is a populated record. Patterns are shown as Python `re` source; the runtime compiles them with `re.IGNORECASE` unless otherwise noted, anchored to a single token / contiguous span (the reader is responsible for token-boundary alignment; the regex itself does not consume surrounding whitespace). ### 1. `decimal-currency-literal` ```yaml name: decimal-currency-literal pattern: \$(\d+)\.(\d{2})\b emits: QUANTITY extracted_fields: (whole: int, cents: int, unit_class: str = "currency") provenance: ADR-0164.1 priority: 10 ``` Rationale: `$18.00`, `$1.50`. The two-decimal-place currency form is its own shape because (a) the cents field is structurally significant (rounding semantics), (b) it must beat both `currency-literal` and `numeric-literal` on the same span. Closed orthographic class: "dollar-sign, integer, dot, exactly two digits." ### 2. `currency-literal` ```yaml name: currency-literal pattern: \$(\d+(?:\.\d+)?)\b emits: QUANTITY extracted_fields: (value: Decimal, unit_class: str = "currency") provenance: ADR-0164.1 priority: 20 ``` Rationale: `$18`, `$1.5`, `$1000`. Currency notation with arbitrary (non-cents-shaped) decimal. Beats `numeric-literal` because the `$` prefix carries the unit. Does *not* cover `$1.5M`, `$18/hour`, `USD 18` — see §Rejected temptations. ### 3. `percentage-literal` ```yaml name: percentage-literal pattern: (\d+(?:\.\d+)?)\s?% emits: QUANTITY extracted_fields: (value: Decimal, unit_class: str = "ratio") provenance: ADR-0164.1 priority: 30 ``` Rationale: `25%`, `7.5 %`. The `%` glyph is a closed unit suffix. Single optional space allowed because percent signs frequently appear detached (`7 %`). The space inside the pattern is a single character, not a `\s+` across structure — the run is still one orthographic shape. ### 4. `fraction-literal` ```yaml name: fraction-literal pattern: (\d+)\s?/\s?(\d+)\b emits: QUANTITY extracted_fields: (numerator: int, denominator: int, unit_class: str = "fraction") provenance: ADR-0164.1 priority: 40 ``` Rationale: `1/2`, `3 / 4`. The slash-separated integer pair is a genuine closed shape. Optional single space on either side of the slash. Reader composes "1/2 of " through composition rules, not through extending this pattern. ### 5. `time-amount-literal` ```yaml name: time-amount-literal pattern: (\d+)[-\s]?(hour|minute|day|week|month|year|second)s?\b emits: QUANTITY extracted_fields: (value: int, unit: str, unit_class: str = "time") provenance: ADR-0164.1 priority: 50 ``` Rationale: `3 hours`, `30-minute`, `2 days`, `1 week`. The "number-plus-time-unit-noun" form is the canonical closed time-amount shape. Unit set is a closed enum; the reader treats `hour`/`hours`/ `hour-` uniformly via the singular-form extraction. Does *not* cover `an hour` (article + bare unit, that's grammar — see §Rejected temptations), and does *not* cover `3 hours per week` (rate phrase, grammar). ### 6. `numeric-literal` ```yaml name: numeric-literal pattern: \d+(?:\.\d+)?\b emits: QUANTITY extracted_fields: (value: Decimal, unit_class: str = "pending") provenance: ADR-0164.1 priority: 100 ``` Rationale: `18`, `1.5`, `12`. The bare number. `unit_class=pending` signals to the reader that a unit attachment is expected from a downstream token (a `count_unit_noun`, `currency_unit_noun`, `time_unit_noun`, etc., from the operational lexicon — ADR-0164 §Operational lexicon). Highest numeric `priority` ensures all other numeric-bearing primitives win the overlap. ### 7. `ordinal-literal` ```yaml name: ordinal-literal pattern: (first|second|third|fourth|fifth|sixth|seventh|eighth|ninth|tenth)\b emits: ORDINAL extracted_fields: (rank: int) provenance: ADR-0164.1 priority: 60 ``` Rationale: `first`, `second`, `third`. Closed list of English ordinal spellings 1–10. Extending the set to `eleventh`–`twentieth` or to numeric ordinals (`1st`, `2nd`) is a teaching-corridor decision, not a silent extension of this ADR. Extracted `rank` is the integer the ordinal denotes. ### 8. `mass-noun-token` ```yaml name: mass-noun-token pattern: (money|profit|interest|income|savings|cost|amount|total)\b emits: UNIT_CATEGORY_TOKEN extracted_fields: (lemma: str, unit_class: str = "currency-mass") provenance: ADR-0164.1 priority: 70 ``` Rationale: ports `_MASS_NOUNS` from `generate/math_candidate_parser.py` into a primitive form. The set is closed (8 lemmas) and orthographic; each lemma is a single token recognized by spelling. The reader composes "how much " via composition rules over `question_open` + `question_continuous_qty` + `UNIT_CATEGORY_TOKEN`, not via extending this pattern across the question stem. > Boundary note: this primitive is at the edge of the rule. A > `mass-noun-token` is a closed set of single tokens, which satisfies > ADR-0165's "contiguous token-class run." It is included as a > primitive (rather than as an operational-lexicon entry under > ADR-0164) because the reader's question-frame composition needs a > structured category at scan time, not a lexicon hit during the > composition pass. If Phase 1 measurement shows this is better > modeled as a lexicon category, supersede this entry. --- ## Overlap precedence A token span can be matched by more than one primitive. The reader runs primitives in `priority` order (lower first), commits the first match that consumes a non-empty span, and skips the consumed span for subsequent primitives. The pairwise table below is the *normative* record of every overlap relevant to the seed set. Each row is a deliberate decision; review of a new primitive must extend this table. | Span shape | Candidate primitives | Winner | Rationale | |---|---|---|---| | `$18.00` | `decimal-currency-literal`, `currency-literal`, `numeric-literal` | `decimal-currency-literal` | Two-decimal cents form is structurally distinct (rounding). Currency-literal would lose the cents semantics; numeric-literal would lose the `$`. Lowest priority (10) wins. | | `$18` | `currency-literal`, `numeric-literal` | `currency-literal` | The `$` prefix carries the unit class. Numeric-literal alone would emit `unit_class=pending` and force the reader to recover the currency unit from context, which is exactly the kind of grammar inference the comprehension layer is meant to do *with* typed evidence, not despite it. | | `25%` | `percentage-literal`, `numeric-literal` | `percentage-literal` | The `%` suffix carries the unit class (ratio). Same rationale as currency. | | `3 hours` | `time-amount-literal`, `numeric-literal` (matching `3`) | `time-amount-literal` | The time-unit noun is part of the closed shape; allowing `numeric-literal` to consume `3` first would orphan `hours` and force the reader to compose a time quantity through the lexicon — bypassing a typed primitive that already exists. The span-commit rule (longest valid match at lowest priority among ties) handles this: time-amount's pattern consumes the full `3 hours` span before numeric-literal's anchor matches. | | `1/2` | `fraction-literal`, `numeric-literal` (matching `1`, then `2`) | `fraction-literal` | The slash makes the two integers a structured pair, not two independent quantities. | | `first` | `ordinal-literal`, (nothing else) | `ordinal-literal` | Listed for completeness; no overlap, but the priority is set above `numeric-literal` so future `ordinal-numeric-literal` (`1st`, `2nd`) — if and when ratified — fits cleanly into the same slot. | | `money` | `mass-noun-token`, (operational-lexicon `currency_unit_noun` per ADR-0164) | `mass-noun-token` | Lexical-primitive scan runs before lexicon lookup (ADR-0164 §Deterministic reader, step 1 → step 2). The primitive emits a typed `UNIT_CATEGORY_TOKEN` carrying `unit_class=currency-mass`, which the lexicon entry would also produce — but the primitive's emission is what the question-frame composition expects. | The precedence is **fixed**; a primitive proposal that would change an existing row in this table requires explicit supersession of this ADR, not a silent priority bump. --- ## Rejected temptations The patterns below are *not* primitives and must not be added as primitives. Each is a grammar template per ADR-0165 §Code-review test and belongs in the reader's composition rules over the operational lexicon. ### Rejected #1 — Rate phrase `$18/hour`, `$18 per hour` A naive primitive might be `\$(\d+(?:\.\d+)?)\s?/\s?(hour|day|week)`. - **What does it match?** A composition of three role-distinct elements: a currency amount, a connector (`/` or the word `per`), and a time unit. Not "one piece of orthographic material" but "a way three pieces of material combine to mean *rate*." - **Closed-set test:** No. The connector class is open (`/`, `per`, `an`, `each`, `every`, `for each`, …). The denominator class is open (any unit noun, not just time). Recognizing rate via regex enumerates surface forms. - **Novel-phrasing test:** Refuses `$18 every hour`, `$18 for each hour worked`, `eighteen dollars an hour`. The refusal is brittle on the same underlying meaning — the diagnostic of a grammar template. Correct home: reader composition rule `[QUANTITY{currency}] + [distributive_modifier|"/"|"per"] + [time_unit_noun]` → `RATE` operation. Currency and time-amount primitives feed the composition; the rate semantics emerge there. ### Rejected #2 — Compound entity `her three friends`, `Tina and Marion` A naive primitive might be `(she|he|her|his|their)\s+(\d+|two|three|four|five|several|a few)\s+(friends|sisters|brothers|cousins)`. - **What does it match?** A possessive determiner, a count, and a relational noun — three role-distinct elements with an open combinatorial product. Pure grammar. - **Closed-set test:** No. Possessive determiners are a small but context-dependent set; counts include numeric literals *and* spelled-out numerals *and* "a few" / "several" / "many" hedges; relational nouns are open-ended (`friends`, `co-workers`, `neighbors`, `kids`, `students`, …). - **Novel-phrasing test:** Refuses `the three friends she invited`, `Tina, Marion, and Jen`, `each of her friends`. Every novel reference shape refuses on the same underlying meaning — group entity binding. Correct home: cross-sentence `ProblemReadingState` (ADR-0164 §Open question #4) plus reader composition rules over `entity_pronoun`, `numeric-literal`, and a `relational_noun` lexicon category. Group binding is the reader's job, not the primitive layer's. ### Rejected #3 — Question stem `How much money will she earn` A naive primitive might be `How\s+much\s+(money|profit|...)\s+(will |did|does)\s+(she|he|they|it)`. - **What does it match?** A five-role grammar template: question opener + continuous-quantity word + mass-noun + auxiliary + pronoun. This is the canonical example of what ADR-0164 deprecates and ADR-0165 §Forbidden uses cites verbatim. - **Closed-set test:** No on every role except `much` and `she/he/ they/it`. The mass-noun set is reviewable but the rest of the composition opens onto open lexical classes (auxiliaries, modal verbs, perfect/progressive constructions, embedded clauses). - **Novel-phrasing test:** Refuses `How much will she have earned by Friday`, `How much money does Tina end up with`, `How much did it cost him in total`. These are the exact 34/47 refusals ADR-0164 §Context documents. Correct home: the reader's question-frame composition rules, consuming primitive emissions (`UNIT_CATEGORY_TOKEN` from `mass-noun-token`) plus operational-lexicon categories (`question_open`, `question_continuous_qty`, `entity_pronoun`, `accumulation_verb` from ADR-0164 §Operational lexicon). ### Rejected #4 — Compound numeric `1,000`, `1.5M`, `1.5 million` A naive primitive might be `\d{1,3}(,\d{3})+(\.\d+)?` plus a sibling `\d+(\.\d+)?[KMB]\b`. - **What does it match?** A composition between digit groups and scaling tokens. Comma grouping is one shape; magnitude suffixes are another; spelled-out scale words are a third. - **Closed-set test:** Borderline. The comma-grouped form alone (`1,000`, `12,345,678`) is a closed orthographic shape and *could* be ratified as a primitive (`grouped-numeric-literal`) through the ADR-0165 corridor when GSM8K evidence demands it. The `1.5M` and `1.5 million` forms are compositions and must not be folded into the same primitive. - **Why rejected from the seed:** No GSM8K `train_sample/v1` case observed in the ADR-0164 §Context evidence requires it. Adding it speculatively violates ADR-0114a (no surface-form additions without evidence) and the cleanup-as-you-find discipline (don't add what you don't need). Correct home: defer to teaching-corridor ratification once a refusal on a comma-grouped numeric is observed. The `M`/`million` scaling case is composition (numeric + magnitude lexicon entry), not a primitive. --- ## Consequences ### Positive 1. **Phase 1 acceptance has a concrete scan-time vocabulary.** Eight primitives, fully specified, ship with the Phase 1 PR. The reader's step-1 (lexical primitive scan) is now implementable. 2. **Overlap behavior is recorded, not discovered.** Every overlap a reviewer might worry about (`$18.00`, `3 hours`, `1/2`, `25%`) has an explicit winner with rationale. 3. **The "where do I draw the line?" question is settled with examples.** Future authors who want to add a primitive for a rate phrase, compound entity, or question stem have §Rejected temptations as the rejection precedent — no relitigation. ### Negative / tradeoffs 1. **Seed set is small.** Eight primitives won't cover every shape GSM8K throws. That's the point — population is the corridor's job (ADR-0165 §Population). Reader refusals on unknown token shapes are evidence that flows back into the queue. 2. **`mass-noun-token` is at the rule boundary.** Listed honestly in §Seed primitive set with an explicit supersede-if path. Phase 1 measurement will settle whether it belongs here or in the operational lexicon. 3. **Overlap precedence table will grow.** Every new primitive must extend the table. This is the cost of fixed precedence, and it's cheaper than the alternative (silent runtime tiebreaking). --- ## Acceptance criteria for this sub-ADR This ADR moves to **Accepted** when: 1. The seed registry above is materialized in `language_packs/data/en_core_math_v1/lexical_primitives.json` (or the equivalent loader format settled in the Phase 1 PR), one record per entry, fields populated verbatim. 2. The manifest checksum hashes the bytes written to disk (CLAUDE.md §Semantic Pack Discipline). 3. The overlap-precedence table has a pinned regression test — given the seed registry, the eight overlap rows resolve to the declared winner on synthetic minimum-pair inputs. 4. ADR-0164 §Open question #1 is checked off in that ADR's open-questions list when ADR-0164 next ships an update. --- ## Cross-references - **Parent:** ADR-0164 — Incremental Comprehension Reader - **Companion (the rule):** ADR-0165 — Regex Scope Rule - **Anti-overfitting doctrine:** ADR-0114a - **Pack discipline:** CLAUDE.md §Semantic Pack Discipline - **Population corridor:** ADR-0150 (contemplation), ADR-0152 (learning-arc proof), ADR-0155 (CI contemplation runner), ADR-0161 (HITL async queue) - **Anchor:** `[[thesis-decoding-not-generating]]` — the primitive set is a decoder's recognizer table. It enumerates the closed orthographic shapes the reader can pick up *as such*; it does not enumerate sentences.