ADR-0164.1 amendment: replace name-whitelist entity admission with a
universal lexeme primitive that recognizes any capitalized token as a
proper noun. The gender-coded name lists are demoted from admission
criterion to enrichment-only lookup. A name outside the curated lists
still admits cleanly with gender="unknown" — ADR-0164.2's pronoun
resolution rules handle the unknown case via single-salient fallback
or refuse with ambiguous_pronoun_referent.
Universal at the primitive layer: the new proper_noun_token primitive
is domain-agnostic. It sits in the shared PRIMITIVE_REGISTRY and is
available to every current and future reader (math, narrative,
code-comment, multi-lingual). The math reader is its first consumer.
Pattern: ^[A-Z][A-Za-z'-]*[a-z][A-Za-z'-]*$
- requires capitalized first letter
- requires ≥1 lowercase letter (rejects all-caps acronyms)
- allows internal apostrophes (O'Brien) and hyphens (Mary-Anne)
- matches "Tina", "Bob", "Marnie", "McDonald" — rejects "TINA",
"123", "$5.00" (those go to their own primitives)
Sentence-initial lookup-first dispatch (lifecycle._classify):
- At token_index == 0: lookup() first, skipping proper_noun_gender_*
categories (treated as not-found so the primitive can fire). If
lookup misses, primitive scan picks up novel names. Inverts the
question from "is this a name?" to "is this a known common word?"
- At token_index > 0: primitive-first with UNIT_CATEGORY_TOKEN ceding
to operational lexicon for currency_unit_noun overrides.
Lexicon rename (per-category source files):
- proper_noun_entity_female.jsonl -> proper_noun_gender_female.jsonl
- proper_noun_entity_male.jsonl -> proper_noun_gender_male.jsonl
Compiled lexicon.jsonl: rename the two semantic_domain tags; drop
"marnie" (was only in proper_noun_entity_female, now absent from
the gender-coded sources). Net: 208 -> 207 entries. New manifest
checksum: 1fb9b0d790258736267d528e8e8a2436ce88b9ce690805fe2813ba077861ba2a
New helper gender_of_proper_noun(surface, lexicon) returns
Literal["female","male","neuter","unknown"] — pure enrichment lookup,
never gates admission.
Measurement (reader_phase1_plus_proper_noun_delta.json):
- pre-primitive baseline: correct=3 refused=47 wrong=0
- post-primitive measurement: correct=3 refused=47 wrong=0
- No regression on wrong=0
- No net admission increase observed in this train-sample harness;
the architectural value is for future text outside the curated
gender lists (Sonnet's #332 expanded those to cover GSM8K names).
Tests:
- test_lexeme_primitives.py: registry count 8 -> 9, proper_noun_token
fires + variants (Bob, Marnie, McDonald, O'Brien, Mary-Anne),
numeric/all-caps refusals, numeric-literal still wins overlap on "123"
- test_reader_question_frame.py: 5 new tests for sentence-initial
dispatch + unknown-gender pronoun resolution + novel-name admission
via primitive (Zelda)
- test_en_core_math_v1_pack.py: category counts updated; mutual-exclusion
between gender_female and gender_male preserved; total 208 -> 207
- test_lexicon.py: category list + lookup assertion updated to renamed
proper_noun_gender_female
- test_proper_noun_primitive_universality.py: new test module asserting
domain-agnostic property of the primitive
Validation:
- pack + lexicon + primitive tests: 147 passed
- reader + universality tests: 22 passed
- smoke lane: 67 passed
Closes the engine_state question by leaving those files untracked
(repo discipline: runtime artifacts never enter PRs).
Refs ADR-0164.1 amendment, ADR-0164.2 §EntityRegistry, ADR-0165
§Legitimate uses (the new primitive passes the three-question test).
Adds the three lifecycle functions for the incremental compositional
reader per ADR-0164.3 §Lifecycle API:
- begin_sentence(problem_state, source_text_offset) -> SentenceReadingState
- apply_word(sentence_state, problem_state, word) -> SentenceReadingState | ReaderRefusal
- end_sentence(sentence_state, problem_state) -> ProblemReadingState | ReaderRefusal
Phase 1 scope is question sentences only. The update rules for the
question_frame live in a single readable table (_QUESTION_FRAME_RULES);
statement-side frames (initial_state_frame, operation_frame,
descriptive_frame) refuse with a Phase-2 diagnostic.
The five Brief-8 GSM8K target question sentences (0007, 0017, 0027,
0036, 0043) produce valid QuestionTargetSlot outputs end-to-end.
_interface_stubs.py provides a thin, functional surface for the
lexeme-primitive scanner (Brief 6) and lexicon loader (Brief 7) so
this PR does not block on them. The stub honours the en_core_math_v1
pack entries and adds a closed Phase-1 supplemental vocabulary marked
for fold-in to the pack once Briefs 6/7 land.
Tests cover determinism (byte-equal canonical bytes), the five GSM8K
target sentences with expected (entity, unit_class, kind) triples,
all token-level and sentence-level refusal modes, and lifecycle
invariants (registry preservation, sentence_index advance).
Stacked on feat/state-two-level-split (PR #323) per ADR-0164.3
§Naming — state types live in state.py.
Shay