Commit graph

4 commits

Author SHA1 Message Date
Shay
4570c2c70e
feat(comprehension): operational lexicon loader for en_core_math_v1 (ADR-0164 §Decision §1) (#325)
Implements generate/comprehension/lexicon.py: loads per-category source
files from en_core_math_v1/lexicon/*.jsonl (full schema including aliases),
verifies manifest checksum against compiled lexicon.jsonl for pack integrity,
and provides O(1) case-folded surface lookups. Module-level cache keyed on
(path, mtime_ns, sha256) avoids redundant I/O.

Exports: LexiconEntry, Lexicon, LexiconLoadError, load_lexicon(), lookup().
MappingProxyType over internal dicts prevents callers from mutating cached state.
29 tests cover load, checksum, category completeness, alias resolution,
mutual-exclusion detection, determinism, and cache identity.
2026-05-26 20:08:27 -07:00
Shay
1a78e36e69
feat(comprehension): lexeme primitive registry (ADR-0164.1) (#324)
Adds generate/comprehension/lexeme_primitives.py with the eight seed
primitives specified by ADR-0164.1:

  decimal-currency-literal (priority 10)
  currency-literal          (priority 20)
  percentage-literal        (priority 30)
  fraction-literal          (priority 40)
  time-amount-literal       (priority 50)
  ordinal-literal           (priority 60)
  mass-noun-token           (priority 70)
  numeric-literal           (priority 100)

LexemePrimitive and LexemeMatch are frozen/slots dataclasses. scan()
runs primitives in priority order and returns the first hit wrapped in
a MappingProxyType over sorted-key extracted_values for canonical-bytes
stability. All patterns use explicit space characters ([ ]?, [- ]?) not
\s so the ADR-0165 compliance invariant holds.

55 tests cover: construction invariants, canonical fires (each
primitive on its own example), overlap precedence ($18.00, 1/2, 50%),
refusal on Tina/empty/verbs, determinism, sorted-key stability, and
the ADR-0165 compliance smoke test.
2026-05-26 20:03:39 -07:00
Shay
957e7c6642
feat(comprehension): split ComprehensionState into ProblemReadingState + SentenceReadingState (ADR-0164.3) (#323)
Reconciles the #321 skeleton with ADR-0164.3's two-level state model.

Changes:
  - Renames ComprehensionState → SentenceReadingState (backward-compat alias
    kept; existing callers need not change)
  - Adds 7 new fields to SentenceReadingState (all defaulted so existing
    construction still compiles):
      frame, pending_quantities, pending_entity_ref, pending_verb,
      token_index, lookback (≤8 entries, validated), partial_frame_payload
  - Introduces SentenceFrame (Literal), VerbReference, AppliedCategory,
    FramePayload (stub, frame_kind validated)
  - Adds ProblemReadingState (outer, problem-scoped) with all 7 fields
    per ADR-0164.3 table order, no defaults (explicit construction required)
  - Introduces PartialInitialPossession and PartialOperation (nullable
    precursors to ADR-0115 types), PronounResolution
  - Adds READER_REFUSAL_REASONS (11-member frozenset, closed/ADR-tracked)
    and ReaderRefusal dataclass with reason validation
  - Adds to_canonical_bytes() standalone function implementing
    ADR-0164.3 §Canonical-bytes rules: sort keys, omit None, Decimal→str;
    handles ProblemReadingState, SentenceReadingState, ReaderRefusal
  - SentenceReadingState.canonical_bytes() kept backward-compatible
    (original 5 fields, null for None) — existing pinned-bytes tests pass
  - 47 tests: all original tests pass; new tests cover ProblemReadingState
    construction, determinism gate, sensitivity gate, ReaderRefusal
    construction and every READER_REFUSAL_REASONS entry

Refs: #320 (ADR-0164.3), #321 (comprehension-state-skeleton)
2026-05-26 19:54:17 -07:00
Shay
6a4fcc8b36
feat(comprehension): add ComprehensionState skeleton (#321) 2026-05-26 19:32:22 -07:00