docs(ADR-0127+0128): exhaustive units pack + sibling numerics pack

ADR-0127 expanded from minimum-viable to EXHAUSTIVE:
- 13 dimensions (7 base + 6 derived via dimension algebra)
- ~150 unit lemmas covering imperial+metric for length/time/mass/
  volume/money/temperature/area, with full irregular-plural set
- Multi-word structural rules (square <length>, <length>/<time>)
- ~25 container nouns, ~6 rate connectors, ~15 symbol/affixes
- Conversion graph EXHAUSTIVE within each dimension (~80 edges)
- Temperature edges carry affine offset field (F/C/K)
- Ratification adds: exhaustive-coverage gate, NIST/ISO provenance,
  dimension-algebra closure

ADR-0128 NEW: en_numerics_v1 sibling pack
- Exhaustive cardinals (0..20 + tens + magnitudes) + compound rule
- Exhaustive ordinals (1st..31st covers calendar/position)
- Fractions (named + symbol + slash-form + mixed numbers)
- Multipliers (double/triple/twice/half + N-times rule)
- Quantifiers with semantic_type (indefinite -> refusal preserves
  wrong==0)
- Comparison anchors migrated from math_roundtrip.py hard-coded
  tables (ratifiability)
- Number-format regexes (digits/decimals/percentages/fractions/
  mixed) with test corpus
- Cross-pack consistency check with en_units_v1 fraction symbols

Joint exit gate: train sample re-run with both packs mounted.
If passed -> sealed holdout. If missed -> real Path-B trigger
(full deterministic design failed).
This commit is contained in:
Shay 2026-05-23 06:48:22 -07:00
parent fde62d713d
commit 6e7766f66e
2 changed files with 439 additions and 31 deletions

View file

@ -67,43 +67,95 @@ obligations.** **No new exit gates beyond ADR-0126's** (`correct ≥
10/50, wrong == 0` on the train sample, this time with units pack
in scope).
### Pack content (`en_units_v1`)
### Pack content (`en_units_v1`) — EXHAUSTIVE SCOPE
Structure mirrors `en_arithmetic_v1`: `lexicon.jsonl` +
`manifest.json` + `glosses.jsonl` + a self-sealing
`.mastery_report.json` ratification artifact.
`manifest.json` + `glosses.jsonl` + `conversions.jsonl` + a
self-sealing `.mastery_report.json` ratification artifact.
**Dimension classes** (entry_id prefix `en-units-dim-`):
| Dimension | Notes |
|-----------|-------|
| `count` | Discrete countable nouns (apples, kids, boxes, books) |
| `length` | feet, inches, miles, yards, meters, centimeters |
| `time` | hours, minutes, days, weeks, months, years |
| `mass` | pounds, ounces, kilograms, grams |
| `money` | dollars, cents (anchored by `$` / `¢` symbols) |
| `temperature` | degrees, Fahrenheit, Celsius |
| `volume` | gallons, quarts, cups, liters |
| Dimension | Canonical unit | Notes |
|-----------|---------------|-------|
| `count` | (dimensionless) | The no-unit dimension |
| `length` | foot | imperial + metric families |
| `time` | minute | seconds through millennia |
| `mass` | pound | imperial + metric |
| `money` | cent | with currency-symbol attachments |
| `temperature` | °F | affine scales (F/C/K/R) |
| `volume` | cup | cooking + liquid + dry |
| `area` | sq ft | **derived** = length × length |
| `speed` | mph | **derived** = length / time |
| `density` | lb/cu ft | **derived** = mass / volume |
| `frequency` | Hz | **derived** = 1 / time |
| `wage` | $/hour | **derived** = money / time |
| `unit_price` | $/item | **derived** = money / count |
**Unit lemmas** (entry_id prefix `en-units-unit-`):
Derived dimensions are encoded *compositionally* via a
`dimension_algebra.jsonl` table. The parser recognizes
"miles per hour" structurally as `length-unit + per +
time-unit`; it does NOT need a `mph` primitive. Reduces unit
combinatorics from O(unit²) to O(unit + connector).
Each unit binds to exactly one dimension. Carries `singular`,
`plural`, and `symbol` (where applicable: `$`, `°`). Initial pack
targets ≤ 60 lemmas — bounded by GSM8K-frequent vocabulary, not
NIST-complete coverage.
**Unit lemmas** (entry_id prefix `en-units-unit-`) — exhaustive
for grade-school math word problems (~150 entries):
**Container nouns** (entry_id prefix `en-units-container-`):
| Dimension | Lemmas |
|-----------|--------|
| length | inch/in, foot/ft, yard/yd, mile/mi, millimeter/mm, centimeter/cm, meter/m, kilometer/km |
| time | second/sec/s, minute/min, hour/hr/h, day, week, month, year/yr, decade, century, millennium |
| mass | ounce/oz, pound/lb, ton, gram/g, kilogram/kg, milligram/mg, metric_ton/tonne |
| volume | teaspoon/tsp, tablespoon/tbsp, cup, pint/pt, quart/qt, gallon/gal, milliliter/mL, liter/L |
| money | cent/¢, dollar/$, plus optional foreign: euro/€, pound_sterling/£, yen/¥, peso |
| temperature | degree/°, Fahrenheit/°F, Celsius/°C, Kelvin/K |
| area | sq_inch, sq_foot, sq_yard, sq_mile, sq_meter, sq_kilometer, acre, hectare |
| count | (open class — head-noun grammar; pack does not enumerate, but provides count-dimension stamp for any unmatched noun in countable position) |
Closed set: `box`, `bag`, `pack`, `basket`, `batch`, `dozen`,
`group`, `set`, `case`, `pair`, `pile`, `stack`. Each declares
its canonical syntax (`<count> <container> of <content>`) and an
optional `default_size` (only `dozen=12`, `pair=2`, others null).
Each unit carries `singular`, `plural`, `symbol`, `dimension`,
`is_canonical_for_dimension` (bool). Plural forms include the
irregulars: foot/feet, child/children (count), person/people
(count), mouse/mice (count), goose/geese (count),
fish/fish (count), sheep/sheep (count), deer/deer (count),
woman/women (count), man/men (count), leaf/leaves,
knife/knives, life/lives.
**Rate connectors** (entry_id prefix `en-units-rate-`):
**Multi-word units** (entry_id prefix `en-units-multiword-`):
Closed set: `per`, `an`, `every`, `each`. Each declares the
template `<money|count> <rate_connector> <time|count>` it
participates in.
Structural composition rules (not lexicon enumeration):
- `square <length-unit>` → area (sq_ft, sq_meter, sq_mile, …)
- `cubic <length-unit>` → volume_derived
- `<length-unit> per <time-unit>` → speed
- `<money-unit> per <count-noun-or-unit>` → unit_price or wage
- `<mass-unit> per <volume-unit>` → density
The pack ships the *patterns*, not every concrete combination —
the parser composes them at recognition time.
**Container nouns** (entry_id prefix `en-units-container-`) —
closed set (~25):
`box`, `bag`, `pack`, `basket`, `batch`, `dozen`, `group`,
`set`, `case`, `pair`, `pile`, `stack`, `bunch`, `bundle`,
`carton`, `crate`, `jar`, `can`, `bottle`, `cup` (when
container, not volume-unit — context-disambiguated), `plate`,
`bowl`, `tray`, `sack`, `container`, `gross` (=144).
Each declares syntax `<count> <container> of <content>` and
optional `default_size` (only `dozen=12`, `pair=2`, `gross=144`,
others null).
**Rate connectors** (entry_id prefix `en-units-rate-`) — closed
set (~6):
`per`, `an`, `every`, `each`, `by`, `in` (as in "every 3 days").
Each declares the dimension-algebra template it participates in.
**Symbol/affix table** (entry_id prefix `en-units-symbol-`):
`$`, `¢`, `°`, `°F`, `°C`, `%`, `½`, `¼`, `¾`, `⅓`, `⅔`, `⅛`,
`⅜`, `⅝`, `⅞`. Each maps to its lexical lemma. Cross-linked to
`en_numerics_v1` (sibling pack, ADR-0128) for fraction symbols.
**Substance qualifiers** (entry_id prefix `en-units-substance-`):
@ -128,11 +180,17 @@ where `to_unit = ratio × from_unit`. The pack ratifies the
{"edge_id":"en-units-conv-006","from":"hours","to":"days","ratio":0.04166666667,"dimension":"time"}
```
**Temperature edges carry an affine `offset` field** because
temperature conversion is not pure multiplication:
`°F = 1.8 × °C + 32`. Schema: `{"from","to","ratio","offset",
"dimension"}`. Multiplicative-only edges have `offset=0`.
The graph is **the conversion table as data**, not as code. The
solver consults it; the parser doesn't need to. Initial coverage
target: complete edges for `length`, `time`, `money`, `mass`,
`volume` between GSM8K-frequent unit pairs — bounded by NIST SI
tables (closed set), not free expansion.
solver consults it; the parser doesn't need to. Coverage target
is EXHAUSTIVE within each dimension's lemma set (~80 edges
total: every pair of in-pack units in the same dimension must be
reachable). Bounded by NIST SI tables (closed set); ratification
proves completeness — see invariants below.
### Why conversions matter
@ -177,10 +235,22 @@ just lexicon well-formedness:
`minutes` for time, `dollars` for money, etc.). All
canonicalization routes through the canonical unit; this
bounds the shortest-path computation to O(1) lookups.
5. **Exhaustive coverage gate** — every dimension's lemma set
must produce a fully connected conversion subgraph. Adding
a unit lemma without conversion edges (or with edges that
don't connect to the canonical) is rejected at ratification.
This prevents the inventory from silently fragmenting.
6. **NIST/ISO provenance** — every conversion ratio cites a
source via `provenance_id` (NIST SP 811 for SI, ISO 4217 for
currency, etc.). Unsourced ratios are rejected.
7. **Dimension algebra closure** — every derived-dimension entry
(speed, density, wage, area) has both decomposed forms
present (e.g., for `speed`: at least one length-unit and one
time-unit are present in their respective dimensions).
These invariants live in `tests/test_adr_0127_pack_ratification.py`
and run at every pack-change PR — the conversion graph cannot
ship broken.
ship broken or incomplete.
### Parser integration

View file

@ -0,0 +1,338 @@
# ADR-0128 — `en_numerics_v1` Pack
**Status:** Proposed (scope-only; sibling to ADR-0127)
**Date:** 2026-05-23
**Author:** CORE agents + reviewers
**Depends on:**
- ADR-0126 (candidate-graph parser + round-trip filter)
- ADR-0127 (`en_units_v1` — units pack, cross-referenced for
shared symbol/affix table)
**Supersedes:** none
**Sibling:** ADR-0127. Both packs are jointly required for the
units-aware candidate-graph parser to produce a fair empirical
result on the GSM8K-math lane. Either pack failing ratification
independently blocks the joint exit gate.
---
## Context
ADR-0127 introduces `en_units_v1` (dimensions, units, conversions,
containers, rate connectors). That pack handles the *physical /
economic* substrate. It does NOT handle the *linguistic forms* a
quantity can take in English math word problems:
- Cardinal number words ("seventeen", "two hundred and fifty")
- Ordinal words ("third", "twentieth")
- Fractions as words ("two-thirds", "a quarter") and as symbols
(`½`, `¾`)
- Multiplier words ("twice", "triple", "half")
- Quantifiers ("all", "some", "each", "every", "many", "few")
- Comparison anchors ("more", "fewer", "less", "additional")
- Numeric format strings ("1,000", "1.5", "1/2", "75%")
Today these are scattered:
- `WORD_NUMBERS` table hard-coded in `generate/math_roundtrip.py`
(one through twelve only)
- `_COMPARE_VERB` / comparison anchors hard-coded in
`generate/math_parser.py`
- Fraction handling absent
- Percentage handling absent
- Multi-digit number-word parsing ("two hundred") absent
The GSM8K train sample shows the cost: e.g., "Half of the kids
are going to soccer camp" (refused — `half` not handled),
"3/4 of its temperature" (refused — `3/4` not recognized as
fractional form). These are unrecognized *quantity forms*, not
unrecognized *units*; they sit on a different lexical axis from
ADR-0127's substrate.
A units pack without a numerics pack solves "5 feet + 8 inches"
but still refuses "half a foot." Both packs are needed for the
architecture to get a fair empirical reading on the train sample.
---
## Decision
Add a `language_packs/data/en_numerics_v1/` ratified semantic pack
that exhaustively encodes the English linguistic forms of
quantities. Parser changes are minimal — most pack content
replaces today's scattered hard-coded tables with ratified
lookups.
### Pack content (`en_numerics_v1`) — EXHAUSTIVE SCOPE
Structure mirrors `en_arithmetic_v1` / `en_units_v1`:
`lexicon.jsonl` + `manifest.json` + `glosses.jsonl` +
`.mastery_report.json`.
**Cardinal number words** (entry_id prefix `en-num-card-`):
EXHAUSTIVE for grade-school range:
- `zero` through `twenty` (21 entries)
- Tens: `thirty`, `forty`, `fifty`, `sixty`, `seventy`, `eighty`,
`ninety` (7 entries)
- Compound rule: `<tens-word>-<unit-word>` (e.g., "twenty-one",
"ninety-nine") — structural composition, not enumerated
- `hundred`, `thousand`, `million`, `billion` (4 entries)
- Compound rule: `<N> hundred [and <M>]`, `<N> thousand
[<conjunction> <M>]` — structural composition
Each entry carries `surface`, `numeric_value` (int), `morphology`
(`cardinal`).
**Ordinal number words** (entry_id prefix `en-num-ord-`):
EXHAUSTIVE for grade-school range (1st31st covers most
calendar / position references in word problems):
- `first` through `tenth` (10 entries — irregular morphology)
- Suffix rule: `<cardinal>th` for 1131 (eleventh, twelfth,
thirteenth, …, thirty-first) — structural composition with
spelling-irregularity table (`fifth` not `fiveth`, `eighth` not
`eightth`, `twelfth` not `twelveth`, etc.)
- `twentieth`, `thirtieth`, `hundredth`, `thousandth` (4 entries)
Each entry carries `surface`, `position` (int), `morphology`
(`ordinal`).
**Fraction words** (entry_id prefix `en-num-frac-`):
- Named fractions: `half` (½), `third` (⅓), `quarter` (¼),
`fifth` (⅕), `sixth` (⅙), `seventh` (⅐), `eighth` (⅛),
`ninth` (⅑), `tenth` (⅒), `sixteenth` (¹/₁₆) — ~10 entries
- Compound rule: `<cardinal>/<ordinal-as-denominator>` (e.g.,
"two-thirds", "three-quarters") — structural composition
- Article-bound: `a half`, `a quarter`, `a third` resolve to
the same numeric value as the bare form
- **Symbol cross-link** to `en_units_v1` symbol table for `½`,
`¼`, `¾`, `⅓`, `⅔`, `⅛`, `⅜`, `⅝`, `⅞`
Each entry carries `surface`, `numerator` (int), `denominator`
(int), `decimal_value` (float — `1/3 = 0.333…` etc.),
`morphology` (`fraction`).
**Multiplier words** (entry_id prefix `en-num-mult-`) — closed
set:
`double` (×2), `triple` (×3), `quadruple` (×4), `quintuple` (×5),
`twice` (×2), `thrice` (×3), `half` (×0.5 — also a fraction
word; both entries cross-reference). Plus structural rule
`N times` for arbitrary integer multipliers.
**Quantifiers** (entry_id prefix `en-num-quant-`) — closed set:
`all`, `none`, `some`, `both`, `each`, `every`, `many`, `few`,
`several`, `most`, `any`, `no`, `single`.
Each declares its `semantic_type`: `total`, `empty`, `partial`,
`paired`, `distributive`, `indefinite`. The parser uses this to
decide whether the quantifier yields a determinate value
(`both = 2`, `single = 1`) or is undeterminate and triggers
refusal (`some`, `many`, `few` — refuse rather than guess; this
preserves `wrong == 0`).
**Comparison anchors** (entry_id prefix `en-num-compare-`):
Migrated from `generate/math_roundtrip.py`'s hard-coded
`COMPARE_ADDITIVE_ANCHORS` / `COMPARE_MULTIPLICATIVE_ANCHORS`.
- Additive: `more`, `fewer`, `less`, `additional`, `extra`,
`missing`, `remaining`
- Multiplicative: `twice`, `thrice`, `times`, `half`, `double`,
`triple`, `quadruple`, `third`, `quarter`
Each cross-references its multiplier or fraction entry where
applicable (avoiding duplicate truth).
**Number formats** (entry_id prefix `en-num-format-`) —
structural rules, not enumeration:
- Digit groups: `1,000` (US thousand separator), `10,000`,
`1,000,000``(\d{1,3})(?:,\d{3})+` → strip commas, parse as int
- Decimals: `1.5`, `3.14`, `0.25``\d+\.\d+` → parse as float
- Slash-fractions: `1/2`, `3/4`, `7/8``(\d+)/(\d+)`
parse as `Fraction`
- Mixed numbers: `1 1/2`, `2 3/4``(\d+) (\d+)/(\d+)`
`whole + numerator/denominator`
- Percentages: `75%`, `1.5%``\d+(?:\.\d+)?%` → divide by 100
- Negative numbers: `-3`, `-0.5` — leading minus (rarely in
grade-school but cheap to include)
Each format rule declares its regex pattern, `parser_function`
(by name), and `output_type` (`int`, `float`, `Fraction`).
### Cross-references with `en_units_v1`
- Fraction symbols `½`, `¼`, `¾`, etc. appear in both packs.
Single source of truth lives in `en_numerics_v1`;
`en_units_v1` symbol-affix table contains a *reference* entry
pointing to the numerics pack via `cross_pack_id`. Ratification
cross-pack consistency check (see ADR-0127 ratification
invariants and below) verifies the references resolve.
- Percentage (`%`) cross-references between packs because it's
both a numeric format and a dimensionless modifier.
- Multipliers (`double`, `twice`) cross-reference because they
also appear as comparison anchors in `en_units_v1`'s parser-
consumed register.
### Ratification invariants (`en_numerics_v1`-specific)
1. **Cardinal exhaustiveness** — every English cardinal 0..20,
every "tens" form, every magnitude word
(hundred/thousand/million) present.
2. **Ordinal exhaustiveness** — every English ordinal 1st..31st
present (covers month days + most grade-school position
references).
3. **Fraction exhaustiveness** — every named fraction 1/2 through
1/10 present + the irregular set (sixteenth, thirty-second).
4. **Cross-pack consistency** — every fraction-symbol entry in
`en_units_v1` resolves to a fraction entry in
`en_numerics_v1`. Verified by joint pack-mount ratification.
5. **Quantifier semantic-type completeness** — every quantifier
lemma carries a `semantic_type` from the closed set
`{total, empty, partial, paired, distributive, indefinite}`.
6. **Format regex test corpus** — each format rule has a
minimum of 10 positive + 10 negative test strings in
`tests/test_adr_0128_numeric_formats.py`. Format ambiguity
(e.g., `1.000` could be `1` or `1000`) is *refused* per
`wrong == 0`, not guessed.
### Parser integration
1. **`generate/math_roundtrip.py`** — replace hard-coded
`WORD_NUMBERS` and `COMPARE_*_ANCHORS` tables with calls into
`en_numerics_v1` loader. Behavior preserved (current entries
are a subset); future extensions land in the pack, not the
source.
2. **`generate/math_candidate_parser.py`** — new value-token
normalization helper `normalize_value_token(token) -> Quantity
| Fraction | None` that consults `en_numerics_v1` format
rules. Handles "two-thirds" → `Fraction(2,3)`, `75%``0.75`,
`1,500``1500`, etc.
3. **Round-trip filter (`roundtrip_admissible`)** — the existing
`_value_grounds` helper is extended to ground word-forms via
the numerics pack loader (current hard-coded `WORD_NUMBERS`
capped at "twelve" widens to the full cardinal table).
4. **Quantifier-driven refusal** — when a candidate's value-token
resolves to an `indefinite` quantifier (`some`, `many`, `few`),
the parser emits NO candidate for that sentence. Empty list →
refusal at decision rule. Preserves `wrong == 0`.
### What ADR-0128 explicitly does NOT do
- Does NOT replace `en_units_v1`. The packs are siblings.
- Does NOT introduce new operation kinds. ADR-0126 + 0122 cover
the operation grammar; this pack supplies value-token forms.
- Does NOT model number-system alternates (Roman numerals,
Chinese numerals, etc.) — out of scope for English math
problems.
- Does NOT model implicit numbers ("a dozen" / "a few") as
determinate values — `a dozen = 12` is encoded via the
container's `default_size` (ADR-0127's container entry, not
here); `a few` is `indefinite` and triggers refusal.
- Does NOT add new exit gates. Joint exit criterion with
ADR-0127 (re-run train sample with both packs mounted).
---
## Invariants preserved / added
| Invariant | Preserved or added | How |
|-----------|--------------------|-----|
| `wrong == 0` | Preserved | Indefinite quantifiers trigger refusal; format ambiguity triggers refusal |
| `trace_hash` byte-equality | Preserved | Pack lookup is deterministic |
| Pack-binding (ADR-0114a #10) | Reinforced | Value-token resolution cites `en-num-*` entry_ids in `SolutionTrace.provenance` |
| Round-trip admissibility | Strengthened | Word-form numbers ground via pack lookup, not regex enumeration |
| Replay equivalence | Preserved | Same pack version → same lookup result |
| `versor_condition(F) < 1e-6` | Untouched | No runtime field changes |
| Manifest checksum SHA-256 of bytes-on-disk | Required | Mastery report self-seals |
## Exit criterion
**Joint with ADR-0127** — both packs ratified, both mounted,
re-run train sample:
```
correct >= 10 / 50 on evals/gsm8k_math/train_sample/v1/cases.jsonl
wrong == 0
```
**If passed:** run sealed holdout once, freeze in ADR-0127/0128
joint results.
**If missed:** **Path-B trigger** — full deterministic design
(candidate-graph + units + numerics) failed to move GSM8K.
Demote benchmark, re-target math expert promotion.
---
## Alternatives considered
### A. Fold numerics into `en_units_v1` as one big pack.
Rejected per ADR-0127 discussion: domain-distinct lexicons,
risk-isolation, future i18n composability.
### B. Keep hard-coded tables in `math_roundtrip.py` / `math_parser.py`.
Rejected — violates pack-binding (ADR-0114a #10); future
language packs (Spanish, German) would have to duplicate the
hard-coded tables; ratification is impossible.
### C. Use a third-party number-parsing library (`word2number`,
`num2words`, etc.). Rejected — same opacity / non-auditability
critique as ADR-0127 alt B.
### D. Ship only cardinals + fractions; defer quantifiers /
ordinals / formats. Rejected — quantifiers (`half`, `some`) are
*specifically* what the GSM8K train sample refuses on. Partial
pack delivers partial empirical signal — same trap ADR-0127
addresses re: scope mismatch.
---
## Implementation plan (proposed sub-phases)
| Phase | Module | Description |
|-------|--------|-------------|
| 0128.1 | `language_packs/data/en_numerics_v1/` | Pack content (lexicon + manifest + glosses + mastery report) |
| 0128.2 | `language_packs/loader.py` | `lookup_cardinal`, `lookup_ordinal`, `lookup_fraction`, `lookup_quantifier`, `match_number_format` |
| 0128.3 | `generate/math_roundtrip.py` | Replace hard-coded `WORD_NUMBERS` + `COMPARE_*_ANCHORS` with pack-backed lookups |
| 0128.4 | `generate/math_candidate_parser.py` | `normalize_value_token` helper; quantifier-driven refusal |
| 0128.5 | `tests/test_adr_0128_*.py` | Pack ratification (exhaustiveness gates) + parser integration + format regex corpus |
| 0128.6 | Joint with ADR-0127.7 | Re-run train sample with both packs mounted |
Regression gates:
- `core test --suite smoke -q`
- `core test --suite math -q`
- `core test --suite packs -q`
- ADR-0126 P3+P4 tests
- ADR-0127 pack ratification
## PR checklist (when proposing for merge)
```
What capability did this add?
→ Exhaustive English linguistic-form ontology for quantities;
sibling substrate to en_units_v1 (ADR-0127).
What invariant proves the field remains valid?
→ Wrong==0 preserved via indefinite-quantifier refusal +
format-ambiguity refusal; cardinal/ordinal/fraction
exhaustiveness ratification gates.
Which CLI suite/eval proves the lane?
→ smoke + math + packs + joint train_sample re-run with
ADR-0127.
Did this avoid hidden normalization, stochastic fallback,
approximate recall, unreviewed mutation?
→ Yes. Pack lookup is deterministic; lexicon is bounded and
ratified; no learned tokenization; no fuzzy matching.
If it touches user input, what trust boundary was enforced?
→ Pack file paths validated via safe_pack_id (ADR-0051).
Format regexes carry test corpora documenting accepted vs
rejected inputs; user input is matched against ratified
patterns only.
```