Exhaustive English linguistic-form ontology for quantities:
cardinals (0..20 + tens + magnitudes + compound rule), ordinals
(1st..31st + decade/magnitude forms), named fractions (1/2..1/10
+ sixteenth/thirty-second) + symbol forms (½ ¼ ¾ ⅓ ⅔ ⅛ ⅜ ⅝ ⅞),
multipliers (double/triple/twice/half), quantifiers with
semantic_type (indefinite triggers refusal at parse time —
preserves wrong==0), comparison anchors migrated for
ratifiability, number-format regexes with positive/negative
corpora.
Loader API in language_packs/numerics_loader.py (sibling module
to be merged into main loader after Gemini's ADR-0127 loader
lands, to avoid concurrent merge conflict).
Ratification invariants gated: cardinal/ordinal/fraction
exhaustiveness, quantifier semantic-type closed set, format-regex
test corpora (10+ positive/negative per format, ambiguity
refused), manifest checksums = SHA-256 of bytes-on-disk,
self-sealing mastery report.
Cross-references en_units_v1 (Gemini ADR-0127): fraction symbols
authoritative here; en_units_v1 symbol-affix table will point to
these entries.
No parser changes (deferred to 0128.3-0128.6). No train-sample
re-run (joint exit gate with ADR-0127 runs after both packs land).
Total: 130 lexicon entries across 7 kinds.
Lanes: smoke 67/0/0, packs 6/0/0, ADR-0128 suite 243/0/0.
Wraps existing math pipeline (parser -> solver -> verifier) against
PR #159's 50-case train sample. Emits deterministic report.json with
per-case verdicts. CLI exit code reflects exit criterion
(correct >= 10 AND wrong == 0).
Baseline against current parser: 0 correct / 0 wrong / 50 refused.
This baseline is the inner-loop gradient signal for ADR-0126's
candidate-graph parser (in flight on feat/adr-0126-candidate-graph).
Registers tests/test_adr_0126_train_sample_runner.py under
'core test --suite math' so the wrong == 0 invariant becomes a hard
CI gate per ADR-0114a Obligation #4 (refuse rather than confabulate).
Depends on PR #159 (gemini/adr-0126-train-sample). Rebase onto main
after #159 lands.
Closes 5 of 8 surface-form gaps Gemini identified in Task 5 on the
99 comparison-bearing sentences my ADR-0123 substrate currently refuses
in the sealed holdout. Pure regex / parser-state work — no graph,
solver, verifier, or pack changes; preserves wrong==0 discipline.
Expansions (in safety order)
- Group 8 (verb): comparison verbs widened from {has} to {has, have,
had, gets, get, got, takes, take, took, buys, buy, bought}.
"lost"/"won" excluded — they semantically invert direction.
- Group 3 (word-form numbers): _WORD_NUMBERS {one..twelve} accepted
wherever digit values are. _parse_compare_number helper centralizes
the dispatch.
- Group 4 (ellipsis / implicit unit): unit slot made optional in
multiplicative patterns (solver already infers unit from reference
state); added "twice|N times as much", "twice|N times the
number/amount of <unit>" variants.
- Group 1 (subjects / references): actor/reference slots widened from
bare proper noun to {proper noun, "the <noun>" collective,
pronoun}. Pronouns resolve via state.last_singular_subject; missing
prior subject raises ParseError (no silent emission with empty
actor). New _resolve_compare_entity helper canonicalizes "The boys"
/ "the boys" to the same entity string.
- Initial-possession + question patterns widened symmetrically so
"the X" subjects round-trip end-to-end:
- _INITIAL_HAS_RE accepts "the <noun>" subject + has/have +
digit-or-word value
- _Q_ENTITY_RE accepts "the <noun>" entity + do/does auxiliary
- _Q_TOTAL_RE now tried first (specificity-ordered) so "do they
have" doesn't get greedily matched as entity="they"
Deferred (per Gemini Task 5c recommendation)
- Group 2 (age): needs new "years_old" attribute model
- Group 5 (nested): needs compound y = mx + c solver operation
- Group 7 (currency): low volume (2 cases), defer
- Group 6 (compound multi-clause "and" split): scoped out of this
PR to keep the wrong==0 risk profile tight; safer to land after
Gemini Task 6 confirms current expansion doesn't introduce
misparses on the sealed set
Test coverage
- 507 existing math + ADR-0122 + ADR-0123 tests pass (no regressions)
- 16 ad-hoc smoke cases pass (3 baseline + 3 Group 8 + 3 Group 3 +
3 Group 4 + 3 Group 1 + 2 refusal guards + 1 rate cross-check)
- smoke suite 67/67, algebra suite 82/82 green
Expected sealed lift
- Gemini Task 5 catalog projected ~65/90 strict-comparison-only
cases unblocked by the 5 included groups (71/99 comparison-bearing
sentences). Empirical sealed measurement pending Gemini Task 6;
PR will be updated with the actual correct/wrong/refused bucket
counts once measured.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ADR-0123-parser-comparison-phrasing as the **surface increment** on
PR #155's substrate (commit c9bd5d4). Closes the last architectural
gap in the comparison-phrasing class: before this commit, the
substrate's solver evaluated comparison problems successfully but
realize() crashed with `unknown operation_kind 'compare_additive'`
when asked for show-your-work prose.
Substrate (PR #155) already shipped:
- `Comparison` typed graph operand
- `compare_additive` / `compare_multiplicative` operation kinds
- parser patterns for the four canonical surfaces
(N more / N fewer / twice / N times / half)
- solver + verifier wiring + pack lemmas
(en-arith-006 compare_additive, en-arith-007 compare_multiplicative)
This surface adds:
- `_compare_additive_sentence(step)` rendering `direction='more'|'fewer'`
- `_compare_multiplicative_sentence(step, entity_units)` rendering
`direction='times'|'fraction'`
- two new branches in `_step_sentence` dispatch
- `_step_sentence` signature widened with optional `entity_units` map
(derived once-per-trace in `realize()` from `graph_initial_state`)
- ADR-0123-parser-comparison-phrasing.md (~15 invariants, substrate
+ surface decomposition rationale, multi-construction barrier
inheritance)
- 26 invariants pinned across canonical surfaces, plurality
independence, byte-determinism, refusal discipline, and
backwards-compatibility with the pre-comparison realizer templates
End-to-end pipeline now operates on all four canonical comparison
shapes:
parse_problem(
"Alice has 5 apples. Bob has 3 more apples than Alice. "
"How many apples does Bob have?"
) -> solve() -> realize().as_prose() ->
"Alice has 5 apples. Bob has 3 more apples than Alice, giving Bob
a total of 8 apples. Bob has 8 apples."
Measurement (this PR):
- 26/28 direct ADR-0123 tests pass; 2 skipped (CORE_HOLDOUT_KEY)
- `core eval cognition` byte-identical: 100/100/100/100
- ADR-0118 stepped-realizer templates re-render byte-identically
- Substrate measurements continue to hold
Honest non-result: sealed `correct_rate` stays at 0/1319. The
realizer cannot create matches the parser refuses; the multi-
construction barrier the substrate ADR documented holds at the
surface layer too. Cumulative lift signal expected only after the
3rd/4th foundational class lands (per ADR-0121's revised
sequencing). `wrong == 0` holds by construction — realizer only
renders successful traces.
Pre-existing failure noted (not introduced by this PR):
`tests/test_adr_0085_gloss_aware_cause.py::test_flag_off_metrics_byte_identical`
fails on substrate base (c9bd5d4) without these changes — an
ADR-0085 cognition baseline drift unrelated to the realizer.
Second parser-expansion ADR after ADR-0122 rate/per-unit. Adds the
comparison algebra substrate (Comparison dataclass + compare_additive /
compare_multiplicative operation kinds + parser patterns + solver /
verifier / pack lemmas) mirroring the substrate-only / lift-deferred
pattern ADR-0122 established.
Substrate
- Comparison(reference_actor, delta: Quantity|None, factor: float|None,
direction: Literal[more,fewer,times,fraction]) frozen dataclass with
direction-discriminated delta/factor enforcement and self-reference
refusal at the Operation boundary
- compare_additive + compare_multiplicative operation kinds admitted in
VALID_OPERATION_KINDS; Operation.operand widened to Quantity|Comparison
with kind-discriminated type enforcement; entity-set validation extended
to cover Comparison.reference_actor
- Parser: _COMPARE_ADDITIVE_RE (more/fewer/less), _COMPARE_TWICE_RE,
_COMPARE_N_TIMES_RE, _COMPARE_HALF_RE happy-path patterns + 5
refusal patterns (ambiguous 'N times more', age comparisons,
combined-with-aggregation, nested additive+multiplicative); inserted
before _try_initial so leading 'has <N>' shape is not greedily
consumed as initial possession with unit='more'/'fewer'
- Solver: _apply_compare_additive (refuses on missing reference state,
overwrite, negative result); _apply_compare_multiplicative (refuses
on missing reference, ambiguous multi-unit reference, overwrite);
unit comes from delta.unit (additive) or reference's unique unit
(multiplicative)
- Verifier: _verify_compare_additive_step + _verify_compare_multiplicative_step
byte-equal replay; tamper-detects after_value, direction, factor
- Pack: en-arith-006 compare_additive + en-arith-007 compare_multiplicative
lemmas + glosses; SHA-256 checksums refreshed; manifest 1.0.0 -> 1.1.0;
provenance tagged adr-0123:comparison_extension:2026-05-23
Measurement (honest; from Gemini empirical sealed run on parallel surface
branch with this substrate)
- Sealed GSM8K correct_rate: 0/1319 (substrate matches zero real cases
alone). Validates the ADR-0122 multi-construction barrier prediction:
comparison constructions in GSM8K rarely appear alone — they bind with
rate (ADR-0124), percentage (ADR-0125), aggregation (ADR-0126), or
conditional ('if') clauses. First lift signal requires composition.
- Sealed GSM8K wrong: 0 (load-bearing positive claim; ADR-0114a
Obligation #4 preserved across all 1,319 sealed problems)
- Regression safety: 0 — all 913 non-comparison cases continue to
refuse exactly as before (refused_parser), no greedy consumption by
the new comparison patterns
Surface-form catalog (from Gemini Task 2 survey, see ADR doc) covers
6 primary forms across Groups A/B/C; Groups D (age), E (combined with
aggregation), F (nested additive+multiplicative) refused as out-of-scope
with typed ParseError naming the missing companion ADR.
Branch isolation
- Landed via dedicated worktree (feat/adr-0123-substrate from origin/main)
after a file-race on the shared umbrella branch. Companion surface +
scaffolding (realizer, ADR doc, tests, README) lands separately as
feat/adr-0123-surface; orchestrator merges both into the umbrella
feat/adr-0123-comparison-phrasing.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Documents the Phase 5 GSM8K-math substrate completion across 7 narrative docs.
All 8 sub-phases of ADR-0119 (5.1 through 5.8) have landed on main; ADR-0114a's
10 anti-overfitting proof obligations are all discharged for the gsm8k_math lane.
Key facts surfaced in each doc:
- CORE-original public split: 150/150 correct, 0 wrong, 0 refused
- Real GSM8K test (sealed holdout): 0 correct, 0 wrong, 1319 refused
- Adversarial suite: 38 cases x 12 families, 0 wrong
- Depth curve: flat at 1.0 across depths 1-8 on public split
- Frontier baselines: Claude 3.5 Sonnet 96.4%, GPT-4 92.0%, Gemini 1.5 Pro 90.8%
- New lane shape gsm8k_capability_shape in LANE_SHAPE_REGISTRY
- New operational pack en_arithmetic_v1 (5 lemmas)
- ADR-0120 (first expert promotion contract) is the next gate
Docs updated: docs/PROGRESS.md, docs/capability_roadmap.md, docs/runtime_contracts.md,
docs/Whitepaper.md (§XIII), docs/Yellowpaper.md (gsm8k_capability_shape formal spec),
README.md, docs/decisions/README.md (current frontier).
No code changes. No new ADRs.
First worked attempt at promoting a domain under the ADR-0120
expert promotion contract. The contract refuses honestly.
Gate evaluation against live state:
ADR-0114a obligations: 10 of 10 pass
ADR-0120 contract-level gates:
audit_passed_holds ✓
correct_rate (public) ✓ 150/150 = 1.0
correct_rate (sealed) ✗ 0/1319 = 0.0 < 0.60 floor
signed_expert_claim ✗ (no entry, downstream of correct_rate)
Decision: mathematics_logic NOT promoted; stays at audit-passed.
Substantive blocker: parser grammar covers 0/1319 of real GSM8K.
What this proves
- The contract is genuinely falsifiable. ADR-0120 §"Threshold
rationale" deliberately set the floor above current measurement
so the first attempt would defer honestly. Same load-bearing
pattern as ADR-0107 → ADR-0110 for audit-passed.
- Wrong-zero discipline holds against real GSM8K (the load-
bearing positive claim). CORE refuses every problem outside
its grammar without confabulating on a single one.
What unlocks the promotion
Multi-ADR parser-expansion arc lifting sealed-GSM8K correct_rate
from 0.0 to ≥ 0.60. Each construction class (rate/comparison/
percentage/time-modal/etc.) ships as its own scoped ADR with:
- parser+solver+verifier+realizer extensions
- re-measurement on sealed holdout
- ADR-0118a OOD re-measurement (no surface-feature regression)
- ADR-0125 perturbation re-measurement (no invariance regression)
- ADR-0119.5 adversarial re-measurement (no new misparses)
Honest-fitting discipline: every lift is graded on the anti-
overfit obligations BEFORE the correct_rate change counts.
Tests: 6/6 with CORE_HOLDOUT_KEY; 4/6 + 2 skipped without (matches
ADR-0119.7 seal discipline).
This deferral demonstrates the expert tier's promotion machinery
is load-bearing — the gate has refused at least once before any
domain reaches it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Defines the `expert` ledger tier (sixth status above `audit-passed`)
and the composition gate that governs every future promotion.
The gate composes ALL TEN ADR-0114a obligations plus three contract-
level checks:
- audit_passed predicate must hold
- correct_rate >= 0.60 on public AND sealed holdout splits
- signed expert_claims entry whose digest reproduces byte-equal
Numeric thresholds (load-bearing choices documented with rationale):
- correct_rate floor: 0.60 ("advanced" — above weak open-source LLMs,
forces real architecture work, raisable later)
- depth-curve flatness ε: 0.05 (accuracy(N) ≥ accuracy(d1) · 0.95^(N-1))
- drift tolerance from promotion: ±0.02
Documents the post-ADR-0120 sequence in Open Candidate Directions:
Phase 1: ADR-0121 first worked math promotion (likely deferral)
Phase 2: parser-expansion arc to lift sealed-GSM8K correct_rate
Phase 3: math expert promotion succeeds
Phase 4: second domain = symbolic_logic (60-70% of math substrate cost;
ProofWriter / PrOntoQA benchmark; same machinery class)
Phase 5: third domain = high-stakes refusal-centric (medical or legal;
wedge-sharpener; needs two prior successes first)
Phase 7: open candidate — multi-reviewer threshold signing for expert
No code lands with this ADR. Implementation ships under ADR-0120a
to keep the contract change reviewable independently.
The first promotion attempt (ADR-0121) will likely defer on the
correct_rate gate — current measurement is 0/1319 on real GSM8K.
That deferral IS the contract working as designed; same load-
bearing pattern ADR-0107 → ADR-0110 demonstrated for audit-passed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The 1,319 GSM8K test cases are now sealed at
evals/gsm8k_math/holdouts/v1/cases.jsonl.age, age-encrypted to the
ADR-0119.1 recipient. Plaintext never touched disk in the working
tree; only ciphertext is committed.
First honest CORE-vs-real-GSM8K measurement
cases_total: 1319
correct: 0
wrong: 0 ← ADR-0114a Obligation #4 holds against external corpus
refused: 1319
overall_pass: True
Zero confabulation. Parser refuses what it can't grammar-handle; the
"wrong == 0" discipline survives the move from CORE-original cases
to a real public benchmark. The 0/1319 correct rate is the truthful
gap that ADR-0120's threshold work will quantify.
What landed
scripts/seal_gsm8k_test.py
- Loads GSM8K via datasets.load_dataset("openai/gsm8k", "main")
- Strips worked-solution prose; extracts final-answer integer/float
after "####" (handles "2,125" → 2125 thousands-separator)
- Reads recipient from docs/holdout_recipients.txt (single repo key
per ADR-0119.1)
- Encrypts via pyrage; writes only ciphertext
- Refuses to overwrite test path with train-derived seal
evals/gsm8k_math/runner.py
- Empty expected_unit (sentinel) skips unit-comparison; grades on
answer value alone. Required because GSM8K answers carry no unit
structurally. wrong-zero discipline preserved.
tests/test_adr_0119_7_sealed_gsm8k.py — 6 invariants:
1. sealed file present + age-formatted
2. no plaintext companion files (sibling-leak guard)
3. decrypted JSONL matches documented schema
4. runner against decrypted suite produces wrong==0
5. tests skip (not fail) when CORE_HOLDOUT_KEY unset
6. case ids match "gsm8k-test-NNNN" pattern
Defensive gitignore: plaintext patterns under
evals/gsm8k_math/holdouts/v1/ are explicitly excluded.
ADR-0114a obligation roll-up
10/10 discharged for the gsm8k_math lane:
#1 ✓ sealed-holdout (fab_control + GSM8K test)
#2..#10 ✓ as before
Phase 5 status: 5.1..5.7 done; 5.8 in flight (PR #149). After 5.8
merges, ADR-0120 (first expert promotion contract) becomes
feasible.
Test plan
- pytest tests/test_adr_0119_7_sealed_gsm8k.py with CORE_HOLDOUT_KEY → 6/6
- pytest without CORE_HOLDOUT_KEY → 3 pass + 3 skip
- core test --suite smoke -q → 67/67
- CLAIMS.md regenerated (no diff)
- HF token NEVER in repo (saved at ~/.cache/huggingface/token, mode 600)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Audit follow-ups from #145/#146 merge review. Five small fixes; no
behavior change on the green path, but failure modes are now explicit
rather than silent.
ADR-0119.6 depth_curve.py
- Add DepthCurveError typed exception
- Raise on case_id missing from lane_report (was: silent → "refused")
- Raise on depth >= 9 (was: silent new bucket key)
- Two new tests pin both refusals
- Removed stale sys.path hack at module top
ADR-0119.4 frontier-baseline tests
- Assert comparison_v1.json's core_measurement reports wrong == 0
(the load-bearing differentiator named in the disclaimer; a
tampered file with wrong > 0 was previously syntactically valid
and would have passed all old assertions)
- Assert frontier citations are dated 2023 or later (freshness
guard; older citations should be refreshed before ADR-0120
gates anything for `expert` promotion)
Tests
- tests/test_adr_0119_6_depth_curve.py: 7 → 9
- tests/test_adr_0119_4_frontier_baseline.py: 5 → 7
- 29/29 across runner + depth-curve + frontier suites; 67/67 smoke
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Umbrella ADR for Phase 5 of ADR-0114. Decomposes the eval-lane work
into eight sub-phases (5.1..5.8) that ship under their own ADRs.
Sub-phase scope:
5.1 — sealed-holdout encryption (one lane, fabrication_control)
5.2 — CORE-original GSM8K-style corpus (dev + public, 200 cases)
5.3 — lane runner (parser → solver → verifier → realizer)
5.4 — frontier-baseline comparison (Obligation #7)
5.5 — adversarial generation; misparse rate zero (Obligation #8)
5.6 — depth-curve measurement harness (Obligation #6, measurement)
5.7 — sealed GSM8K test as the holdout (Obligation #1, lane-side)
5.8 — overall lane gate; new gsm8k_capability_shape
Roadmap only — no code lands with this ADR. Each sub-phase ships
independently and discharges a specific ADR-0114a obligation.
Critical invariant: dev + public splits are CORE-original; the real
GSM8K test set enters only via the encrypted holdout under 5.7.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 4 of the ADR-0114 GSM8K-math roadmap. Consumes a SolutionTrace
and emits one sentence per step plus setup + answer sentences. Pure
function; same trace → byte-equal RealizedTrace.
What landed
generate/math_realizer.py
- realize(initial_state, trace) -> RealizedTrace
- Frozen RealizedTrace dataclass with canonical_bytes() + as_prose()
- Per-kind sentence rules (add / subtract / transfer / multiply×2 /
multiply×3 / multiply-general / divide)
- Singular/plural surface rule matches parser canonicalization
- Typed RealizerError on unrecognized step kinds
tests/test_math_realizer.py — 60 cases pinning five invariants:
1. All 50 dev-set cases realize without error
2. Determinism: byte-equal RealizedTrace across two calls
3. Setup sentence count == initial_state count
4. Step sentence count == operation count
5. Answer sentence contains the resolved value + unit
ADR-0114a obligation discharge update
ADR-0118 hardens determinism (#9) across a third layer (realizer)
and makes #3 / #10 human-inspectable via the prose surface. No
obligation is directly newly discharged by ADR-0118; it's substrate
for ADR-0119 GSM8K eval lane.
Round-trippability of the prose through the parser is explicitly
out of scope for this phase. The trace is the verifiable artifact
(ADR-0117); the prose is human-readable documentation.
Tests: 60 new realizer cases; 546 total green across realizer +
parser + solver + verifier + OOD; 67/67 smoke green.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
