* feat(evals): add deterministic symbolic equivalence generated corpus
* feat(evals): add symbolic equivalence replay helpers
* feat(evals): load generated symbolic equivalence corpus
* feat(evals): emit symbolic equivalence replay manifest
* feat(symbolic): support multivariable integer polynomials
* feat(symbolic): support exact rational polynomial coefficients
* feat(symbolic): align equivalence API with multivariable normalization
* test(ADR-0131.1.B): reconcile v1 expectations to v1.B scope expansion
The v1.B refactor (univariate int → sparse multivariable Fraction) deliberately
admits multivariable polynomials and constant-denominator division. The v1
dataset and tests pinned the old refusal behavior, so the lane runner reported
wrong=4 and 10 unit tests failed.
Reconcile:
- cases.jsonl: flip sym-eq-v1-0029 ('x+y' vs 'x+1') and sym-eq-v1-0030
('x/2' vs 'x') from expected=refused to expected=not_equivalent; rename
categories to multivariable_distinct / constant_denominator_distinct;
extend provenance with adr-0131.1b:scope-expanded.
- generated_cases.py: split _refusal_cases into scope_expanded (admits)
and templates (still refused); the first two adversarial cases move to
the scope-expanded list with expected=not_equivalent.
- test_math_symbolic_normalizer.py: replace test_undefined_variable and
test_unknown_operator_division with positive scope-expansion tests +
symbolic-denominator refusal; rewrite TestPolynomialInvariants for the
new terms/variables constructor (Polynomial(terms={...}, variables=(...)))
with float-rejection and zero-coef-collapse invariants.
- test_math_symbolic_equivalence.py: TestRefused.test_empty_left reason
string matches new normalizer error; flip multivariable + constant-
denominator cases to NOT_EQUIVALENT; add symbolic-denominator-refused
case; relax canonical_a assertion in test_a_normalizes_b_refuses (engine
now zeroes both on either-side refusal).
- report.json + manifest.json: regenerated; lane PASS 185/185 wrong=0.
Lane invariants reaffirmed by the new tests: wrong==0, refusal-first for
truly out-of-scope inputs (symbolic denominator, transcendental, malformed,
negative exponent), determinism via byte-equal report.
ADR-0131 Benchmark 1 substrate — the primary discriminator for the
mathematics_logic expert promotion under the architecture-aligned
benchmark composite proposed in ADR-0131.
WHAT LANDED:
generate/math_symbolic_normalizer.py
Deterministic univariate polynomial normalizer. Scope: single
variable, integer coefficients, +/-/*/** operators, parens, no
division, no transcendentals. Pipeline: tokenize -> recursive-
descent parse -> expand-and-collect -> canonical string. Refusal
is first-class via SymbolicError; out-of-scope inputs refuse
rather than guess (preserves wrong == 0).
generate/math_symbolic_equivalence.py
check_equivalence(a, b) -> EquivalenceVerdict
Returns EQUIVALENT / NOT_EQUIVALENT / REFUSED with canonical
strings + reason. Compares byte-equal canonical forms.
evals/math_symbolic_equivalence/v1/
cases.jsonl — 30 hand-curated cases across 18 algebraic
identity categories + 2 out-of-scope refusals.
Coverage: commutative, distributive, square +
cube of binomial, difference of squares, FOIL,
collect like terms, zero cancellation, factoring,
exponent combination, unary negation.
runner.py — CLI entry point. Loads cases, builds report,
writes JSON, exits 0/1 on gate pass/fail.
README.md — methodology, scope, dataset categorization,
exit criterion, baseline result.
tests/
test_math_symbolic_normalizer.py — 44 tests covering parser,
algebra primitives,
canonical-form invariants,
and every refusal path.
test_math_symbolic_equivalence.py — 16 tests on the public
check_equivalence API.
test_adr_0131_1_symbolic_equivalence_lane.py
— 8 tests gating the lane:
dataset integrity, exit
criterion, wrong == 0,
determinism (byte-equal
report across runs).
EMPIRICAL RESULT (the lane PASSED):
correct = 30 / 30 (100.0%)
wrong = 0 / 30 (wrong == 0 invariant satisfied)
refused = 0 / 30 (refusals all matched expected)
correct_rate = 1.00
exit_criterion: PASSED (>= 0.95 required)
CONTRAST WITH ADR-0127-0128 GSM8K TRAIN-SAMPLE RESULT (0/0/50):
This is the first benchmark on the mathematics_logic lane where
the architecture's structural strengths fully express. The result
is the empirical inverse of the GSM8K result — and that's
exactly the architecture-benchmark fit ADR-0131 was written to
re-target toward.
REGRESSION: 1033/1033 existing tests green across math + ADR-0126
+ pack ratification + runner. Zero regressions.
SCOPE DISCIPLINE (per ADR-0131.1 v1 plan):
v1 deliberately narrow (univariate, integer, polynomial). Future
ADR-0131.1.B expansions documented in README: multi-variable,
rationals, larger dataset (~500), sealed holdout per ADR-0119.7
pattern.
PARALLEL WORK (per ADR-0131 plan to run all 3 sub-phases concurrently):
- ADR-0131.2: CORE-native teaching-corpus eval (separate PR)
- ADR-0131.3: bounded-grammar word-problem set (separate PR)
These are independent of ADR-0131.1; no shared files, no
cross-PR coordination required beyond final composite gate.
Shay