core-labs/core
core-labs/core
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

feat(trackb): S1 symbolic structure-mapping vertical slice (off-serving) #85

Merged
core-labs merged 2 commits from feat/trackb-symbolic-sme-s1 into main 2026-07-20 02:33:47 +00:00
Conversation 0 Commits 2 Files changed 13 +2063
core-labs commented 2026-07-20 02:01:52 +00:00
Owner
Copy link

Summary

Track B Increment 1 for ADR-0252: the first vertical slice of symbolic structure-mapping comprehension (Gentner SME), after §5 ruled geometric SME NO-GO (conformal_procrustes presupposes correspondence). Closes the geometric path; PRs #83 and #84 already closed with the supersession note.

What this PR adds (off-serving only)

  • Role-predicate schema + deterministic MathProblemGraph → role-graph converter
  • Explicit S1 canonical skeleton: compare(b, a, k) ∧ total(a, b, sum) (role vars only)
  • Blind symbolic mapper: match + binding or refuse — gold labels never enter the mapper
  • Solve via existing corridors: classical solve+verify and multi-register Hamiltonian certificate; emit only when both agree
  • Isolated scoring labels under evals/structure_mapping/scoring/
  • Measure script + unit tests + research report with command-backed outputs

What this PR does not do

  • No S2–S4 generalization
  • No serving-reader / organ cutover
  • No sealed-test touch
  • No geometric Procrustes revival

Measured results (see docs/research/trackb-s1-slice-2026-07-20.md)

  • S1 right-for-right-reason on holdout organ cohort: correct=5 wrong=0 refused=0 with per-case derivation
  • False positives: holdout non-S1 layer is vacuous (only 5 graphs parse on the split — claimed honestly, not as separability). Real FP tests: synthetic 0/5, other-corpus non-compare graphs 0/135
  • Surface-variant: rename entities + change numbers → re-parse still maps S1 with new bindings (answer 66)
  • Vs organ: side-by-side match on all 5 S1 cases

[Verification]

  • uv run pytest tests/test_trackb_s1_structure_mapping.py -q — 16 passed
  • uv run core test --suite smoke -q — 176 passed (~130s) before and after (byte-identical serving path; serving module diff = 0)
  • Holdout baseline unchanged by construction: correct=5 wrong=0 refused=495 (no live-path edits)
  • Measure commands in research doc / scripts/measure_trackb_s1_slice.py

STOP → RULING

Human reviewer verifies this slice against the code before S2–S4 or any serving change.

## Summary Track B Increment 1 for ADR-0252: the first vertical slice of **symbolic** structure-mapping comprehension (Gentner SME), after §5 ruled geometric SME **NO-GO** (`conformal_procrustes` presupposes correspondence). Closes the geometric path; PRs #83 and #84 already closed with the supersession note. ### What this PR adds (off-serving only) - Role-predicate schema + deterministic `MathProblemGraph` → role-graph converter - Explicit S1 canonical skeleton: `compare(b, a, k) ∧ total(a, b, sum)` (role vars only) - Blind symbolic mapper: match + binding or refuse — **gold labels never enter the mapper** - Solve via existing corridors: classical `solve`+`verify` **and** multi-register Hamiltonian certificate; emit only when both agree - Isolated scoring labels under `evals/structure_mapping/scoring/` - Measure script + unit tests + research report with command-backed outputs ### What this PR does **not** do - No S2–S4 generalization - No serving-reader / organ cutover - No sealed-test touch - No geometric Procrustes revival ### Measured results (see `docs/research/trackb-s1-slice-2026-07-20.md`) - **S1 right-for-right-reason** on holdout organ cohort: `correct=5 wrong=0 refused=0` with per-case derivation - **False positives**: holdout non-S1 layer is **vacuous** (only 5 graphs parse on the split — claimed honestly, not as separability). Real FP tests: synthetic 0/5, other-corpus non-compare graphs **0/135** - **Surface-variant**: rename entities + change numbers → re-parse still maps S1 with new bindings (answer 66) - **Vs organ**: side-by-side match on all 5 S1 cases ## [Verification] - `uv run pytest tests/test_trackb_s1_structure_mapping.py -q` — 16 passed - `uv run core test --suite smoke -q` — 176 passed (~130s) before and after (byte-identical serving path; serving module diff = 0) - Holdout baseline unchanged by construction: `correct=5 wrong=0 refused=495` (no live-path edits) - Measure commands in research doc / `scripts/measure_trackb_s1_slice.py` ## STOP → RULING Human reviewer verifies this slice against the code before S2–S4 or any serving change.
core-labs added 1 commit 2026-07-20 02:01:56 +00:00 
ADR-0252 §5 geometric SME is NO-GO; this is Track B Increment 1.
Adds role-predicate conversion from MathProblemGraph, S1 canonical skeleton,
blind symbolic mapper (match/refuse + binding), and solve via classical
verify plus multi-register certificate. Research report and holdout measures
included. Serving reader unchanged; no S2–S4 generalization.
core-labs added 1 commit 2026-07-20 02:10:17 +00:00 
Mapper required only that total include {a,b}; rebuild-from-binding then
dropped third entities and could emit a certified wrong answer (e.g. 15
instead of 115). Require exact total part-set {a,b}, refuse any contain
outside reference a, and agree with original-graph classical solve before
emit. Adds multi-entity refuse unit tests.
core-labs merged commit f6fd5030bb into main 2026-07-20 02:33:47 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: core-labs/core#85
No description provided.
Delete branch "feat/trackb-symbolic-sme-s1"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?