feat(eval): deterministic relational operator ablation + HE/GRC Logos roadmap

core-labs commented

2026-07-20 04:14:48 +00:00

Owner

Summary

Evidence-led cartography and a narrow Deterministic Relational Operator Ablation for fraction_decrease, plus a separate audit-only Hebrew/Koine Greek Logos pack capability roadmap. No English→ancient-language pseudo-evidence; no giant pack build.

What shipped

Cartography ledger — live-path maps (chat / packs / ProblemFrame-math / GSM8K eval), design-truth matrix, claim corrections, go/no-go for the ablation.
Ablation vertical slice — baseline (frame rational) vs geometric operator vs inactive depth vs metadata-only vs adversarial refuse; sealed n=8 fixture; wrong=0.
Tests + measurement — tests/test_relational_operator_ablation.py (20 tests); committed evals/relational_operator_ablation/v1/report.json.
HE/GRC Logos roadmap — audit-only matrices, dual pack systems, holonomy limits, no-go list, recommended next branch feat/observed-he-morph-constraint-v0.

Scientific boundary

English-only fraction_decrease depth was intentionally inert (no observed morphology). That does not validate Hebrew/Greek as executable constraints. Next work (separate PR) should use observed HE/GRC input only.

Key files

Path	Role
`generate/relational_operator_ablation.py`	Condition runners
`evals/relational_operator_ablation/v1/`	Sealed cases, runner, report
`tests/test_relational_operator_ablation.py`	Unit/integration/adversarial
`docs/analysis/relational-operator-ablation-cartography-2026-07-19.md`	Cartography
`docs/analysis/relational-operator-ablation-dossier-2026-07-19.md`	Ablation dossier
`docs/analysis/hebrew-koine-greek-logos-pack-capability-roadmap-2026-07-19.md`	Logos audit

Ablation results (n=8, correct/wrong/refused)

Condition	correct	wrong	refused
baseline / operator / depth / metadata_only	2	0	6
invalid (adversarial subset)	0	0	6

Metadata-only inert ✓ · depth inactive = operator ✓ · deterministic ✓.

Not in this PR

Full GSM8K solve claims
Pack lexicon expansion
Morph→role universals (nominative=agent, etc.)
Holonomy crown “proof” claims (in-tree: not robust)

Test plan

PYTHONPATH=. python3 -m pytest -q tests/test_relational_operator_ablation.py → 20 passed
PYTHONPATH=. python3 -m evals.relational_operator_ablation.v1.runner → passed
uv run core test --suite smoke -q → 176 passed (~132s)

[Verification]: Smoke suite passed locally (176 passed, ~132s); ablation tests 20 passed; ablation runner n=8 wrong=0

Reviewer checklist

Ablation does not invent EN→HE/GRC morph evidence
Metadata-only control is inert
wrong=0 preserved on sealed fixture
Preexisting A2k scale>1 gap isolated, not masked
Logos roadmap is planning-only (no pack build authorization)
Recommended next branch is clear: feat/observed-he-morph-constraint-v0

Risks / rollback

Shared organ hazard helpers for baseline refuse gates (scalar vs geometric comparison honesty).
Rollback: revert branch commits; no serving-flag catalog changes.

## Summary Evidence-led cartography and a **narrow** Deterministic Relational Operator Ablation for `fraction_decrease`, plus a **separate audit-only** Hebrew/Koine Greek Logos pack capability roadmap. No English→ancient-language pseudo-evidence; no giant pack build. ### What shipped 1. **Cartography ledger** — live-path maps (chat / packs / ProblemFrame-math / GSM8K eval), design-truth matrix, claim corrections, go/no-go for the ablation. 2. **Ablation vertical slice** — baseline (frame rational) vs geometric operator vs inactive depth vs metadata-only vs adversarial refuse; sealed n=8 fixture; wrong=0. 3. **Tests + measurement** — `tests/test_relational_operator_ablation.py` (20 tests); committed `evals/relational_operator_ablation/v1/report.json`. 4. **HE/GRC Logos roadmap** — audit-only matrices, dual pack systems, holonomy limits, no-go list, recommended next branch `feat/observed-he-morph-constraint-v0`. ### Scientific boundary English-only `fraction_decrease` depth was **intentionally inert** (no observed morphology). That does **not** validate Hebrew/Greek as executable constraints. Next work (separate PR) should use **observed HE/GRC** input only. ### Key files | Path | Role | |------|------| | `generate/relational_operator_ablation.py` | Condition runners | | `evals/relational_operator_ablation/v1/` | Sealed cases, runner, report | | `tests/test_relational_operator_ablation.py` | Unit/integration/adversarial | | `docs/analysis/relational-operator-ablation-cartography-2026-07-19.md` | Cartography | | `docs/analysis/relational-operator-ablation-dossier-2026-07-19.md` | Ablation dossier | | `docs/analysis/hebrew-koine-greek-logos-pack-capability-roadmap-2026-07-19.md` | Logos audit | ### Ablation results (n=8, correct/wrong/refused) | Condition | correct | wrong | refused | |-----------|---------|-------|---------| | baseline / operator / depth / metadata_only | 2 | 0 | 6 | | invalid (adversarial subset) | 0 | 0 | 6 | Metadata-only inert ✓ · depth inactive = operator ✓ · deterministic ✓. ### Not in this PR - Full GSM8K solve claims - Pack lexicon expansion - Morph→role universals (nominative=agent, etc.) - Holonomy crown “proof” claims (in-tree: not robust) ## Test plan - [x] `PYTHONPATH=. python3 -m pytest -q tests/test_relational_operator_ablation.py` → 20 passed - [x] `PYTHONPATH=. python3 -m evals.relational_operator_ablation.v1.runner` → passed - [x] `uv run core test --suite smoke -q` → 176 passed (~132s) `[Verification]: Smoke suite passed locally (176 passed, ~132s); ablation tests 20 passed; ablation runner n=8 wrong=0` ## Reviewer checklist - [ ] Ablation does not invent EN→HE/GRC morph evidence - [ ] Metadata-only control is inert - [ ] wrong=0 preserved on sealed fixture - [ ] Preexisting A2k scale>1 gap isolated, not masked - [ ] Logos roadmap is planning-only (no pack build authorization) - [ ] Recommended next branch is clear: `feat/observed-he-morph-constraint-v0` ## Risks / rollback - Shared organ hazard helpers for baseline refuse gates (scalar vs geometric comparison honesty). - Rollback: revert branch commits; no serving-flag catalog changes.

core-labs added 5 commits 2026-07-20 04:14:53 +00:00

docs(analysis): cartography ledger for relational operator ablation df029a8485

Evidence matrix, live-path maps, claim corrections, and go/no-go for a
narrow fraction_decrease ablation (not ancient-language GSM8K magic).

feat(generate): deterministic relational operator ablation runners 1d4d51bc9b

Baseline scalar frame path, live geometric operator path, inactive depth
contribution provenance, metadata-only control, and fail-closed invalid
condition for fraction_decrease — without inventing EN→HE/GRC role maps.

feat(eval): sealed relational operator ablation lane v1 87b01868fa

Eight-case fixture, measurement runner, and committed report for baseline
/ operator / depth / metadata_only / invalid conditions (wrong=0).

test(eval): prove relational operator ablation invariants + dossier 84f7a6f7cc

Unit, integration, and adversarial tests; isolate pre-existing A2k scale>1
gap; final engineering dossier with measurement table.

docs(analysis): land HE/GRC Logos pack capability roadmap 08881ab665

Audit-only dossier: current-state matrices, dual pack systems, holonomy
limits, no-go list, and recommended next branch
feat/observed-he-morph-constraint-v0. Cross-link cartography and ablation
dossiers; no pack expansion or ablation fixture changes.