core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

feat(field-wedge): ablation verdict — field is decoration on additive (C3) (Phase W.2)

Browse source 
The falsifiable experiment's measurements #2 (ablation) and #3 (diversity). Builds the
competent, code-disjoint SYMBOLIC reader (the control arm AND the C3 capability path)
and the ablation instrument that runs both readers through the real
verify_tier2_agreement gate.

VERDICT: C3 — the field is decoration on this domain (a sanctioned, honest negative):
- field_wrong_commits = []  (wrong=0 holds; the per-step drift guard refuses bad ints)
- field_caught_symbolic_errors = []  (the field caught ZERO symbolic errors)
- per-class diversity = 0 everywhere (both readers agree and are both correct)
- the only admitted-set change is the field LOSING coverage at the precision ceiling.

Insight: on forward-substitutable relations, geometric translation IS arithmetic
addition, so there is no metric over-determination for the field to exploit — field and
symbol are common-mode (Knight-Leveson), not a genuine second derivation. This is the
deductive finding's twin: logic was combinatorial (field can't earn it), additive is
arithmetically trivial (field adds nothing). The field needs metric-nontrivial AND
arithmetically-hard structure to earn a reasoning role — dedicated research, not
near-term. Field-as-reasoner is NOT earned; no field vote enters any serving path; the
field stays a servant. Capability path = symbolic (C3), not shipped here.

- generate/relational_symbolic_reader.py: competent independent reader (pure int).
- evals/relational_metric/ablation.py: the reusable decoration instrument.
- docs/analysis/field-wedge-ablation-result-2026-06-04.md: the recorded verdict.

All prior artifacts STAY (field reader = real wrong=0 read demo + 3rd panel domain).
Green: full wedge suite 104; 53 architectural invariants.
This commit is contained in:
Shay 2026-06-04 19:44:22 -07:00
parent 145d797196
commit 5c77c9eece
5 changed files with 483 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

83
docs/analysis/field-wedge-ablation-result-2026-06-04.md Normal file
View file

@ -0,0 +1,83 @@
<!-- CANONICAL | docs/analysis/field-wedge-ablation-result-2026-06-04.md | 2026-06-04 | research/experimental-result | the field-reasoner wedge ablation verdict: the field reads correctly (wrong=0) but adds no independent signal over a symbolic reader on the metric-trivial micro-domain — C3 | verified: ablation run committed; field_wrong=[], field_caught_symbolic_errors=[] -->
# The field-reasoner wedge ablation — result: C3 (decoration on this domain)
This records the outcome of the falsifiable experiment designed in
[`field-reasoner-wedge-design-and-falsification-2026-06-04.md`](./field-reasoner-wedge-design-and-falsification-2026-06-04.md).
The verdict is the dossier's sanctioned negative: **the field reads correctly but is
decoration here.** That is a success — it answers the field-as-reasoner question with
evidence instead of deferring it.
## What was measured
Two readings of the same forward-substitutable quantitative-relational micro-domain
(additive / part-whole), scored against an independent arithmetic oracle:
- **Field reader** (`generate/relational_field_reader.py`) — reads TEXT into conformal
points on the e1 number line; additive relations are conformal translator versors;
the answer reads back by projective dehomogenization. (The geometric SUT.)
- **Symbolic reader** (`generate/relational_symbolic_reader.py`) — a competent,
code-disjoint reading by plain integer arithmetic; no `algebra` (INV-27 disjoint).
The ablation (`evals/relational_metric/ablation.py`) runs both through the real
`verify_tier2_agreement` gate and asks the only questions that matter.
## Result
| Measurement | Value | Meaning |
|---|---|---|
| **field_wrong_commits** | `[]` | wrong=0 holds — the field never commits a bad integer (the per-step drift guard refuses instead). |
| **field_caught_symbolic_errors** | `[]` | The field caught **zero** comprehension errors the symbolic reader made. (Measurement #2 = fail.) |
| **field_lost_coverage** | `[rm-v1-0015]` | The only admitted-set change is the field **refusing a correct answer** at the precision ceiling — a liability, not signal. |
| **per-class diversity** | `0` everywhere | On every committable class the readers AGREE and are BOTH CORRECT: `disagree=0`, `double_fault=0`. (Measurement #3 = no diversity.) |
| **verdict** | **`C3`** | The field is decoration / a coverage liability on this domain. |
## Why — the load-bearing insight
**On forward-substitutable relations, geometric translation *is* arithmetic addition.**
`versor_apply(T_δ, embed[x]) = embed[x+δ]` is exactly `x + δ`. There is no metric
over-determination — no betweenness, incidence, or continuous-constraint structure —
for the field to exploit that a competent symbolic reader lacks. So the two readings are
**redundant** (the common-mode the dossier predicted via Knight–Leveson), and agreement
adds no independent soundness signal. The field's *one* genuine marginal check
(over-determination coherence) is matched by the symbolic reader's `over_determined_conflict`
refusal, so even there it is not unique. And the field's precision ceiling makes it a
strict coverage liability versus pure integer arithmetic.
This is the deductive-logic finding's twin: there, logic was *combinatorial* not metric,
so the field could not earn it; here, the metric domain is *arithmetically trivial*, so
the geometry adds nothing. **The field earns a reasoning role only where geometry
genuinely exceeds the symbolic alternative — not yet demonstrated on any built domain.**
## What this changes
- **Field-as-reasoner is NOT earned.** No field vote enters any serving or capability
path. The field stays a servant (its existing cognition-turn role). No "the field
reasons" claim is made.
- **The capability path is symbolic (C3).** The symbolic reader is correct (15/15 here,
including the case the field refuses). Shipping quantitative capability safely is the
dossier's Phase 3+ on the symbolic path — **two** code-disjoint symbolic readings
agreeing + independent/sealed gold (one reader alone is the resolve_pooled risk). Not
built here; this slice answered the experiment, it did not ship serving.
- **The `t2_precision` lever stays dormant** for this domain — there is no genuine second
derivation to feed it (field⟂symbol is common-mode here).
## What stays (all load-bearing, none reverted)
- The **f64 conformal foundation** (Phase 0A) — net infrastructure hardening.
- The **reader-lineage firewall + INV-27** (Phase 0B/0C) — backs any future agreement gate.
- The **field reader + relational-metric lane** — a real wrong=0 demonstration that the
field *can* read (Phase W.1), and a valid 3rd independently-golded panel domain.
- The **symbolic reader + ablation instrument** — the C3 reader and a reusable decoration
test for the next domain.
## The honest next bet (dedicated research, not the near-term sequence)
Field-as-reasoner remains an open question for a domain where geometry genuinely beats
arithmetic: **continuous / over-determined / multi-constraint** structure (relative
position, betweenness, incidence among many points, ratio as cross-ratio on a
non-degenerate frame) — where a symbolic step-arithmetic reader and a field constraint
reading could genuinely diverge, and agreement would be a real second derivation. Until
such a domain is built and the ablation shows `field_caught_symbolic_errors > 0`, the
field stays a servant. Two negative domains (logic: combinatorial; additive: trivial)
now bound the search: the field needs *metric-nontrivial, arithmetically-hard* structure.

162
evals/relational_metric/ablation.py Normal file
View file

@ -0,0 +1,162 @@
"""The decoration instrument — does the geometric FIELD reader add independent signal?
Measurements #2 (ablation) and #3 (per-class diversity) of the field-reasoner wedge
falsifiable experiment. For each case it runs BOTH readers and the real
``verify_tier2_agreement`` gate, then asks the only questions that matter:
- **field_caught_symbolic_errors** — cases where the SYMBOLIC reader commits a WRONG
answer (≠ gold) but the agreement gate REFUSES (the field disagreed or refused).
This is the dossier's measurement-#2 PASS signal: the field refusing a comprehension
error the symbol path alone admits.
- **field_lost_coverage** — cases where the symbolic reader commits the CORRECT answer
but the gate refuses *because of the field* (field refused or disagreed). The field
subtracting a correct answer is a liability, not signal.
- **admitted_set_changed** — does the gate's admitted set differ from symbolic-alone?
- per-class double-fault / agreement diversity.
VERDICT:
- ``PASS`` iff ``field_caught_symbolic_errors > 0`` (the field adds error-catching
signal) and never commits a wrong answer.
- ``C3`` (field is decoration / a coverage liability on this domain) otherwise — a
sanctioned, honest outcome: the symbolic reading carries the capability.
Run: PYTHONPATH=. .venv/bin/python -m evals.relational_metric.ablation
"""
from __future__ import annotations
import json
import sys
from collections import defaultdict
from pathlib import Path
from typing import Any
from core.reasoning import OperatorEvidence, verify_tier2_agreement
from evals.relational_metric.oracle import OracleError, oracle_answer
from generate.relational_field_reader import READER_LINEAGE as FIELD_LINEAGE
from generate.relational_field_reader import read_relational as field_read
from generate.relational_symbolic_reader import READER_LINEAGE as SYM_LINEAGE
from generate.relational_symbolic_reader import read_relational as sym_read
_CASES = Path(__file__).resolve().parent / "v1" / "cases.jsonl"
def _commit_key(answer: int, unit: str | None) -> str:
return f"{answer}:{unit}"
def _gate_admits(field_r: Any, sym_r: Any) -> int | None:
"""Run the real Tier-2 gate; return the agreed answer or None (refuse)."""
if field_r.refused or sym_r.refused:
return None
ev_field = OperatorEvidence(
domain="relational_metric", operator="field_read", outcome="committed",
reason="field_commit", input_keys=(), check_keys=(),
commitment_key=_commit_key(field_r.answer, field_r.answer_unit),
structural_signature="field.geometric.number_line",
reader_lineage=FIELD_LINEAGE,
)
ev_sym = OperatorEvidence(
domain="relational_metric", operator="symbolic_read", outcome="committed",
reason="symbolic_commit", input_keys=(), check_keys=(),
commitment_key=_commit_key(sym_r.answer, sym_r.answer_unit),
structural_signature="symbolic.arithmetic.schema",
reader_lineage=SYM_LINEAGE,
)
verdict = verify_tier2_agreement((ev_field, ev_sym))
return field_r.answer if verdict.verified else None
def run() -> dict[str, Any]:
cases = [json.loads(l) for l in _CASES.read_text().splitlines() if l.strip()]
rows: list[dict[str, Any]] = []
per_class: dict[str, dict[str, int]] = defaultdict(
lambda: {"n": 0, "agree_commit": 0, "disagree": 0,
"both_correct": 0, "double_fault": 0}
)
field_caught: list[str] = []
field_lost_coverage: list[str] = []
field_wrong: list[dict[str, Any]] = []
for case in cases:
cid, cls = case["id"], case["class"]
try:
gold = oracle_answer(case["relations"], case["query"])
except OracleError:
continue
f = field_read(case["text"])
s = sym_read(case["text"])
gate = _gate_admits(f, s)
f_ans = None if f.refused else f.answer
s_ans = None if s.refused else s.answer
sym_alone = s_ans # symbolic-alone admits its committed answer
pc = per_class[cls]
pc["n"] += 1
if f_ans is not None and s_ans is not None:
if f_ans == s_ans:
pc["agree_commit"] += 1
if f_ans == gold:
pc["both_correct"] += 1
else:
pc["disagree"] += 1
if f_ans != gold and s_ans != gold:
pc["double_fault"] += 1
# The field's own wrong=0 (it must never commit a wrong answer).
if f_ans is not None and f_ans != gold:
field_wrong.append({"id": cid, "field": f_ans, "gold": gold})
# measurement #2 signals
if s_ans is not None and s_ans != gold and gate is None:
field_caught.append(cid) # symbolic committed WRONG; gate refused
if s_ans is not None and s_ans == gold and gate is None:
field_lost_coverage.append(cid) # symbolic correct; gate refused
rows.append({
"id": cid, "class": cls, "gold": gold,
"field": f_ans, "field_refused": f.refused and f.refusal_reason,
"symbolic": s_ans, "symbolic_refused": s.refused and s.refusal_reason,
"gate_admits": gate, "symbolic_alone": sym_alone,
})
gate_admitted = {r["id"]: r["gate_admits"] for r in rows if r["gate_admits"] is not None}
symbolic_admitted = {r["id"]: r["symbolic_alone"] for r in rows if r["symbolic_alone"] is not None}
admitted_changed = gate_admitted != symbolic_admitted
verdict = (
"PASS" if (field_caught and not field_wrong)
else "C3_field_is_decoration_or_liability"
)
return {
"total": len(rows),
"verdict": verdict,
"field_caught_symbolic_errors": field_caught,
"field_lost_coverage": field_lost_coverage,
"field_wrong_commits": field_wrong,
"admitted_set_changed": admitted_changed,
"gate_admitted_count": len(gate_admitted),
"symbolic_alone_admitted_count": len(symbolic_admitted),
"per_class": {k: dict(v) for k, v in per_class.items()},
"rows": rows,
}
def main() -> int:
report = run()
summary = {k: v for k, v in report.items() if k != "rows"}
print(json.dumps(summary, indent=2))
# The instrument is non-failing: a C3 verdict is a sanctioned, honest outcome.
# It only hard-fails if the field ever commits a WRONG answer (wrong=0 breach).
if report["field_wrong_commits"]:
print("FIELD COMMITTED A WRONG ANSWER (wrong!=0 breach):",
report["field_wrong_commits"], file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())

127
generate/relational_symbolic_reader.py Normal file
View file

@ -0,0 +1,127 @@
"""The SYMBOLIC reader for forward-substitutable quantitative relations.
The second, code-disjoint reading of the same micro-domain as
``generate.relational_field_reader`` — but it resolves by plain integer arithmetic
over a parsed schema, importing **no** ``algebra`` / ``field`` (proven import-disjoint
from the field reader by INV-27). It is:
- the control arm of the ablation experiment (does the geometric FIELD reader add any
signal over a competent symbolic reader, or is it decoration?), and
- the C3 capability path if the field does not earn its role (a second independent
reading whose agreement with another reader is the wrong=0 gate).
It is a *competent* reader, not a strawman: it reads the same grammar and applies the
same fences (multiplicative/ratio, forward-substitutability, negatives), and it
detects over-determination (a quantity stated two ways that conflict) and refuses —
so the ablation is a fair test. It has NO float/precision limit (pure int), so unlike
the field it commits arbitrarily large quantities.
"""
from __future__ import annotations
import re
from dataclasses import dataclass
# Stable identity for the Tier-2 gate (registered in INV-27).
READER_LINEAGE: str = "symbolic.relational_schema"
_FENCED_CUE = re.compile(
r"\b(times|twice|double|triple|thrice|half|quarter|ratio|per|each)\b"
)
# Independent regex set (own spelling/order) — same grammar, different implementation.
_REL_MORE = re.compile(r"(\w+) (?:has|have) (\d+) more \w+ than (\w+)")
_REL_FEWER = re.compile(r"(\w+) (?:has|have) (\d+) (?:fewer|less) \w+ than (\w+)")
_REL_FACT = re.compile(r"(\w+) (?:has|have) (\d+) (\w+)")
_Q_SINGLE = re.compile(r"how many (\w+) does (\w+) have")
_Q_SUM = re.compile(r"how many (\w+) do (\w+) and (\w+)(?: and (\w+))? have")
class SymbolicReaderError(ValueError):
"""Out-of-grammar case → typed refusal."""
@dataclass(frozen=True, slots=True)
class SymbolicReading:
refused: bool
answer: int | None = None
answer_unit: str | None = None
refusal_reason: str | None = None
reader_lineage: str = READER_LINEAGE
def _refuse(reason: str) -> SymbolicReading:
return SymbolicReading(refused=True, refusal_reason=reason)
def read_relational(text: str) -> SymbolicReading:
if not isinstance(text, str) or not text.strip():
return _refuse("empty_input")
if _FENCED_CUE.search(text.lower()):
return _refuse("fenced_multiplicative")
try:
return _read(text.lower())
except SymbolicReaderError as exc:
return _refuse(str(exc))
def _set(values: dict[str, int], entity: str, value: int) -> None:
"""Assign, detecting over-determination (a conflicting re-statement)."""
if entity in values and values[entity] != value:
raise SymbolicReaderError("over_determined_conflict")
values[entity] = value
def _read(text: str) -> SymbolicReading:
parts = [s.strip() for s in re.split(r"[.?!]", text) if s.strip()]
unit: str | None = None
q_single: str | None = None
q_sum: list[str] | None = None
for s in parts:
if "how many" not in s:
continue
if (m := _Q_SUM.search(s)) is not None:
unit = m.group(1)
q_sum = [g for g in m.groups()[1:] if g]
elif (m := _Q_SINGLE.search(s)) is not None:
unit, q_single = m.group(1), m.group(2)
else:
raise SymbolicReaderError("unparsed_query")
break
if q_single is None and q_sum is None:
raise SymbolicReaderError("no_query")
values: dict[str, int] = {}
for s in parts:
if "how many" in s:
continue
if (m := _REL_MORE.search(s)) is not None:
ent, n, ref = m.group(1), int(m.group(2)), m.group(3)
if ref not in values:
raise SymbolicReaderError("non_forward_substitutable")
_set(values, ent, values[ref] + n)
elif (m := _REL_FEWER.search(s)) is not None:
ent, n, ref = m.group(1), int(m.group(2)), m.group(3)
if ref not in values:
raise SymbolicReaderError("non_forward_substitutable")
_set(values, ent, values[ref] - n)
elif (m := _REL_FACT.search(s)) is not None:
ent, n = m.group(1), int(m.group(2))
_set(values, ent, n)
else:
raise SymbolicReaderError("unparsed_sentence")
if q_single is not None:
if q_single not in values:
raise SymbolicReaderError("query_entity_unresolved")
answer = values[q_single]
else:
assert q_sum is not None
if any(p not in values for p in q_sum):
raise SymbolicReaderError("query_entity_unresolved")
answer = sum(values[p] for p in q_sum)
if answer < 0:
raise SymbolicReaderError("negative_quantity")
return SymbolicReading(refused=False, answer=answer, answer_unit=unit)

52
tests/test_relational_ablation.py Normal file
View file

@ -0,0 +1,52 @@
"""The ablation verdict — pinned as the honest experimental result.
The field-reasoner wedge's measurement #2/#3 on the simple additive/part-whole
micro-domain: the geometric field reader is **decoration** — it catches no
comprehension error the symbolic reader makes, and the only place it changes the
admitted set is by *losing* coverage (refusing a correct answer at the precision
ceiling). This test pins that finding; if a future change makes the field genuinely
catch a symbolic error, ``field_caught_symbolic_errors`` becomes non-empty and the
PASS assertion here must be revisited (a deliberately meaningful pin).
The one INVARIANT that must always hold regardless of the verdict: the field never
commits a wrong answer (wrong=0 is structural).
"""
from __future__ import annotations
from evals.relational_metric.ablation import run
def test_field_never_commits_wrong():
"""wrong=0 — the non-negotiable. The field refuses rather than commit a bad int."""
report = run()
assert report["field_wrong_commits"] == [], report["field_wrong_commits"]
def test_verdict_is_honest_c3_on_this_domain():
"""On forward-substitutable additive relations the field adds no independent signal:
geometric translation == arithmetic addition, so it catches zero symbolic errors."""
report = run()
assert report["field_caught_symbolic_errors"] == []
assert report["verdict"].startswith("C3")
def test_field_only_changes_admitted_set_by_losing_coverage():
"""The admitted set differs from symbolic-alone only because the field REFUSES a
correct answer (the precision ceiling) — a liability, not error-catching signal."""
report = run()
assert report["admitted_set_changed"] is True
assert report["gate_admitted_count"] < report["symbolic_alone_admitted_count"]
assert report["field_lost_coverage"] # non-empty: the ceiling case
def test_readers_agree_and_are_both_correct_on_committable_classes():
"""Per-class diversity is ZERO on every committable class — the readers are
redundant here (the dossier's predicted common-mode on a metric-trivial domain)."""
report = run()
for cls, stats in report["per_class"].items():
if cls.startswith("coverage_"):
continue
assert stats["disagree"] == 0
assert stats["double_fault"] == 0
assert stats["both_correct"] == stats["n"]

59
tests/test_relational_symbolic_reader.py Normal file
View file

@ -0,0 +1,59 @@
"""The symbolic reader — the ablation control arm and the C3 capability path.
It must be a *competent* reader (correct on the grammar, fences the same out-of-domain
cases, detects over-determination), so the ablation is a fair test of whether the field
adds anything over it. Unlike the field it has no precision limit (pure int).
"""
from __future__ import annotations
from generate.relational_symbolic_reader import READER_LINEAGE, read_relational
def test_commits_additive_more_than():
r = read_relational(
"Tom has 3 marbles. Jane has 5 more marbles than Tom. "
"How many marbles does Jane have?"
)
assert not r.refused and r.answer == 8
assert r.reader_lineage == READER_LINEAGE
def test_commits_part_whole_sum():
r = read_relational(
"Tom has 3 marbles. Jane has 5 more marbles than Tom. "
"How many marbles do Tom and Jane have?"
)
assert not r.refused and r.answer == 11
def test_commits_beyond_field_ceiling():
"""The symbolic reader has no f64 precision limit — it commits where the field
refuses (over_ceiling). This is the field's coverage liability, not a symbolic bug."""
r = read_relational("Gus has 9000000 apples. How many apples does Gus have?")
assert not r.refused and r.answer == 9000000
def test_fences_multiplicative():
r = read_relational(
"Tom has 3 marbles. Jane has twice as many marbles as Tom. "
"How many marbles does Jane have?"
)
assert r.refused and r.refusal_reason == "fenced_multiplicative"
def test_detects_over_determination():
"""A competent reader refuses a conflicting re-statement (so the field's geometric
coherence check is not a unique advantage)."""
r = read_relational(
"Tom has 3 marbles. Tom has 5 marbles. How many marbles does Tom have?"
)
assert r.refused and r.refusal_reason == "over_determined_conflict"
def test_refuses_forward_reference():
r = read_relational(
"Jane has 5 more marbles than Tom. Tom has 3 marbles. "
"How many marbles does Jane have?"
)
assert r.refused and r.refusal_reason == "non_forward_substitutable"