core/docs/assessment
Claude 7728a25dee
feat(rnd): ADR-0252 §5 run to a verdict — NO-GO, and the two prior GOs are void
Track A of docs/assessment/50-execution-plan.md §6, executed under its own
protocol. Criterion pre-registered at dfc394d2 (that commit carries the
thresholds as importable constants and NO results); this commit carries the run.

VERDICT: NO-GO. Full credit by ADR-0252 §5.4's own terms.
  docs/research/sme-experiment-verdict-797ebad5.md
  evals/structure_mapping/adr0252_s5/results/report-797ebad5.json
  deterministic_digest b3d9d27592e213104c51bcace7415fd819d722a1b06f7d0a5ca65bd5a234e4ec

53 cases, 1378 pairs, 0.0% non-convergence — no exception contributed to any
number. RS-A (structure-only) separates perfectly, AUC 1.0000, margin +0.0867,
and fails structure-sensitivity. RS-B (attribute-bearing) fails all three.

The mechanism, measured rather than argued: the similarity quotient that would
deliver attribute-invariance is the same quotient that annihilates structural
contrast. An add-vs-subtract minimal pair — one entity, identical numbers, one
relation kind changed — aligns at residual exactly 0.0, its two configurations
being related by a PROPER rotation about e1. Sweeping the attribute weight
(committed as diagnostic_sweep), structure-sensitivity fails at every setting,
and every regime where the SME property survives is a regime where attributes
contribute nothing: AUC 1.00 -> 0.97 -> 0.83 -> 0.69 as they begin to matter.

Scope stated narrowly: this refutes H1 for embeddings that encode role-structure
as point positions aligned by conformal Procrustes under similarity. The
argument is about the quotient, so it generalises across that class; it does not
refute every Cl(4,1) representation, and it does not touch the symbolic
structure-mapping lane already in evals/structure_mapping/.

Two findings that were on nobody's list:

N-8 — the experiment was NOT unrun. rnd/sme-experiment-v2 @ 96e5f468 is titled
"Verdict: GO" and rnd/structure-mapping-experiment @ fc9d0c14 carries an earlier
one. Neither survives inspection: attempt 1 leaked the S1-S4 label into the
embedding; attempt 2 was blind and then took its separability from
`except ValueError: res = 1000.0` — a solver exception counted as a distance —
on a corpus 46/51 of which is outside holdout_dev/v1 with nothing marked, with
duplicate graphs, colliding ids, and an extractor that was never committed. The
register, the plan and the ADR all read the same absence and inherited the same
error for nine days. A branch tip is not a record.

G-21 — the math reader returns a selected graph for 5 of 500 holdout_dev/v1
cases (1.0%), all one skeleton; 24/150 on the public lane. That is why §5.1's
four-structure corpus is not extractable, and it is a sharper measurement of the
comprehension frontier than G-3's construction count.

Registers updated: G-1 carries the verdict, H-10 confirmed and discharged, G-21
added, plan §0 gains N-8, Track A closed, synthesis frontier 2 rewritten.

Pins: tests/test_adr_0252_s5_blindness.py, each observed red before green
(label leak and duplicate-graph sabotage both caught).

Off-serving — evals/structure_mapping/adr0252_s5/ is imported by no serving
path, emits no answers, and changes no flag.

[Verification]: 797ebad5 + this branch — `uv run core test --suite smoke -q`
641 passed in 199.76s; `uv run ruff check` clean; report regenerated after lint
with an unchanged digest.
2026-07-28 02:55:31 +00:00
..
10-layer-cards docs(assessment): Phase 3 — component descent; the correction chain continues into code 2026-07-27 15:57:36 -07:00
20-component-cards docs(assessment): Phase 3 — component descent; the correction chain continues into code 2026-07-27 15:57:36 -07:00
00-scope-and-method.md docs(assessment): Phase 4 — the gap register and the hindrance audit 2026-07-27 16:04:22 -07:00
01-phase0-ground-truth.md
02-layer-taxonomy.md docs(assessment): Phase 1 — layer taxonomy and card schema (the keystone) 2026-07-27 15:28:42 -07:00
03-card-schema.md docs(assessment): Phase 1 — layer taxonomy and card schema (the keystone) 2026-07-27 15:28:42 -07:00
04-phase2-findings.md docs(assessment): Phase 2 — nine layer cards, verified against code not documents 2026-07-27 15:45:52 -07:00
05-phase3-findings.md docs(assessment): Phase 3 — component descent; the correction chain continues into code 2026-07-27 15:57:36 -07:00
30-gap-register.md feat(rnd): ADR-0252 §5 run to a verdict — NO-GO, and the two prior GOs are void 2026-07-28 02:55:31 +00:00
31-hindrance-audit.md feat(rnd): ADR-0252 §5 run to a verdict — NO-GO, and the two prior GOs are void 2026-07-28 02:55:31 +00:00
40-assessment.md feat(rnd): ADR-0252 §5 run to a verdict — NO-GO, and the two prior GOs are void 2026-07-28 02:55:31 +00:00
50-execution-plan.md feat(rnd): ADR-0252 §5 run to a verdict — NO-GO, and the two prior GOs are void 2026-07-28 02:55:31 +00:00
50-rulings.md docs(assessment): the pre-push hook makes the same parity claim (R-14, N-3) 2026-07-27 17:37:58 -07:00
README.md docs: apply the Wave 0 record amendments that need no ruling (N-1…N-7, H-12) 2026-07-27 17:38:12 -07:00

docs/assessment/ — The Holistic Macro→Micro Assessment (2026-07-27)

A read-only, evidence-bearing assessment of CORE's cognitive-cycle design versus implementation fulfillment, conducted under docs/conceptualizing_engineering_mastery.md at forgejo/main @ 8927c563. Phases 05 change no runtime behavior, fix no defect, and decide nothing — they produce evidence and judgments for ruling. Phase 6 adds the execution plan built on them and the ruling packet they require.

Start here: 40-assessment.md — the synthesis (the verdict, the five frontiers, the recommended attack order). Then 50-execution-plan.md for what follows from it, and 50-rulings.md for what is waiting on a decision. The registers rank the work; the cards are the evidence base.

File / dir Phase What it is
00-scope-and-method.md Charter, method, rules of engagement, phase/executor table
01-phase0-ground-truth.md 0 Corpus triage, the five unreconciled articulations, system-map recovery
02-layer-taxonomy.md 1 The two-axis taxonomy: 7 macro layers + 2 cross-cuts over 33 zones; the Candidate Register (CR-1…4); completeness criteria
03-card-schema.md 1 The card metadata schema: liveness ⊥ fitness, design ⊥ build, sabotage-tested evidence, verified_at stamps
10-layer-cards/ 2 Nine layer cards (M0M6, MG, MV), every liveness label re-verified against code
04-phase2-findings.md 2 Stage-coverage audit; corrections to Phase 0; findings F-1…F-5
20-component-cards/ 3 Eight component cards: the four zero-subsystem zones + always-on, derivation organs, surface selection, attention
05-phase3-findings.md 3 Corrections C-1…C-5; findings F-6…F-10; the consolidated Phase-4 seed list
30-gap-register.md 4 The live gap register — 20 entries, 4 tiers, each with evidence + deciding authority (proposes superseding docs/gaps.md); four entries amended in Phase 6
31-hindrance-audit.md 4 Twelve hindrances with fitness verdicts and better homes; five candidates examined and cleared
40-assessment.md 5 The synthesis
50-execution-plan.md 6 The execution plan — five waves + five frontier tracks over every G/H entry, with the dependency gates and the risks. §0 carries seven corrections to the assessment found while sizing it
50-rulings.md 6 The ruling packet — R-1…R-14, each with evidence, options, a recommendation, and the exact diff that follows from each choice. Wave 0's deliverable

Maintenance contract (from §8 of the synthesis): a card whose verified_at falls behind a load-bearing arc is testimony, not evidence. Update cards when their subsystems move, or this directory becomes the next dead instrument it was built to replace.

Standing note: the PR #138 fabrication findings appear throughout as measured & pinned, fix held for ADR + ratification — recorded, never re-discovered, never fixed here, per explicit instruction.