core/docs/assessment
Claude ca5e614ec8
feat(tests,assessment): G-23 — the domain/pack binding is stated twice and checked one way
core/capability/domains.py's DOMAIN_PACKS declares domain -> pack membership
for the capability ledger. Each packs/data/<pack>/manifest.json independently
declares domain_id. Two statements of one fact, and only one direction was
ever checked: domain_contract_predicates P3 validates domain_id -> a known
ledger domain. Nothing validated DOMAIN_PACKS -> the manifest agrees.

P3 passes VACUOUSLY on a manifest with no domain_id at all — the same shape as
its own test_pack_without_contract_reports_absent. So a pack that DOMAIN_PACKS
places in a domain, but whose manifest never says so, is invisible to every
check in the repository.

Measured: 7 of 9 bound, 0 contradictory, 2 absent — en_core_cognition_v1 and
en_core_meta_v1, both placed in philosophy_theology, which is one of the four
bands queued to earn a SERVE license behind R-8. The domain whose license is
next to be earned is the one whose binding exists in only one place.

Why nothing caught it: tests/test_domain_contract_predicates.py exercises
P1-P9 entirely against synthetic tmp_path manifests, never against the real
nine, and is itself in no curated suite. A predicate proven correct on
fabricated input and never run on the real input is this arc's defect class
stated precisely — its success state and "it never ran here" are
indistinguishable.

tests/test_domain_pack_binding.py, registered in smoke (on the gate):
  - every pack DOMAIN_PACKS names has a manifest on disk
  - a manifest that CONTRADICTS the table always fails
  - a manifest with NO domain_id fails unless the pack is in UNBOUND_BASELINE
  - the reverse direction: a pack may not self-assign a ledger domain the
    table does not grant (clean across all 30 packs today, pinned because a
    new manifest copying domain_id from a sibling is the obvious mistake and
    would silently widen a licensed domain)
  - both directions on the baseline: a pack that gains domain_id must leave it
  - the parse is guarded against vacuity (N-1 has produced three false
    conclusions in this repo)
  - the count is pinned so the gap moving is a reviewed decision

Four sabotages, each observed red, each caught by its intended pin.

PINNED, NOT FIXED, deliberately. Whether en_core_cognition_v1 is a
philosophy_theology domain pack or merely grouped under that domain for ledger
accounting decides what the license actually covers. That is
ratification-adjacent content, not engineering, and it is routed to R-8 /
PR-14 where that band's license is decided. Sealing that ledger without
answering it licenses a domain whose membership is asserted in one place and
denied by silence in the other.

How it was found, recorded because the method is the point. An external
assessment proposed collapsing the pack/domain machinery, stating its method
up front: "I have enough from the directory and CLAIMS.md to reason
seriously." That is the method AGENTS.md protocol rule 1 forbids, and five of
its six proposals are falsified by opening the file — recorded in full in
31-hindrance-audit.md, in short:

  - conftest.py "almost certainly" holds suite-membership logic: it holds two
    autouse fixtures and two data registries, and its collection hook stamps
    markers and never skips. Zero membership logic.
  - cli_test.py "almost certainly a nested conditional or a big dict": a flat
    dict, 17 keys, zero conditionals in the module body.
  - field/algebra/morphology as a peer triad needing a single serve(): 72, 15
    and 3 importers respectively, morphology is 97 lines, and no def serve(
    exists anywhere in core/, chat/ or generate/.
  - the grade space is richer than the serve layer consumes: all 15
    EpistemicState members have non-test consumers.
  - constraints: blocks in 314 ADRs: that is 314 new places that must be kept
    true, inside ratified immutable records, in the repository whose dominant
    defect class is records drifting from reality. The existing design puts
    the machine-readable half in the checker that reads the ADR, which is
    better on purpose.
  - a root contracts.toml as "the highest-leverage single change": the
    principle is already ADR-0263 Rule 5 and is implemented in the very file
    the assessment cites. Centralizing four tables with four authorities adds
    a fifth copy that must agree with four generators.

The seam it pointed at was real even though every specific was wrong. Bad
method, useful direction; both halves recorded, and the triage rule earns a
second demonstration — every rejected proposal failed on a file its author did
not open.

AGENTS.md — the standing philosophy revised to eleven, per Shay's review.
  - "be quick to be humble" -> "investigate yourself first". Speed is a
    consequence of reflex, not a criterion; writing it in makes velocity sound
    like the point. Examples replaced with three that are verifiable in-repo:
    the H-13 refcount claim, and the two rulings that were about to be
    recorded stronger than they were made.
  - the defect-class principle gains its prescriptive half as its own
    sentence, and gains this commit's instance: a predicate that passes
    vacuously on the input it was written for.
  - NEW #2: don't build a ratchet to guard a ratchet. Placed adjacent to
    "less is more" and named as its corollary so it does not read as
    duplication. The arc paid for it — a membership ratchet needed a
    reachability ratchet to mean anything, and what actually shrank the gap
    was deleting four suites nobody called.
  - NEW #10: a closed issue is not evidence the problem is gone. G-7 was
    CLOSED and REOPENED-PARTLY inside one arc; H-13 was closed with a fix that
    did not work.

Two review points were already satisfied in the file and are recorded as such
rather than re-applied: the defect-class principle already stood alone on its
first line, and already sat at position 3, not 8. The review was reading a
chat summary, not AGENTS.md — which is, exactly, reviewing the record instead
of the reality.

PR-5 additionally owes one page: an index of which declared tables exist and
what pin makes each true. That is the defensible residue of the contracts.toml
proposal, and explicitly not a fifth copy of four tables.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wcw2pnMBwyvmNyQg4uPEt4
2026-07-28 06:13:10 +00:00
..
10-layer-cards fix(tests,claims): G-22 — main was red, and CLAIMS.md was publishing evidence that no longer existed 2026-07-28 05:25:28 +00:00
20-component-cards docs(assessment): re-stamp the eight cards this arc touched, and post the board 2026-07-28 04:36:13 +00:00
00-scope-and-method.md docs(assessment): Phase 4 — the gap register and the hindrance audit 2026-07-27 16:04:22 -07:00
01-phase0-ground-truth.md
02-layer-taxonomy.md feat(wave-1): retire the dead instruments, delete the aliases nobody calls (PR-3, PR-3b, R-7) 2026-07-28 05:40:45 +00:00
03-card-schema.md
04-phase2-findings.md docs(assessment): Phase 2 — nine layer cards, verified against code not documents 2026-07-27 15:45:52 -07:00
05-phase3-findings.md docs(assessment): Phase 3 — component descent; the correction chain continues into code 2026-07-27 15:57:36 -07:00
30-gap-register.md feat(tests,assessment): G-23 — the domain/pack binding is stated twice and checked one way 2026-07-28 06:13:10 +00:00
31-hindrance-audit.md feat(tests,assessment): G-23 — the domain/pack binding is stated twice and checked one way 2026-07-28 06:13:10 +00:00
40-assessment.md feat(rnd): ADR-0252 §5 run to a verdict — NO-GO, and the two prior GOs are void 2026-07-28 02:55:31 +00:00
50-execution-plan.md feat(tests,assessment): G-23 — the domain/pack binding is stated twice and checked one way 2026-07-28 06:13:10 +00:00
50-rulings.md docs(assessment,governance): record the adopted docket and the standing philosophy 2026-07-28 05:57:11 +00:00
README.md feat(tests,assessment): G-23 — the domain/pack binding is stated twice and checked one way 2026-07-28 06:13:10 +00:00

docs/assessment/ — The Holistic Macro→Micro Assessment (2026-07-27)

A read-only, evidence-bearing assessment of CORE's cognitive-cycle design versus implementation fulfillment, conducted under docs/conceptualizing_engineering_mastery.md at forgejo/main @ 8927c563. Phases 05 change no runtime behavior, fix no defect, and decide nothing — they produce evidence and judgments for ruling. Phase 6 adds the execution plan built on them and the ruling packet they require.

Start here: 40-assessment.md — the synthesis (the verdict, the five frontiers, the recommended attack order). Then 50-execution-plan.md for what follows from it, and 50-rulings.md for what is waiting on a decision. The registers rank the work; the cards are the evidence base.

File / dir Phase What it is
00-scope-and-method.md Charter, method, rules of engagement, phase/executor table
01-phase0-ground-truth.md 0 Corpus triage, the five unreconciled articulations, system-map recovery
02-layer-taxonomy.md 1 The two-axis taxonomy: 7 macro layers + 2 cross-cuts over 33 zones; the Candidate Register (CR-1…4); completeness criteria
03-card-schema.md 1 The card metadata schema: liveness ⊥ fitness, design ⊥ build, sabotage-tested evidence, verified_at stamps
10-layer-cards/ 2 Nine layer cards (M0M6, MG, MV), every liveness label re-verified against code
04-phase2-findings.md 2 Stage-coverage audit; corrections to Phase 0; findings F-1…F-5
20-component-cards/ 3 Eight component cards: the four zero-subsystem zones + always-on, derivation organs, surface selection, attention
05-phase3-findings.md 3 Corrections C-1…C-5; findings F-6…F-10; the consolidated Phase-4 seed list
30-gap-register.md 4 The live gap register — 23 entries (G-21, G-22, G-23 added during execution), 4 tiers, each with evidence + deciding authority; supersedes docs/gaps.md and the substrate-liveness ratchet per R-7
31-hindrance-audit.md 4 Fourteen hindrances with fitness verdicts and better homes; seventeen candidates examined and cleared — five in Phase 4, six in the first 2026-07-28 external-assessment triage, six in the second (of which one, the pack/domain seam, survived as G-23)
40-assessment.md 5 The synthesis
50-execution-plan.md 6 The execution plan — five waves + five frontier tracks over every G/H entry, with the dependency gates and the risks. §0 carries nine corrections to the assessment found while sizing and executing it; §2.1 carries the adopted ruling docket and its execution order
50-rulings.md 6 The ruling packet — R-1…R-14, each with evidence, options, a recommendation, and the exact diff that follows from each choice. RULED 2026-07-28: twelve adopted, R-10 and R-14 stricken, §5 NO-GO ratified. Wave 0's deliverable, and Wave 0 is closed

Maintenance contract (from §8 of the synthesis): a card whose verified_at falls behind a load-bearing arc is testimony, not evidence. Update cards when their subsystems move, or this directory becomes the next dead instrument it was built to replace.

Execution status (2026-07-28, d0bedfc1). Phases 06 produced the evidence; the arc that followed executed everything in it that needed no ruling, and Wave 0 then closed: twelve rulings adopted, two stricken, the §5 NO-GO ratified. Landed: Track A's §5 verdict (pre-registered), PR-4 (+ pin 3), PR-6, PR-7, PR-9, PR-1, PR-3 + PR-3b (Wave 1, per R-7), and three defect fixes (H-13, H-8e, G-22). Closed: G-7, G-9, G-22, H-7, H-11, H-13. Added: N-8, N-9, G-21, G-22, H-13, H-14. What remains is owed work, not open questions — see 50-execution-plan.md §Status for the board and §2.1 for the adopted docket and its execution order.

Standing note: the PR #138 fabrication findings appear throughout as measured & pinned, fix held for ADR + ratification — recorded, never re-discovered, never fixed here, per explicit instruction.