assessment: independent Tier-S verification, ratification ceremony, local-first CI runner (recovered from cloud session) #113

Merged

core-labs merged 10 commits from claude/core-architecture-assessment-recovered into main 2026-07-25 21:11:17 +00:00

Conversation 0 Commits 10 Files changed 48 +2881 -85

core-labs commented 2026-07-25 21:08:31 +00:00

Owner

Copy link

Recovers the 10-commit Claude Code web/cloud branch that never reached either remote (the sandbox git proxy accepted every git push and reported success, but nothing propagated). Delivered as a git bundle and fetched locally, so history and signatures are intact — not a content replay.

Supersedes PR #112, which carries only the stale 2026-05-22 a2054b58 commit (a duplicate docs/decisions/ADR-0109 stamped Proposed, contradicting the Accepted canonical at docs/adr/ADR-0109-…) and is mergeable: false. Recommend closing #112 unmerged.

48 files, +2884/−88. Substantive contents:

• docs/research/architecture-assessment-verification-2026-07-25.md — independent re-verification of an external 18-observation assessment against the tree at 6a06cb2. Falsifies ~1/3 of the proposed work items (the parents[n] audit: 168 sites, 0 resolve outside the repo root; assert_corpus_sound already structural; the Promotion Oracle already exists as the passing lane gate), and diagnoses the flagship articulation item at the wrong layer (EntailmentTrace carries opaque BDD node keys — nothing is dropped because no intermediate steps exist; multi-step articulation is engine work with a real soundness surface).
• Live defect found that no prior doc names: Workbench renders a proved deduction answer as ungrounded. workbench/api.py::_coerce_grounding_source whitelists {pack, teaching, vault, partial, oov, none} and returns "none" for the "deduction" label the runtime stamps — live since deduction_serving_enabled was ratified ON (2026-07-24). epistemic_state degrades honestly; grounding_source asserts a falsehood.
• teaching/ratification.py (+382) + tests/test_ratification_ceremony.py (14) — ratification ceremony and core proposal-queue ratify.
• scripts/ci/local-ci.sh (+185) — one-command local-first runner, fail-closed on interpreter.
• tests/test_lane_sha_verifier.py — two arc lanes were never registered (red on clean main).
• tests/test_derived_close_proposals.py — a test was writing proposals into the live in-repo sink; a vacuous or True assertion removed.
• docs/adr/INDEX-by-domain.md + tests/test_adr_index.py (5) — domain-keyed ADR index with dead-link and coverage pins.

Two items that need a ruling rather than a rubber stamp:

1. 3b2c760 relaxes requires-python ==3.12.13 → >=3.12,<3.13. Rationale: cpython-3.12.13 is not published for linux-x86_64, so uv sync --locked failed outright there and no local run was possible at all — and the local run is the merge bar. uv lock regenerated to a one-line diff; all 217 packages resolve identically. Bounded below 3.13.
2. 50fa287 reverts the bidirectional smoke-parity assertion added earlier in this same branch, keeping the CI ⊆ local direction. tests/test_cli_test_suites.py::test_cli_smoke_suite_covers_ci_smoke_gate survives intact with a guard comment. This does not touch the Tier-S S5 parity work.

[Verification]: uv sync --locked + uv run core test --suite smoke -q → 236 passed in 142.16s, run in an isolated worktree on canonical Python 3.12.13 on the M1. This closes the "canonical re-run on 3.12.13" item the originating session listed as still owed (its runs were all on 3.12.11). main confirmed aligned at 6a06cb29 across local, forgejo, and the GitHub mirror.

Not merged, per standing policy — green PRs sit until explicit merge authorization.

Recovers the 10-commit Claude Code web/cloud branch that never reached either remote (the sandbox git proxy accepted every `git push` and reported success, but nothing propagated). Delivered as a `git bundle` and fetched locally, so history and signatures are intact — not a content replay. **Supersedes PR #112**, which carries only the stale 2026-05-22 `a2054b58` commit (a duplicate `docs/decisions/ADR-0109` stamped `Proposed`, contradicting the `Accepted` canonical at `docs/adr/ADR-0109-…`) and is `mergeable: false`. Recommend closing #112 unmerged. 48 files, +2884/−88. Substantive contents: - `docs/research/architecture-assessment-verification-2026-07-25.md` — independent re-verification of an external 18-observation assessment against the tree at `6a06cb2`. Falsifies ~1/3 of the proposed work items (the `parents[n]` audit: 168 sites, 0 resolve outside the repo root; `assert_corpus_sound` already structural; the Promotion Oracle already exists as the passing lane gate), and diagnoses the flagship articulation item at the wrong layer (`EntailmentTrace` carries opaque BDD node keys — nothing is dropped because no intermediate steps exist; multi-step articulation is engine work with a real soundness surface). - **Live defect found that no prior doc names:** Workbench renders a proved deduction answer as ungrounded. `workbench/api.py::_coerce_grounding_source` whitelists `{pack, teaching, vault, partial, oov, none}` and returns `"none"` for the `"deduction"` label the runtime stamps — live since `deduction_serving_enabled` was ratified ON (2026-07-24). `epistemic_state` degrades honestly; `grounding_source` asserts a falsehood. - `teaching/ratification.py` (+382) + `tests/test_ratification_ceremony.py` (14) — ratification ceremony and `core proposal-queue ratify`. - `scripts/ci/local-ci.sh` (+185) — one-command local-first runner, fail-closed on interpreter. - `tests/test_lane_sha_verifier.py` — two arc lanes were never registered (red on clean `main`). - `tests/test_derived_close_proposals.py` — a test was writing proposals into the live in-repo sink; a vacuous `or True` assertion removed. - `docs/adr/INDEX-by-domain.md` + `tests/test_adr_index.py` (5) — domain-keyed ADR index with dead-link and coverage pins. Two items that need a ruling rather than a rubber stamp: 1. `3b2c760` relaxes `requires-python` `==3.12.13` → `>=3.12,<3.13`. Rationale: cpython-3.12.13 is not published for linux-x86_64, so `uv sync --locked` failed outright there and no local run was possible at all — and the local run is the merge bar. `uv lock` regenerated to a one-line diff; all 217 packages resolve identically. Bounded below 3.13. 2. `50fa287` reverts the *bidirectional* smoke-parity assertion added earlier in this same branch, keeping the CI ⊆ local direction. `tests/test_cli_test_suites.py::test_cli_smoke_suite_covers_ci_smoke_gate` survives intact with a guard comment. This does **not** touch the Tier-S S5 parity work. [Verification]: `uv sync --locked` + `uv run core test --suite smoke -q` → **236 passed in 142.16s**, run in an isolated worktree on canonical **Python 3.12.13** on the M1. This closes the "canonical re-run on 3.12.13" item the originating session listed as still owed (its runs were all on 3.12.11). `main` confirmed aligned at `6a06cb29` across local, `forgejo`, and the GitHub mirror. Not merged, per standing policy — green PRs sit until explicit merge authorization.

core-labs added 10 commits 2026-07-25 21:08:32 +00:00

docs(assessment): independent verification of the Tier-S arc assessment ... 676b6555b4

Re-checks every load-bearing claim of an external architectural assessment
(18 observations + a five-pillar blueprint) against the tree, by reading the
implicated code and, where possible, executing it.

Headline: Workbench records a proved deduction answer as ungrounded. With
deduction_serving_enabled ratified ON, chat/runtime.py stamps
TurnEvent.grounding_source="deduction", and workbench/api.py's
_coerce_grounding_source whitelist silently rewrites it to "none". Verified by
running workbench.api._run_chat_turn, not by reading. Scope is one field:
epistemic_state still reads epistemic_state_needed (honest) because
workbench/api.py:818 prefers the TurnEvent's own value. The unregistered path
degrades honestly; the hand-copied whitelist asserts a falsehood.

Falsified items, with evidence:
- parents[n] audit — run: 168 sites, 0 outside the repo root.
- realizer-guard exemption — documented at both guard sites.
- T12 — present in the weekly-audit stragglers TODO.
- assert_corpus_sound — already domain-parameterized and called unconditionally.
- promotion oracle — already the wrong=0 lane gate, per tier-s-housekeeping §3.
- accrue_realized_knowledge — a session-memory deployment profile, not a debt clock.
- cross-subject proof — resolve_domain fails closed on ambiguity.

Also corrects the blueprint's flagship item: EntailmentTrace carries BDD node
keys, not derivation steps, so multi-step articulation is engine work
(proof-term extraction), not a rendering-layer gap. And records that the
"second subject arena" premise is falsified by the curriculum-volume
measurement — 11 bands across 4 subjects already route, all 24x-73x short.

Companion Apple Silicon/MLX blueprint assessed: both premises contradicted by
the repo's own benchmark report (no MLX in any runtime path; Rust backend
opt-in and off). One of five items adopted (typestate), one rejected on
numerics (bf16 against a 1e-6 gate).

[Verification]: docs-only change; no code paths touched. Findings themselves
were executed against a scratch venv — the repo pins ==3.12.13, which uv
cannot fetch for linux-x86_64, so the pin was left untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

feat(provenance,teaching): close the lateral gaps the assessment actually found ... 71bf04fb44

Squashes the arc's work into one commit; the workflow-file edit it originally
carried is excluded (see the end of this message).

## Lane 1 — Workbench recorded a proved answer as ungrounded

With deduction_serving_enabled ratified ON (ADR-0256), workbench/api.py's live
chat route builds a bare ChatRuntime(), so the deduction composer decides
Workbench turns and stamps grounding_source="deduction" — but
_coerce_grounding_source carried a hand-copied whitelist of the six pre-arc
labels and silently rewrote anything else to "none". The runtime comment
reasoned this was inert because "REPL turns do not flow through Workbench's
CognitivePipelineRecord path". True, and irrelevant: the traffic flows the
other way. Stale since 2026-07-24.

Scope is one field. workbench/api.py:818 prefers TurnEvent.epistemic_state,
which read epistemic_state_needed — honest. So the UNregistered path degraded
honestly while the hand-copied whitelist asserted a falsehood; a second copy of
a closed enum was worse than no copy. Hence registration AND derivation:
GROUNDING_SOURCES exposes the Literal's members, and the coercion reads it.
workbench-ui badges/tokens/snapshot follow; enumCoverage.test.ts forces atomicity.

## Lane 2 — the ratification ceremony

The discovery loop was instrumented but not closed. teaching/ratification.py
turns a reviewed decision into a chain record, a corpus commit, and a receipt.

Its design turns on one observation: _ratified_rows DROPS unadmissible rows
silently — correct when serving, a trap when ratifying, because the file grows,
the commit lands, and the band count does not move. So the ceremony refuses to
call an append a ratification until it has re-read the curriculum through the
real loader and seen the chain arrive; a non-admitted append is rolled back.
Validation is a pre-flight courtesy, admission is the proof.

Arena queue entry and ledger reseal are deliberately NOT performed (bridge rule
1); the receipt names them. Front door: `core proposal-queue ratify`, a sibling
of `review` rather than a flag on it.

## Lane 3 — structural closures

- ADR-0263 gains rule 5: absence policy is DECLARED in CAPABILITY_LEDGERS, not
  passed at the call site. An AST-matched test fails if a serving path passes
  missing_ok again.
- Deductive suite added WHOLE to the pre-push gate: 285 tests in 29s against
  smoke's 216 in 62s, so no coverage trade was needed.
- Smoke/CI parity assertion made bidirectional. It was one-directional, and had
  drifted.
- test_prior_surface_deduction_binding.py pins correction binding on the
  deduction path. The review's diagnosis did NOT reproduce — hash_surface moves
  in lockstep — so it pins what is there. Mutation-checked.
- Domain-keyed ADR index over 312 flat-numbered files, explicitly partial.
- Arc-close brief template, plus this arc's own brief filled in against it.

## Lanes 4 and 5 — two premises falsified by measurement, one of them mine

Math 4.2: baseline reproduced (correct=5 wrong=0 refused=495); all four named
cases traced to one seam with each gap isolated by one-variable probes. Then the
number that changes the recommendation: the gap blocking case 0000 affects 1
case in 500, the 'than' gap blocking 0001 affects 2. ADR-0251's prohibition on
per-case growth now rests on a count. No reader change made.

CGA: versor_condition is 0.22% of a turn, not the "~10x proof latency" I claimed
— that multiplied an isolated microbenchmark by a call count and compared it to
a single verdict's latency. The real cost is geometric_product at 33,986
calls/turn (~73%) via cga_inner in search paths. The obvious closed form is NOT
bit-exact (954/4000 in f32); backend.vault_recall's serial fold IS (3000/3000,
worst-rel 0) and is the correct target. cargo test could not run —
static.crates.io is denied by the sandbox network policy — so the Rust parity
question stays open and the typestate lane is carried forward, not shipped
uncompiled.

## Not landed: three lines owed to .github/workflows/smoke.yml

The CI smoke gate is narrower than the local one —
test_pack_draft_serve_boundary.py (ADR-0253 INV-33) has been local-only, unseen
because the parity pin checked one direction. The edit was authored and rejected
at push for lacking the `workflow` OAuth scope, so it is recorded as a named,
dated PENDING_IN_CI exception rather than dropped: the assertion still fires on
any new divergence, and a second guard fires once the three land.

[Verification]: pre-push gates all green — smoke 236 passed, warmed_session 10
passed, deductive 285 passed. Ratification 14, ADR index 5, CLI suites 10.
Grounding/epistemic sweep 741 passed 1 skipped. workbench-ui 598 passed across
73 files, tsc -b clean. capability index 11 passed, digest unchanged. Math
holdout correct=5 wrong=0 refused=495. Committed chain corpora byte-unchanged
after the tests that write to them.
Environment caveat: the repo pins requires-python ==3.12.13, which uv cannot
fetch for linux-x86_64, so `uv sync --locked` fails. All Python runs used a
scratch venv on 3.12.11 with declared deps — not the locked universe, not the
full ~12k suite. The pin was left untouched. Re-run on a 3.12.13 host before
treating this as merge evidence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

ci(local): one-command local-first runner; fix a lane roster red on main ... 7bc7131fd9

Two things, both surfaced by finally being able to run the tree.

## A pre-existing silent red

tests/test_lane_sha_verifier.py::TestExpectedLaneCoverage was RED on clean
main — confirmed by running it in a pristine worktree at origin/main, and the
file is untouched by any of this branch's work. deduction_serve_v1 (ADR-0256)
and curriculum_serve_v1 (ADR-0262) shipped during the generalization arc and
neither was added to EXPECTED_LANES.

The roster's `extra` assertion is a deliberate tripwire: a new lane is supposed
to fail it once so an author acknowledges the addition. Nobody did, and nothing
caught it, because that file is in neither `smoke` nor `deductive` — no local
gate and no CI job runs it. Same silent-red family as the exact-tuple pin S5
found in passing, and as the register-axis e2e tests that sat red for two days.
This commit is the acknowledgement the tripwire was asking for.

That it took running the full tree to find is the argument for the rest of this
commit.

## scripts/ci/local-ci.sh

AGENTS.md says the merge bar IS the local run. There was no single entry point
for it, and on any host without Python 3.12.13 exactly there was no local run
at all: pyproject pins requires-python == "3.12.13", so uv sync --locked fails
outright and every gate becomes unrunnable.

  sh scripts/ci/local-ci.sh --tier smoke|gate|full

`gate` is the three pre-push steps; `full` is the whole tree in parallel. Suite
membership is read from core/cli_test.py::TEST_SUITES through the CLI and never
restated, so this runner cannot drift from the hook the way smoke.yml drifted
from the local list.

Interpreter contract, deliberately fail-closed: the pinned interpreter is
CANONICAL, and the runner refuses on anything else with instructions.
--allow-interpreter-fallback opts into any 3.12.x and stamps every run
NON-CANONICAL. Same discipline as ratified_ledger's missing_ok and the MLX
skip path: a degraded mode is legitimate, a degraded mode reporting itself as
the real thing is not.

Evidence recorded in the script header so the flag is not cargo-culted: the
committed-SHA / trace_hash / content_sha256 / lane-SHA pin set (119 tests) was
run on 3.12.11 and 118 passed, the one failure being the stale roster above,
which fails on 3.12.13 too. That is evidence the exact pin is not load-bearing
for bit-exactness — it is NOT a ruling that it should be relaxed. Relaxing it
is a reproducibility decision and belongs to a human.

## Observed, not fixed

Something in the suite writes a real proposal artifact into
teaching/proposals/derived_close_facts/ instead of a tmp_path. The three test
files that name that sink all override it correctly, so the leak is elsewhere —
likely an idle_tick/contemplation path exercising DEFAULT_SINK. Untracked
residue, removed here, flagged for follow-up. Notable because it is only
visible now that the parents[3] bug is fixed and the sink resolves inside the
repo.

[Verification]: gate tier via the new runner — smoke 236, warmed_session 10,
deductive 285, all passed, correctly stamped NON-CANONICAL. Fail-closed path
verified (exit 3 with instructions). lane_sha_verifier 6 passed. Full tests/
tree in progress at 63% with 0 failures at time of commit.
NON-CANONICAL: Python 3.12.11, not the pinned 3.12.13. Not merge evidence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

fix(tests): stop a test writing proposals into the live in-repo sink ... b318501c55

Found by running the full tree: every full-suite run left an untracked
speculative proposal artifact in teaching/proposals/derived_close_facts/.

test_runtime_flag_on_emits_after_consolidation passes engine_state_path=tmp_path,
but that does not redirect the proposal sink — the runtime calls
emit_derived_close_proposals(ctx) with no sink argument, so it resolves
DEFAULT_SINK: the real, repo-relative directory. Fixed by monkeypatching the
module-level default the runtime resolves through.

Its neighbour was defending against exactly this with:

    assert not any(sink.glob("*.json")) or True   # may have pre-existing from other tests

which is vacuously true, and whose comment was describing the leak it was
papering over rather than the contract it claimed to check. With the flag-ON
test no longer writing there, an honest assertion is possible: snapshot the
live sink BEFORE the tick and require it unchanged after. (First attempt at
this took the snapshot after idle_tick(), which compares the sink to itself —
corrected here.)

Only visible now that the parents[3] DEFAULT_SINK bug is fixed and the sink
resolves inside the repo; before 2026-07-24 these artifacts were landing
outside the tree entirely, which is why nobody saw them.

[Verification]: 28 passed across test_derived_close_proposals,
test_proposal_queue, test_idle_proposal_review; working tree clean afterward,
no residue.
NON-CANONICAL: Python 3.12.11, not the pinned 3.12.13.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

revert(tests): drop the local-to-CI smoke parity assertion — CI is not a gate ... 50fa287d17

Reverts the bidirectional half of the smoke parity pin added earlier today,
and records why at the assertion site so it is not "completed" again.

I read the suite list's comment ("so the local-first pre-push gate equals the
CI gate") as promising equality, made the assertion symmetric, and reported
tests/test_pack_draft_serve_boundary.py as drift because it sits in the local
gate and not in .github/workflows/smoke.yml.

That was wrong, and AGENTS.md says so in a line I had already read: GitHub
Actions are billing-locked and produce dead signals, never chase them. The
Forgejo host (e2-micro) cannot run workflows either. CI here is run MANUALLY
in-worktree — usually faster anyway on UMA hardware with real RAM — which is
what scripts/ci/local-ci.sh now front-ends. smoke.yml is secondary
observability, not a gate.

So the local suite is the SOURCE and is free to be broader. Asserting the
reverse converts a file nobody executes into a maintenance obligation, and one
agents cannot discharge at that — pushing workflow changes needs an OAuth scope
the push credential lacks. The three "owed" lines were never owed.

The surviving direction (CI subset-of local) still protects something real: if
smoke.yml ever names a path the local gate lacks, the local gate is narrower
than a published claim about it.

Arc-close brief updated to match: the "three files owed to smoke.yml" open item
is replaced by a recorded non-finding, so the next reader does not re-derive it.
Its shipped-work table now lists what actually landed in its place — the local
CI runner, the lane-roster fix, and the proposal-sink test-isolation fix.

[Verification]: 21 passed across test_cli_test_suites, test_lane_sha_verifier,
test_adr_index; ruff clean on the changed test.
NON-CANONICAL: Python 3.12.11, not the pinned 3.12.13 (unavailable for
linux-x86_64). Hash-pinned tests reproduce committed values regardless.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

build: relax requires-python from ==3.12.13 to >=3.12,<3.13 ... 3b2c7608a9

The exact patch pin made every gate unrunnable on any host without that precise
build. cpython-3.12.13 is not published for linux-x86_64, so `uv sync --locked`
failed outright there and there was no local run at all — for a project whose
merge bar IS the local run, that is the failure mode that matters. It is how
test_lane_sha_verifier.py and 99 register-matrix tests sat red on clean main
without anyone seeing them.

Safe because the pin was not load-bearing for reproducibility: the committed
trace_hash / content_sha256 / lane-SHA pins were established under 3.12.13 and
reproduce byte-for-byte under 3.12.11 (119 hash-pinned tests, 118 pass; the one
failure is a stale lane roster that fails on 3.12.13 too). The minor version
stays bounded — 3.13 is a real upgrade decision, not a patch bump.

.python-version follows to "3.12" so uv resolves any patch release.

[Verification]: `uv lock` regenerated — the ENTIRE diff is one line,
requires-python "==3.12.13" -> "==3.12.*". All 217 packages resolve to
identical versions; no dependency moved. `uv sync --locked` now succeeds on
linux-x86_64, which it could not do before this change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

fix(cognition): expose hash_surface; point the register-matrix pin at it ... bf17f42834

*** VERIFICATION PENDING — see the bottom of this message. ***

99 of the 107 full-suite failures are one test, red on clean main since
2026-07-23, and it is guarding the wrong field.

WHAT IS NOT WRONG. test_register_matrix_trace_hash_invariant — the load-bearing
ADR-0072 truth-path-isolation claim — passes on all 99 registers on pristine
main. trace_hash does not vary across the register axis. The invariant holds;
the seals and wrong=0 are untouched.

WHAT IS. test_register_matrix_canonical_surface_byte_identical asserts on
CognitiveTurnResult.surface, on its docstring's claim that this is "the
truth-path field compute_trace_hash consumes". That stopped being true when
Phase 0 flipped resolve_surface to response-first precedence and introduced
SurfaceResolution.hash_surface as the register-invariant capture. Since then
pipeline.py folds hash_surface into compute_trace_hash, while result.py
documents surface as "final voiced surface (what the user sees)".

So the test demanded that registers NOT differ on the served surface — the
opposite of what the register axis exists for, and a direct contradiction of
test_register_substantive_consumption.py, which pins that they DO differ and is
green in smoke. Two tests on main asserting opposite things about the same
bytes; the red one is in neither gate, so it stayed red.

ROOT CAUSE, and why this is a code fix rather than a test edit: hash_surface was
pipeline-local. The invariant was unobservable from a turn result, which is why
the test reached for the nearest visible field and drifted. Exposing it makes
the contract testable at the seam that owns it:

- CognitiveTurnResult.hash_surface — new field, populated from the value the
  pipeline already computed
- evals/cognition CaseResult carries it through
- the test asserts byte-identity on hash_surface, with the history recorded so
  it cannot drift back

No stored baseline to regenerate — the fixture computes it live.

[Verification]: NOT YET RUN. The register matrix is ~16 min and was still
executing when the working tree had to be committed; its output buffers through
tail, so no partial signal was available. Committed to preserve the work with an
honest label rather than to claim it passes. DO NOT MERGE until
`pytest tests/test_cognition_eval_register_matrix.py` is green — expected 0
failed where main has 99, with test_register_matrix_trace_hash_invariant still
passing. Result to follow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

fix(evals): carry hash_surface on the CaseResult the register matrix actually uses ... bec76d4889

Corrects bf17f42, which claimed a fix that did not work.

That commit added hash_surface to evals/cognition/runner.py::CaseResult. The
test imports run_eval from evals/run_cognition_eval.py, which builds
evals/metrics.py::CaseResult — a DIFFERENT class with the same name. Thirteen
files in this repo define one. I matched on the class name instead of following
the import, so the field never reached the assertion and _diff_rows raised
AttributeError: 'CaseResult' object has no attribute 'hash_surface'. That is why
the count went 99 -> 100: the same 99 divergences, plus one error.

The diagnosis in bf17f42 was right; only its aim was wrong. Confirmed directly
at the pipeline seam:

  register=None         surface = 'Truth is what is true. pack-grounded...'
                   hash_surface = 'Truth is what is true. pack-grounded...'
  register=socratic_v1  surface = 'Consider the question: Truth is what is...'
                   hash_surface = 'Truth is what is true. pack-grounded...'

surface varies across the register axis (as it must), hash_surface does not (as
it must not), trace_hash matches. So the field added to CognitiveTurnResult in
bf17f42 is correct and stays; this moves the eval-side plumbing to the class the
test really consumes, and reverts the edit to the one it does not.

[Verification]: PARTIAL. Three registers green — socratic_v1, terse_v1,
manifesto_v1, 24 passed, where all three failed before. The full 99-register run
(~17 min) was still executing when the tree had to be committed; result to
follow, and bf17f42's DO NOT MERGE stands until it is green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

docs(register-matrix): 801 passed, 0 failed — clears the DO NOT MERGE on bf17f42 ... bbca86b72f

The full 99-register run is green: 801 passed, 0 failed, up from 99 failures on
clean main. bf17f42's "DO NOT MERGE until green" is hereby satisfied; the fix in
bf17f42 + bec76d4 is verified.

Records the whole thing, including what was NOT broken: the load-bearing
ADR-0072 claim (test_register_matrix_trace_hash_invariant) passed throughout, on
main and after. trace_hash folds hash_surface, so a real truth-path leak would
have moved it, and it never did. The invariant was intact the entire time — it
just was not observable, because hash_surface was pipeline-local, which is why
the test drifted onto `surface` when Phase 0 changed what that field means.

Also records the wrong turn: the first patch aimed at
evals/cognition/runner.py::CaseResult, while the test consumes
evals/metrics.py::CaseResult via run_cognition_eval. Thirteen files here define
a class called CaseResult. Follow the import, not the name.

[Verification]: register matrix 801 passed / 0 failed (17m50s), including the
trace_hash invariant across all 99 registers; capability index + eval harness 23
passed. evals/metrics.py::CaseResult gained a DEFAULTED field and its only other
consumers (calibration/tune.py, calibration/replay.py) never construct it.
NON-CANONICAL: Python 3.12.11.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

fix(tests): clear the last reds — a machine-specific path and a stale drift pin ... b10cbde6b8

Two of main's remaining failures, diagnosed rather than silenced. Neither was a
broken invariant.

## test_adr_0119_1_sealed_holdout (2) — hardcoded developer-machine path

    IDENTITY_PATH = Path("/Users/kaizenpro/.config/core/holdout_keys/repo_holdout.txt")

An absolute path to one laptop, in a test. It passed there and failed
unconditionally on every other machine — a second developer, a fresh clone, any
agent session. Now read from CORE_HOLDOUT_KEY (the same env var
evals/holdout_runner.py already uses) with ~/.config/... as the fallback, which
still resolves to the original location on the authoring machine, so those tests
keep running for real there.

Absent the key the tests now SKIP with the reason stated. A machine without the
holdout key cannot run them; that is not the same as the sealed-holdout contract
being violated, and the two must not report identically.
evals/holdout_runner.py is untouched — it still refuses any plaintext fallback
once an identity is supplied. Only the tests' ability to execute is conditional.

## test_adr_0244_gamma_calibration (1) — genuine drift, re-calibrated

The first TEN pinned leakage values are byte-identical to the 2026-07-17 pin.
Divergence starts at probe turn 12 ("ice is cold") and persists — the signature
of a state trajectory splitting at one turn, not a numeric wobble. Deterministic
across runs and IDENTICAL on pristine main, so it is engine drift accumulated
across the generalization arc, not a change from this branch.

Re-pinned, because this is the guard's own documented action ("regenerate with
collect_live_benign_leakages()"; the test says a hit means "re-calibrate and
reconsider the flag flip"), and because the conclusion it protects is unchanged:
benign leakage still overlaps the attack distribution, so flag_flip_authorized
stays False and identity_wave_gate stays OFF. Reconsidered; still not authorized.
These are measurements of an EXPERIMENTAL off-by-default path where every probe
turn is a boundary-violated refusal — that over-triggering on benign traffic is
the finding the calibration exists to record.

Recorded honestly in the constant's comment: the turn where drift begins is
known, the arc commit that caused it is NOT isolated, and the comment says so.

## Not fixed here, because they were already fixed

test_issue_300_versor_margin (3) and test_adr_0172_w0_1_trace_replay_equivalence
(1) were subprocesses shelling out to `uv run core chat` and dying on
"No interpreter found for Python 3.12.13" — the exact-pin problem. Both pass on
this branch already, fixed by 3b2c760. Eight such failures in the main baseline
traced to that one cause.

## Observation, not changed

tests/test_adr_0119_1_sealed_holdout.py carries the full decrypted holdout cases
as EXPECTED_PLAINTEXT, in cleartext, next to the .age file they are encrypted
into. That may be deliberate; flagging rather than touching it.

[Verification]: the six formerly-red files — versor_margin, trace_replay,
gamma_calibration, sealed_holdout, lane_sha_verifier, derived_close_proposals —
44 passed, 2 skipped (holdout key absent, reason stated). Full-tree run in
progress.
NON-CANONICAL: Python 3.12.11.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx

core-labs merged commit 116cd18996 into main 2026-07-25 21:11:17 +00:00

core-labs deleted branch claude/core-architecture-assessment-recovered 2026-07-25 21:11:17 +00:00

core-labs referenced this pull request from a commit 2026-07-25 21:11:19 +00:00

Merge pull request 'assessment: independent Tier-S verification, ratification ceremony, local-first CI runner (recovered from cloud session)' (#113) from claude/core-architecture-assessment-recovered into main

core-labs referenced this pull request from a commit 2026-07-25 21:43:48 +00:00

chore(governance): stamp nine merged ADRs Accepted + pin the invariant

core-labs referenced this pull request 2026-07-25 21:44:10 +00:00

chore(governance): stamp nine merged ADRs Accepted + pin the invariant in smoke #114

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: core-labs/core#113

No description provided.