core/docs/handoff/BRIEF-CLOSE-assessment-verification-2026-07-25.md
Claude 50fa287d17
revert(tests): drop the local-to-CI smoke parity assertion — CI is not a gate
Reverts the bidirectional half of the smoke parity pin added earlier today,
and records why at the assertion site so it is not "completed" again.

I read the suite list's comment ("so the local-first pre-push gate equals the
CI gate") as promising equality, made the assertion symmetric, and reported
tests/test_pack_draft_serve_boundary.py as drift because it sits in the local
gate and not in .github/workflows/smoke.yml.

That was wrong, and AGENTS.md says so in a line I had already read: GitHub
Actions are billing-locked and produce dead signals, never chase them. The
Forgejo host (e2-micro) cannot run workflows either. CI here is run MANUALLY
in-worktree — usually faster anyway on UMA hardware with real RAM — which is
what scripts/ci/local-ci.sh now front-ends. smoke.yml is secondary
observability, not a gate.

So the local suite is the SOURCE and is free to be broader. Asserting the
reverse converts a file nobody executes into a maintenance obligation, and one
agents cannot discharge at that — pushing workflow changes needs an OAuth scope
the push credential lacks. The three "owed" lines were never owed.

The surviving direction (CI subset-of local) still protects something real: if
smoke.yml ever names a path the local gate lacks, the local gate is narrower
than a published claim about it.

Arc-close brief updated to match: the "three files owed to smoke.yml" open item
is replaced by a recorded non-finding, so the next reader does not re-derive it.
Its shipped-work table now lists what actually landed in its place — the local
CI runner, the lane-roster fix, and the proposal-sink test-isolation fix.

[Verification]: 21 passed across test_cli_test_suites, test_lane_sha_verifier,
test_adr_index; ruff clean on the changed test.
NON-CANONICAL: Python 3.12.11, not the pinned 3.12.13 (unavailable for
linux-x86_64). Hash-pinned tests reproduce committed values regardless.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FduW6Krm3PPQv3P5iwBYtx
2026-07-25 13:44:30 +00:00

11 KiB
Raw Blame History

Arc-Close Brief — Assessment Verification & Lateral Closures

Arc: assessment-verification-2026-07-25 · Closed: 2026-07-25 · Closing tier: Opus 5 Plan of record: independent verification of an external Tier-S assessment + its five-pillar blueprint ADRs minted: none (one amendment: ADR-0263 gains rule 5, recorded in core/ratified_ledger.py)

First use of ARC-CLOSE-TEMPLATE.md, written in the same arc. Filling it in was also the test of it.

1. What shipped

Capability Enforcement site Evidence
deduction/curriculum registered as grounding sources, cross-stack core/epistemic_state.py, workbench-ui/ badges + snapshot tests/test_workbench_deduction_provenance.py (14)
Coercion derives its whitelist from the enum instead of restating it workbench/api.py::_coerce_grounding_source same file, parametrized over the registration
Ledger absence policy declared, not passed (ADR-0263 rule 5) core/ratified_ledger.py::CAPABILITY_LEDGERS tests/test_ratified_ledger_bridge.py::TestCapabilityManifest
Deductive suite in the pre-push gate scripts/hooks/pre-push measured 285 tests / 29s vs smoke 216 / 62s
One-command local CI runner, fail-closed on interpreter scripts/ci/local-ci.sh gate tier verified: 236 + 10 + 285
Lane roster red on clean main fixed tests/test_lane_sha_verifier.py two arc lanes were never registered
Test writing proposals into the live in-repo sink tests/test_derived_close_proposals.py plus a vacuous or True assertion removed
Correction-binding lockstep pinned on the deduction path tests/test_prior_surface_deduction_binding.py (6) mutation-checked
Ratification ceremony + core proposal-queue ratify teaching/ratification.py, core/cli_proposal_queue.py tests/test_ratification_ceremony.py (14)
Domain-keyed ADR index docs/adr/INDEX-by-domain.md tests/test_adr_index.py (5)
Arc-close brief template docs/handoff/ARC-CLOSE-TEMPLATE.md this file

2. Flag state at close

Flag Default at close Ratified by Notes
deduction_serving_enabled True ADR-0256, 2026-07-24 Unchanged. Its suite is now a pre-push gate, not async CI.
curriculum_serving_enabled False Unchanged. Blocked on volume, not machinery; the ceremony is the volume path.
accrue_realized_knowledge False Unchanged, and not debt: a deployment profile (session memory), ON in production L10.

No flag was flipped in this arc.

3. Falsified assumptions

The assessment assumed Measurement showed Evidence
parents[n] audit is high-payoff 168 sites, 0 outside the repo root audit run
Guard exemption undocumented Documented at both guard sites with rationale chat/runtime.py:2371, :3007
T12 ledger entry missing Present as an open pattern-watch item weekly-audit-2026-07-22-stragglers-todo.md:170
assert_corpus_sound has one impl Already domain-parameterized, called unconditionally evals/curriculum_serve/runner.py:71,105-108
Promotion oracle must be built Already the wrong=0 lane gate tier-s-housekeeping-2026-07-24.md §3
EntailmentTrace drops proof chains ROBDD tautology check; no intermediate steps exist generate/proof_chain/entail.py:125-190
Cross-subject proof could emerge accidentally resolve_domain refuses ambiguous_reading chat/curriculum_surface.py:138
Second subject arena is next 11 bands across 4 subjects already route; all 24×73× short curriculum-volume-quantification-2026-07-24.md §2
(mine) versor_condition is the hot spot 0.22% of a turn cga-hot-path-measurement-2026-07-25.md §2
(mine) Workbench also falsifies epistemic_state It does not — reads epistemic_state_needed, honest workbench/api.py:818

4. Binding constraint going into the next arc

Unchanged and unmoved: ratified curriculum volume. Eleven live bands, every one 24×73× short of the entailed-bucket floor, and physics — the deepest — has 16 chains against a ~219 floor. This arc built the machine that converts a reviewed decision into a routable chain and proved it moves the band (causal: 7 → 8 on a dry run), but it ratified no content. The ceremony is a pipe; someone still has to put curriculum through it. Nothing else in the roadmap unblocks until that number moves.

5. Trigger state

Core Logos Phase 5 (multi-step articulation) is NOT triggered, contrary to the assessment's reading. The trigger was described as "the renderer discards intermediate proof chains" — there are none to discard. evaluate_entailment_with_trace decides by asking whether (P1 & … & Pn) -> Q is a tautology; EntailmentTrace carries five opaque BDD node keys.

What must land in a higher tier before any implementation: a feasibility study for proof-term extraction over the ROBDD — a derivation calculus with a real soundness surface, not a data-structure refactor. Scoping it as the latter produces a renderer with nothing to render.

6. What is NOT next, and why

  • Widening the math reader for cases 0000/0001/0148/0082. Traced; their blocking gaps affect 1 and 2 cases out of 500. ADR-0251's prohibition now rests on a count.
  • A .metal Cl(4,1) kernel / MLX fusion / bf16. The Cayley table already exists at compile time in cl41.rs; the fusion target is 0.22% of a turn; bf16's ε ≈ 7.8e-3 against a 1e-6 gate. And there is an exact CPU win available first.
  • The diagonal cga_inner shortcut. Tempting, 32× fewer ops, and not bit-exact (954/4000 in f32). It would move lane pins.
  • Discovery-yield stratification. Blocked on new persisted per-source counters in the engine_state manifest.
  • Band-level capability index. Blocked on a scoring decision — 25 near-1.0 entries inflate coverage_geomean against the index's own anti-gaming claim.
  • engine_refused as a standing gate. Classifier already CI-gated; pinning a count on an empty bucket is the stale-pin failure mode.

7. Open items carried forward

Item Kind Recorded
Does core_rs hold bit-exact parity? blocked (network policy denies static.crates.io) cga-hot-path-measurement-2026-07-25.md §7
Rust typestate UnverifiedClaimVersorClaim blocked on a build same, §7
Serial-fold cga_inner in the search paths untested (exactness proven, change not made) same, §6
Audit-ledger R7 — telemetry sink gets pre-override events deferred docs/specs/runtime_contracts.md
hash_surface on TurnEvent, or not undecided same
Arena queue entry + ledger reseal after ratification deferred by design (bridge rule 1) RatificationReceipt.pending_stages
Forgejo zero-length-object claim unverified from this environment assessment doc, preamble
A non-finding, recorded so it is not re-found. Mid-arc this brief listed
"three files owed to smoke.yml" as blocked work: test_pack_draft_serve_boundary.py
plus the two new provenance files sit in the local gate and not in the CI yaml,
which a newly-bidirectional parity assertion flagged as drift.

It is not drift, and the assertion was wrong to exist. Per AGENTS.md, GitHub Actions are billing-locked dead signals, the Forgejo host cannot run workflows either, and CI is run manually in-worktree — which is faster anyway on UMA hardware with real RAM. smoke.yml is secondary observability, not a gate. The local suite is the source and is free to be broader; asserting the reverse turns a file nobody executes into a maintenance obligation that agents cannot even discharge (pushing workflow changes needs an OAuth scope the push credential lacks).

Reverted the same day, with the reasoning left at the assertion site so the "completion" is not attempted again. The surviving direction — CI ⊆ local — still protects something real: if smoke.yml ever names a path the local gate lacks, the local gate is narrower than a published claim about it.

8. Gates and their state at close

Gate Command Result
smoke core test --suite smoke -q 236 passed (216 baseline + 20 new)
deductive core test --suite deductive -q 285 passed
workbench-ui vitest run 598 passed / 73 files; tsc -b clean
capability index pytest tests/test_capability_* 11 passed, baseline digest unchanged
math holdout evals/gsm8k_math/holdout_dev/v1/runner.py correct=5 wrong=0 refused=495
proposal/curriculum sweep pytest -k "proposal_queue or ratification or curriculum" 246 passed
grounding/epistemic sweep pytest -k "workbench or epistemic or grounding or …" 741 passed, 1 skipped

Environment caveat, and it matters for how much these are worth. The repo pins requires-python == 3.12.13, which uv cannot fetch for linux-x86_64, so uv sync --locked fails. All Python runs above used a scratch venv on 3.12.11 with the declared deps — not the locked dependency universe, and not the full ~12k suite. The pin was left untouched. Re-run the real gates on a 3.12.13 host before treating any of this as merged-quality evidence.

9. Ground truth corrections for the next opener

  1. chat/runtime.py:471-481 was stale and is now rewritten. It argued the unregistered deduction label was "inert today" because REPL turns do not reach Workbench. True, and irrelevant — Workbench builds a bare ChatRuntime(), so the traffic flows the other way. Live from the moment the flag was ratified ON. This is the class of staleness nobody greps for: a comment reasoning from a premise that has since changed.

  2. A second copy of a closed enum is worse than no copy. The unregistered path degraded honestly (epistemic_state_needed = "needs determination"). The hand-copied whitelist asserted a falsehood (none = ungrounded). Derive from the enum; do not restate it.

  3. Silent-drop is correct at serving time and a trap at ratification time. _ratified_rows dropping unadmissible rows is right when serving. Appending a row the loader will drop looks identical to appending one that works. Any future corpus-writing path needs the same confirm-admission discipline.

  4. Do not compare a microbenchmark to a latency figure. §3's last two rows are both mine, and both came from reasoning about measurements instead of taking them. The turn is the denominator.

  5. assert_corpus_sound is two functions with one nameevals/deduction_serve/practice/gold.py:1163 (no args, practice corpus) and evals/curriculum_serve/runner.py:71 (domain, cases, lane contract). Not a missing contract; a name collision. Worth renaming before a third appears.