cbfc8ccbf7
The CORE-Logos "Crown Proof" — ADR-0015's claim that cross-language holonomy
closure is THE validation gate of meaning — was measured against a criterion
registered before the run, and refused.
G1 separation, all negatives AUC 0.557 (chance 0.500, bar 0.80) FAIL
G2 hardest class, cross-pairing AUC 0.664 (bar 0.75) FAIL
G3 word-order sensitivity 0.644 (bar 0.90) FAIL
G4 no collapse re-entry 0 lost (bar: 0) PASS
G4 is the half that succeeded: the repairs work. The claim is what failed --
which is the outcome the registration called "worth as much as a GO", because
it retires the largest piece of unearned design in the system.
Both preconditions had to be BUILT before the question could be asked at all,
which is why the 2026-06-14 negative constrained nothing: the shipped compiler
collapsed 37-53 coordinates, and holonomy_encode never closed. Measured on the
repaired ground with a genuine loop, over 1,016 aligned pairs and 58,375
mechanically-generated negatives in three classes, nothing hand-picked.
Diagnosis, not just a number: the encoding reacts more strongly to REORDERING a
clause (median deviation 41.2) than to CHANGING WHAT IT IS ABOUT (32.2). It
measures path shape, not content -- and that follows from the algebra, since
permutation acts on an ordered product through non-commutativity, a first-order
effect, while substituting one of three factors perturbs it only through that
factor. The negative classes do order correctly by meaning-distance
(aligned 27.2 < lexical 32.2 < order 41.2 < cross-pair 53.2), so the geometry is
weak signal rather than noise -- a correlation where the design asked for a gate.
Consequences, all landed here:
* ADR-0005 and ADR-0015 AMENDED to record the refusal. The three-language
architecture, the pack contract, morphology-as-structure and Hebrew root
folding into geometry all survive untouched; only the claim that alignment
or closure VALIDATES meaning is retired.
* tests/test_fa1_gate_verdict.py is the tripwire: if a future encoding makes
the gate real, it fails, and the amendment is withdrawn in the same commit
that replaces it with a proof. Two sabotages observed red.
* allow_cross_language_recall RULED by measurement (work-order item 3): it
gates vault-recall depth over field states, has no language argument and no
cross-language path, and reaches only walk_surface, which runtime_contracts
declares telemetry. Stays ON, reclassified DEPLOYMENT, flip condition
recorded. G-25's own claim that it was "the pillar's own switch" is
CORRECTED -- that read the flag's name instead of its call graph, which is
the error the audit exists to find.
* A sixth defect found while building the harness: en_collapse_anchors_v1
declares role "collapse_anchor", which is not a LanguageRole member, so
load_pack() raises. It is registered in the resolver, consumed by opening
lexicon.jsonl on a raw path that bypasses the loader, and is the target of
all 24 alignment edges that still resolve to nothing. The enum's own comment
records the identical failure once before (ADR-0097, domain_seed).
The repairs cross to the keel on their own merit regardless of the verdict:
geodesic blending removes every coordinate collision, and mount-wide edge
resolution connects 39 of the 63 dead edges. The collapse was damaging English's
own vocabulary; seeing that required no theory of meaning.
FA-1 is CLOSED: all four work-order items discharged, L2 verdict recorded as
defective-but-repairable with its central design claim retired.
[Verification]: uv run core test --suite smoke -q -> 786 passed in 239.45s, EXIT=0
(+4 tests, +15s vs the prior 782/224s -- the FA-1 pin's stated cost).
Doc-parsing pins re-run after the ADR and register amendments:
test_adr_status_governance + test_adr_index -> 321 passed;
test_flag_register -> 6 passed.
125 lines
6.8 KiB
Python
125 lines
6.8 KiB
Python
"""FA-1's verdict, pinned as a tripwire (G-25, L2, the CORE-Logos validation-gate claim).
|
|
|
|
Measured 2026-07-28 against the criterion registered at `94f05ba8`: on a semantic ground with
|
|
**zero** coordinate collisions and a genuinely closed loop, cross-language holonomy separates
|
|
aligned clauses from meaning-breaking ones at **AUC 0.557** — a chance floor is 0.500. The
|
|
design's own word-order test holds only **64.4%** of the time. **NO-GO.**
|
|
|
|
Full result and diagnosis: `docs/analysis/fa1-holonomy-gate-verdict-2026-07-28.md`.
|
|
|
|
This pin exists for the reason the 2026-06-14 holonomy finding gave for its own tripwires: a
|
|
negative result that nothing checks decays into folklore, and the *interesting* failure mode
|
|
is the good one — a future encoding that makes the gate real, landing silently while four
|
|
documents still say it was measured and refused. If any of these assertions goes red because
|
|
the numbers improved, that is not a regression: replace the guard with a proof, amend the ADRs
|
|
back, and update the verdict document in the same commit.
|
|
|
|
Both directions are pinned. The verdict may not silently become GO, and the measured values
|
|
may not silently drift either — the gate is deterministic, so any movement means the ground,
|
|
the encoder, or the corpus changed, and each of those is a decision someone should be making
|
|
on purpose.
|
|
|
|
Cost: ~16s. Paid deliberately for the tripwire on the system's central design claim, the way
|
|
the daemon pin's 9.4s was paid — stated here rather than estimated.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from evals.logos.fa1_gate import G1_AUC, G2_AUC, G3_ORDER_SENSITIVITY, run
|
|
|
|
#: Measured 2026-07-28 at `94f05ba8`+1, deterministic and re-run bit-identical.
|
|
RECORDED = {
|
|
"G1_separation": 0.5570894467955959,
|
|
"G2_cross_pair": 0.6641552204384346,
|
|
"G3_word_order": 0.6444881889763779,
|
|
"G4_no_collapse": 0,
|
|
}
|
|
RECORDED_ALIGNED_PAIRS = 1016
|
|
RECORDED_CONCEPTS = 24
|
|
RECORDED_UNRESOLVED_EDGES = 24 # every one of them into the unloadable collapse-anchor pack
|
|
|
|
|
|
@pytest.fixture(scope="module")
|
|
def verdict() -> dict:
|
|
return run()
|
|
|
|
|
|
def test_the_repairs_still_work(verdict: dict) -> None:
|
|
"""G4 and the controls. If these fail, the NO-GO below is uninterpretable.
|
|
|
|
This is the half of the result that SUCCEEDED: the geodesic blend keeps every distinction
|
|
(0 lost coordinates where the shipped compiler loses 53 on this mount), and the closed loop
|
|
reports closure on a self-pair. A null result measured with a broken instrument constrains
|
|
nothing, which is exactly why the June measurement had to be re-run.
|
|
"""
|
|
assert verdict["ground_after_repair"]["coordinates_lost"] == 0, (
|
|
"G4 FAILED: the repaired ground now collapses coordinates. Every discrimination number "
|
|
"in this file is measured on a different ground than the verdict document describes."
|
|
)
|
|
assert verdict["controls"]["max_self_loop_deviation"] < 1e-4, (
|
|
"the loop no longer closes on a self-pair — the metric is broken, not the hypothesis"
|
|
)
|
|
assert verdict["controls"]["aligned_pairs"] == RECORDED_ALIGNED_PAIRS, (
|
|
f"corpus changed: {verdict['controls']['aligned_pairs']} aligned pairs vs "
|
|
f"{RECORDED_ALIGNED_PAIRS} recorded — re-baseline deliberately, in the verdict doc too"
|
|
)
|
|
|
|
|
|
def test_the_verdict_is_still_no_go(verdict: dict) -> None:
|
|
"""The tripwire. Red here means the design became real — go read the failure message."""
|
|
assert verdict["VERDICT"] == "NO-GO", (
|
|
"FA-1's holonomy gate now PASSES. This is good news and it is not a test failure to "
|
|
"silence: replace this guard with a proof, amend ADR-0005/0015 back (they were amended "
|
|
"to record the negative), and rewrite "
|
|
"docs/analysis/fa1-holonomy-gate-verdict-2026-07-28.md in the same commit."
|
|
)
|
|
criterion = verdict["criterion"]
|
|
assert not criterion["G1_separation"]["pass"], f"G1 now passes at {criterion['G1_separation']['observed']}"
|
|
assert not criterion["G2_cross_pair"]["pass"], f"G2 now passes at {criterion['G2_cross_pair']['observed']}"
|
|
assert not criterion["G3_word_order"]["pass"], f"G3 now passes at {criterion['G3_word_order']['observed']}"
|
|
|
|
|
|
def test_the_recorded_numbers_have_not_drifted(verdict: dict) -> None:
|
|
"""Deterministic gate, exact record. Movement in either direction is a reviewed decision."""
|
|
for gate, expected in RECORDED.items():
|
|
observed = verdict["criterion"][gate]["observed"]
|
|
assert observed == pytest.approx(expected, abs=1e-9), (
|
|
f"{gate} moved: {observed} vs {expected} recorded. The gate is deterministic, so "
|
|
"the ground, the encoder, or the corpus changed. Update this constant and the "
|
|
"verdict document together — two records that disagree is the G-22 disease."
|
|
)
|
|
assert verdict["corpus"]["concepts"] == RECORDED_CONCEPTS
|
|
assert verdict["corpus"]["edges_unresolved_under_R3"] == RECORDED_UNRESOLVED_EDGES, (
|
|
"the unresolved-edge count moved. All 24 point into en_collapse_anchors_v1, which "
|
|
"cannot load because its manifest declares role='collapse_anchor' and that is not a "
|
|
"member of packs.schema.LanguageRole. If this dropped to 0 the enum was widened — good; "
|
|
"record it, because those edges then join the corpus and every number above moves."
|
|
)
|
|
|
|
|
|
def test_the_gate_is_not_vacuous(verdict: dict) -> None:
|
|
"""A criterion that cannot pass proves nothing (R-11's lesson, PR-12's hollow guard).
|
|
|
|
The thresholds must be reachable and the classes must be ordered by meaning-distance —
|
|
otherwise "NO-GO" would be an artefact of an impossible bar rather than a measurement.
|
|
"""
|
|
assert 0.5 <= G1_AUC <= 1.0 and 0.5 <= G2_AUC <= 1.0 and 0.5 <= G3_ORDER_SENSITIVITY <= 1.0, (
|
|
"a threshold outside [0.5, 1.0] would be unreachable or trivially met by chance"
|
|
)
|
|
measurements = verdict["measurements"]
|
|
assert measurements["median_aligned"] < measurements["median_negative"], (
|
|
"aligned pairs no longer close tighter than negatives — the signal reversed sign, which "
|
|
"is a different (and larger) finding than the recorded weak-but-correct-direction one"
|
|
)
|
|
# The classes order themselves by distance from the original meaning. This is the part of
|
|
# the design that DID hold, and it is what makes the null a measurement rather than noise.
|
|
assert (
|
|
measurements["median_lexical"] < measurements["median_word_order"] < measurements["median_cross_pair"]
|
|
), (
|
|
"the negative classes no longer order by meaning-distance "
|
|
f"(lexical={measurements['median_lexical']}, order={measurements['median_word_order']}, "
|
|
f"cross={measurements['median_cross_pair']}) — re-diagnose before trusting the verdict"
|
|
)
|
|
assert verdict["controls"]["negatives_generated"]["cross_pair"] > 0, "no negatives generated"
|