* research(evals): phi separation probe for ADR-0081 follow-up
Lab artifact at evals/lab/phi_separation_probe.py. Tests whether a
candidate embedding
phi : Proposition -> Cl(4,1)
produces a contemplation differential
Delta(chain) = ||sandwich(R_connective, phi(subject)) - phi(object)||
that separates known-compatible chains from synthesized
known-contradicting twins.
Why this exists
---------------
A "Topological Stress Field" miner (read-only Rust kernel sweeping
the vault footprint and emitting SPECULATIVE findings from high-Delta
regions) was discussed as a successor to #55. That miner can only
earn its Rust cycles if Delta actually correlates with semantic
contradiction. Until phi is empirically validated, ||Delta|| is a
hash, not a signal.
This probe is the falsification harness for phi. Promotion criterion
encoded in the run output: ``auc >= 0.80`` on the pair set below
before any geometric stress miner is built.
Method
------
- 21 real chains pulled from teaching/cognition_chains/cognition_chains_v1.jsonl.
- Contradicting twins synthesized via 8 connective-antonym pairs
(requires<->rejects, reveals<->obscures, grounds<->undermines,
supports<->contradicts, enables<->prevents, confirms<->refutes,
informs<->misleads, verifies<->falsifies).
- Two phi candidates: phi.v1.summed_domains (grade-mixed sum of
CGA point embeddings of the lemma's semantic_domains) and
phi.v2.centroid_point (centroid of domain hash points embedded
once, staying on the CGA null cone).
- Two distance metrics: principled CGA point-distance and Frobenius.
Result (v1)
-----------
All four (phi, metric) combinations land at AUC ~ 0.5 (chance).
Distributions for compatible vs contradicting overlap completely
(mean diff <= 0.04). Hash-derived phi does NOT encode contradiction
under any tested metric.
This is the right kind of failure: it tells us the geometric stress
miner has no signal to consume yet, and validates the decision to
not build it speculatively.
Two side findings worth pinning
-------------------------------
1. algebra.versor.versor_apply projects non-null inputs back onto the
unit-versor manifold (runtime field-state closure), collapsing
sum-of-multivectors phi outputs to scalar identity. The probe
uses raw R*F*reverse(R) directly. Any future geometric kernel
needs a raw sandwich primitive distinct from runtime versor_apply.
2. For two CGA null vectors X, Y the correct distance is
d = sqrt(-2 * <X, Y>), not sqrt(-2 * <X-Y, X-Y>). The latter
evaluates to a negative number that f32 numerics silently clamp
to zero. First version of the probe returned identically-zero
distances because of this.
Boundary
--------
- Lives in evals/lab/ (research-only, never imported by runtime).
- No new package surface; no Rust code; no pack/vault writes.
- No tests required (lab convention); the promotion criterion in
the run output is the falsification gate.
* research(evals): add IDF-weighted phi variants (v3, v4)
Adds two more phi candidates to the separation probe:
- phi.v3.idf_weighted — sum of CGA embeddings, weighted per
semantic_domain by smoothed IDF across the pack. Same shape as
v1 (grade-mixed) but rare domains get larger weight than common
ones like ``logos.core`` that appear in most cognition lemmas.
- phi.v4.idf_centroid — null-cone sibling of v3. IDF-weighted
centroid in R^3, embedded once.
Hypothesis tested: v1's null result was "common-domain noise drowning
out the distinguishing axes."
Result
------
All four (phi, metric) combinations still at AUC ~ 0.5:
phi.v1.summed_domains cga AUC=0.481 frob AUC=0.451
phi.v2.centroid_point cga AUC=0.490 frob AUC=0.492
phi.v3.idf_weighted cga AUC=0.481 frob AUC=0.449
phi.v4.idf_centroid cga AUC=0.497 frob AUC=0.501
IDF reweighting does not separate compatible from contradicting.
Diagnostic refinement
---------------------
v4 shows compat mean (0.559) < contra mean (0.572) — directionally
correct (contradictions land farther) but the effect is dwarfed by
the within-group std (~0.24). This is a hint, not signal.
What this *does* tell us: the lemma encoding is not the load-bearing
variable. The bottleneck is the **connective rotor**. Antonym pairs
should produce rotors that send vectors in opposite directions, but
hash-derived R(requires) and R(rejects) are statistically
independent — there is no encoded relationship between a connective
and its antonym in the current scheme.
Next phi candidate worth trying: encode connectives as rotors derived
from a learned or curated antonym structure (e.g., R(antonym) =
reverse(R(original))), so the antonym structure is GEOMETRICALLY
guaranteed instead of coincidentally absent. Until something on the
rotor axis carries structural signal, varying only the lemma
encoding is rearranging deck chairs.
* research(evals): antonym-rotor oracle variants (v5, v6)
Adds two upper-bound probes that hardcode the antonym structure
into rotor space:
R(antonym) := reverse(R(canonical))
so the antonym relationship is geometrically guaranteed instead
of coincidentally absent. This is NOT a phi proposal — it is an
oracle probe. What it measures: "if antonym relations *were*
perfectly encoded geometrically, would the rest of the encoding
separate the two groups?"
Variants:
- phi.v5.centroid_antonym_oracle — v2 lemmas + antonym oracle
- phi.v6.idf_centroid_antonym_oracle — v4 lemmas + antonym oracle
Result
------
Both still at chance:
v5 cga AUC=0.503 frob AUC=0.503
v6 cga AUC=0.526 frob AUC=0.517
v6 shows a slight directional effect — contradicting mean (0.575)
slightly above compatible mean (0.559) — but the gap is dwarfed by
within-group std (~0.20).
Diagnostic (the deeper finding)
-------------------------------
Even with the antonym oracle, the lemma encoding cannot see
contradiction. The reason: for the rotor sandwich to place
phi(subject) NEAR phi(object) on compatible chains, the rotor must
encode the specific subject->object relationship — not just "a
rotation." Hash-derived rotors send phi(subject) to a random
point, so compatible chains have large Delta and contradicting
twins also have large Delta. We never recover the "compatible is
small" half of the separation.
Implication: the lemma encoding itself must carry relational
structure (positions in phi space such that a small canonical set
of rotations consistently take subjects to their related objects),
or the encoding must be jointly learned with the connective rotors
against a coherence loss. Either way, hash-derived phi cannot work
in principle — not just in this implementation.
This quantitatively validates ADR-0081's thesis that phi is the
critical-path research blocker. It is not a tuning problem.
Refactor:
- delta_cga / delta_frobenius now take both phi_l and phi_c so
new variants can vary the connective encoder independently.
- _PHI_VARIANTS is now (name, phi_l, phi_c) triples.
* research(evals): corpus-graph aware phi variants (v7, v8)
Adds two structural-only graph-aware phi candidates:
phi.v7.corpus_graph — corpus neighborhood centroid
phi.v8.corpus_graph_antonym_oracle — v7 lemmas + antonym oracle rotors
For each lemma, embed the centroid (in R^3) of hash points derived
from its graph neighborhood in the reviewed teaching corpus:
out_signature = "OUT:" + connective + "/" + object_lemma
in_signature = "IN:" + subject_lemma + "/" + connective
Lemmas with similar neighborhoods (same connectives used toward the
same kinds of partners) land near each other in R^3.
CAVEAT: structural only. This does NOT fit lemma positions to
satisfy R_c * phi(s) ~ phi(o) along the corpus relations. A joint
fit (TransE-style) would require a training loop, train/test split,
and convergence criteria — outside the single-file lab probe shape.
Result
------
v7 cga AUC=0.451 frob AUC=0.474
v8 cga AUC=0.444 frob AUC=0.458
Both lower than chance — contradicting twins land *closer* on average
than compatible ones, but within 1 std (~0.29), so it is noise, not
signal. The structural opposite of what would pass.
Closure on closed-form phi
--------------------------
The probe has now systematically falsified every closed-form phi
candidate available without training:
v1-v2: hash-derived domain encodings — chance
v3-v4: IDF-weighted domain encodings — chance
v5-v6: above + antonym oracle on connectives — chance
v7-v8: corpus-graph neighborhood encoding — chance (anti)
No reweighting of domains, no oracle on connectives, no graph-aware
neighborhood centroid is enough. This is consistent across 8
variants and 4 (lemma, connective) encoding combinations.
Remaining options
-----------------
1. Trained phi (TransE/RotatE-style): fit lemma + connective
embeddings jointly against a corpus coherence loss. Tiny
corpus (21 chains) means heavy overfitting risk; need
leave-one-out cross-validation to report honestly. Real
infrastructure, not a probe.
2. Larger labelled corpus: 21 chains is too few to discriminate
"encoding cannot work" from "encoding cannot work *on this
data*." Expanding the teaching corpus would let the probe
distinguish those.
3. Park geometric contemplation. The falsification stands; the
ADR-0080 contemplation loop remains the operational read-only
doctrine. Geometric stress mining waits until a forcing
function appears.
Recommendation: option 3. This probe has earned its keep — it
quantitatively validated ADR-0081's "phi is the load-bearing
research blocker" thesis across the full closed-form design space.
Shay