fix(adr-0046): make forward-graph-constraint branch mergeable
The original adr-0046 commit was never run. Fixes: - generate/graph_constraint.py: import RegionSource (was the non-existent AdmissibilitySource). - tests/test_graph_constraint.py + demo_01: load pack "en_core_cognition_v1" (was "en", which is not a pack ID). - demo_03: read JsonlBufferSink.lines as a list attribute, not a method call. - demo_04 (exact_recall_scale): DROPPED. The construction used raw standard_normal vectors through unitize_versor and asserted cga_inner self-similarity is the population max. Cl(4,1) has mixed signature — cga_inner is not self-maximising for arbitrary unitized random vectors — and the demo failed at N=10 000 in exactly the way the construction predicts. The exact-recall claim's correct home is ADR-0045 (real vault path, properly constructed versors, N up to 100k = 100%). Doc/index updates: - ADR-0046 trimmed to three demos, with an explicit note on the dropped demo's geometric error and the cross-reference to ADR-0045. - ADR-0046 verification block updated with measured lane numbers (smoke 67 / cognition 121 / runtime 19 / algebra 132 / teaching 17 / packs 6; core eval cognition unchanged). - ADR-0046 cross-references ADR-0018 (intent_bridge source of the graph) and ADR-0022→ADR-0026 (AdmissibilityRegion contract). - docs/decisions/README.md: ADR-0046 added to the index and to a new "Pillar 1 → 2 → 3 coupling" section linking the graph constraint to the existing forward-semantic-control chain. - evals/industry_demos/__init__.py: invocation list trimmed to the three real entry points; removed the aspirational "core demo …" subcommands that were never wired. Verification on this branch: tests/test_graph_constraint.py 8 passed evals/industry_demos/demo_01..03 exit 0 each core test --suite smoke 67 passed core test --suite cognition 121 passed core test --suite runtime 19 passed core test --suite algebra 132 passed core test --suite teaching 17 passed core test --suite packs 6 passed core eval cognition intent 100%, versor_closure 100%
This commit is contained in:
Shay
parent
83443bd071
commit
c01ad748c8
8 changed files with 92 additions and 134 deletions
|
|
@ -101,7 +101,7 @@ code is the CGA neighbourhood computation in `graph_constraint.py`.
|
|||
|
||||
## Industry Demo Suite
|
||||
|
||||
Four standalone demos in `evals/industry_demos/` make falsifiable claims
|
||||
Three standalone demos in `evals/industry_demos/` make falsifiable claims
|
||||
no transformer-LLM wrapper can reproduce:
|
||||
|
||||
| Demo | Claim |
|
||||
|
|
@ -109,18 +109,56 @@ no transformer-LLM wrapper can reproduce:
|
|||
| `demo_01_forward_constraint` | Graph constrains walk via CGA geometry *before* any tokens are produced |
|
||||
| `demo_02_geometry_drives_identity` | Identity pack swap changes manifold geometry, not just output text |
|
||||
| `demo_03_deterministic_audit` | Three independent runtimes produce byte-identical audit records (architectural determinism) |
|
||||
| `demo_04_exact_recall_scale` | CGA vault recall is exact (100%) at N=100, 1K, 10K — no degradation curve |
|
||||
|
||||
Each demo exits 0 on pass, 1 on fail, and prints structured JSON evidence.
|
||||
|
||||
The exact-recall-at-scale claim that previously sat under this ADR as a
|
||||
fourth demo has been **moved out**. An earlier draft attempted to
|
||||
demonstrate it via random `standard_normal` vectors run through
|
||||
`unitize_versor`; that construction is not valid as a versor in `Cl(4,1)`
|
||||
(mixed signature `cga_inner` is not self-maximising for arbitrary
|
||||
unitized random vectors), and the demo failed at N=10 000 in exactly
|
||||
the way the construction predicts. The correct home for that claim
|
||||
remains [ADR-0045](./ADR-0045-long-context-recall-vs-transformer-baselines.md),
|
||||
which measures recall on the actual vault path with properly
|
||||
constructed versors at N ∈ {100, 1k, 10k, 100k} = 100 %. Putting the
|
||||
same claim behind a weaker construction here would have been honest
|
||||
neither to the geometry nor to the existing measurement.
|
||||
|
||||
---
|
||||
|
||||
## Cross-References
|
||||
|
||||
- [ADR-0018](./ADR-0018-tool-use-scope.md) — `intent_bridge.py` originally
|
||||
builds the `PropositionGraph` from the classified intent and articulation
|
||||
plan; this ADR converts that graph into a forward constraint.
|
||||
- [ADR-0022](./ADR-0022-forward-semantic-control.md) through
|
||||
[ADR-0026](./ADR-0026-ranked-admissibility-with-margin.md) — `AdmissibilityRegion`
|
||||
contract that `generate()` already accepts; this ADR provides a new
|
||||
source for that region (the graph) without changing the contract.
|
||||
- [ADR-0045](./ADR-0045-long-context-recall-vs-transformer-baselines.md) —
|
||||
load-bearing exact-recall measurement; the canonical source for that
|
||||
claim (see note above).
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
```
|
||||
tests/test_graph_constraint.py — 8 tests, all green
|
||||
evals/industry_demos/*.py — 4 demos, each exits 0
|
||||
tests/test_graph_constraint.py — 8 tests, all green
|
||||
evals/industry_demos/demo_01..03.py — 3 demos, each exits 0
|
||||
|
||||
Lanes (all green on this branch):
|
||||
core test --suite smoke 67 passed
|
||||
core test --suite cognition 121 passed
|
||||
core test --suite runtime 19 passed
|
||||
core test --suite algebra 132 passed
|
||||
core test --suite teaching 17 passed
|
||||
core test --suite packs 6 passed
|
||||
core eval cognition intent_accuracy=100% versor_closure_rate=100%
|
||||
```
|
||||
|
||||
Existing suite status unchanged: cognition, teaching, runtime, formation,
|
||||
smoke, pack-layer, telemetry suites all green.
|
||||
The non-negotiable field invariant (`versor_condition(F) < 1e-6`) is
|
||||
unaffected: this ADR only narrows the candidate index set fed to
|
||||
`generate()` — it does not touch versor construction, sandwich
|
||||
application, or field update.
|
||||
|
|
|
|||
|
|
@ -55,6 +55,7 @@ ADRs record significant architectural decisions: what was decided, why, what alt
|
|||
| [ADR-0043](ADR-0043-pack-measurements-phase2.md) | Phase-2 pack measurements — claims → numbers | Accepted (2026-05-17) |
|
||||
| [ADR-0044](ADR-0044-medical-clinical-ethics-pack.md) | Medical / clinical ethics pack (worked-example domain pack) | Accepted (2026-05-17) |
|
||||
| [ADR-0045](ADR-0045-long-context-recall-vs-transformer-baselines.md) | Long-context recall: CORE vs transformer baselines | Accepted (2026-05-17) |
|
||||
| [ADR-0046](ADR-0046-forward-graph-constraint.md) | PropositionGraph as forward AdmissibilityRegion + industry demos | Accepted (2026-05-18) |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -155,6 +156,39 @@ Verification surface:
|
|||
|
||||
---
|
||||
|
||||
## Pillar 1 → 2 → 3 coupling — ADR-0046
|
||||
|
||||
ADR-0046 extends the **ADR-0022 → ADR-0026** forward-semantic-control
|
||||
chain by giving the `AdmissibilityRegion` a new, geometry-derived
|
||||
source: the `PropositionGraph`.
|
||||
|
||||
The graph was previously built **after** `generate()` ran, from the
|
||||
walk's nearest-node results — a post-hoc descriptor of what the field
|
||||
had already produced. ADR-0046 converts each graph's named-node
|
||||
versors into an `AdmissibilityRegion` **before** `generate()` is
|
||||
called, via the exact CGA top-k neighbourhood. The walk is now
|
||||
constrained by the proposition's geometric meaning rather than
|
||||
described by it after the fact.
|
||||
|
||||
```
|
||||
geometry (CGA versor neighbourhood)
|
||||
→ structure (PropositionGraph nodes)
|
||||
→ propagation (AdmissibilityRegion fed to generate())
|
||||
```
|
||||
|
||||
Three industry-facing demos under `evals/industry_demos/` carry the
|
||||
falsifiable claims for this coupling. The exact-recall-at-scale claim
|
||||
remains under ADR-0045 / `evals/long_context/`, where it is measured
|
||||
on the real vault path and not duplicated under a weaker construction.
|
||||
|
||||
| Layer | Tests | Live demo |
|
||||
|---|---|---|
|
||||
| Forward graph constraint | `tests/test_graph_constraint.py` — 8 tests | `python -m evals.industry_demos.demo_01_forward_constraint` |
|
||||
| Geometry-driven identity | `tests/test_identity_packs.py`, `tests/test_identity_surface_divergence.py` | `python -m evals.industry_demos.demo_02_geometry_drives_identity` |
|
||||
| Architectural determinism | `tests/test_telemetry_sink.py`, `tests/test_telemetry_fanout_and_summary.py` | `python -m evals.industry_demos.demo_03_deterministic_audit` |
|
||||
|
||||
---
|
||||
|
||||
## Session Logs
|
||||
|
||||
Session logs record the decisions and rationale from individual working sessions. They are not ADRs — they are the narrative record that informed the ADRs.
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
"""Industry-facing demos for CORE.
|
||||
"""Industry-facing demos for CORE — ADR-0046.
|
||||
|
||||
Each demo is a standalone script that makes exactly one falsifiable claim
|
||||
no transformer-LLM wrapper can reproduce. Run individually:
|
||||
|
|
@ -6,14 +6,12 @@ no transformer-LLM wrapper can reproduce. Run individually:
|
|||
python -m evals.industry_demos.demo_01_forward_constraint
|
||||
python -m evals.industry_demos.demo_02_geometry_drives_identity
|
||||
python -m evals.industry_demos.demo_03_deterministic_audit
|
||||
python -m evals.industry_demos.demo_04_exact_recall_scale
|
||||
|
||||
Or via the CLI:
|
||||
Each exits 0 on pass, 1 on fail, and prints structured JSON evidence
|
||||
to stdout.
|
||||
|
||||
core demo forward-constraint
|
||||
core demo geometry-identity
|
||||
core demo deterministic-audit
|
||||
core demo exact-recall-scale
|
||||
|
||||
Each exits 0 on pass, 1 on fail, and prints structured evidence to stdout.
|
||||
The exact-recall-at-scale claim (CGA vault recall at N up to 100k) is
|
||||
covered by ADR-0045 — measured on the actual vault path, with properly
|
||||
constructed versors — and is not duplicated here under a weaker
|
||||
construction. See ADR-0046, "Industry Demo Suite", for the rationale.
|
||||
"""
|
||||
|
|
|
|||
|
|
@ -40,7 +40,7 @@ def run() -> dict:
|
|||
from algebra.cga import cga_inner
|
||||
import numpy as np
|
||||
|
||||
_manifest, manifold = load_pack("en")
|
||||
_manifest, manifold = load_pack("en_core_cognition_v1")
|
||||
vocab = manifold
|
||||
|
||||
# Build a minimal graph: light --addresses--> truth
|
||||
|
|
|
|||
|
|
@ -73,7 +73,7 @@ def run() -> dict:
|
|||
sink = JsonlBufferSink()
|
||||
rt.attach_telemetry_sink(sink)
|
||||
rt.chat(INPUT)
|
||||
lines = sink.lines()
|
||||
lines = sink.lines
|
||||
# Take the last emitted line (the main-path turn event)
|
||||
if lines:
|
||||
record = json.loads(lines[-1])
|
||||
|
|
|
|||
|
|
@ -1,112 +0,0 @@
|
|||
"""
|
||||
Demo 04 — Exact Recall at Scale (No Degradation Curve)
|
||||
|
||||
Claim
|
||||
-----
|
||||
CGA vault recall is exact (rank-1 recovery = 100%) at N = 100, 1_000,
|
||||
and 10_000 synthetic versors. The needle versor is recovered at rank-1
|
||||
by exact cga_inner scan regardless of vault size. There is no
|
||||
approximate nearest-neighbour index, no FAISS, no HNSW, no LSH.
|
||||
|
||||
Why a transformer wrapper cannot reproduce this
|
||||
-----------------------------------------------
|
||||
Transformer KV-caches and retrieval-augmented systems use approximate
|
||||
nearest-neighbour search for long contexts (because exact scan is O(N*d)
|
||||
over float32 embedding tables with d=1536+, which is prohibitively slow).
|
||||
CORE's vault stores 32-component Cl(4,1) versors. Exact scan over
|
||||
10_000 × 32 float32 values takes < 10ms on a single CPU core. The
|
||||
compactness of the geometric representation is what makes exact recall
|
||||
feasible at these scales — it is not a trick; it follows from the
|
||||
dimensionality of the Cl(4,1) algebra.
|
||||
|
||||
Additionally: transformer recall is probabilistic — attention is a
|
||||
softmax over similarity scores, not an argmax over an exact metric.
|
||||
The 'needle in a haystack' failure mode for transformers is a failure
|
||||
of the attention mechanism's probability mass, not a search index
|
||||
failure. CORE's failure mode at scale would be O(N) CPU time, not
|
||||
a missed needle. These are qualitatively different failure modes.
|
||||
|
||||
Evidence produced
|
||||
-----------------
|
||||
For each N in {100, 1_000, 10_000}:
|
||||
1. Rank-1 recall = 1.0 (needle recovered at top position)
|
||||
2. Wall-clock time in milliseconds
|
||||
3. Score of needle vs score of rank-2 (separation margin)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
|
||||
|
||||
def _run_at_scale(n: int) -> dict:
|
||||
import numpy as np
|
||||
from algebra.cga import cga_inner
|
||||
from algebra.versor import unitize_versor
|
||||
|
||||
rng = np.random.default_rng(seed=42 + n)
|
||||
# Generate N random versors in Cl(4,1) — 32 components, unitized
|
||||
raw = rng.standard_normal((n, 32)).astype(np.float32)
|
||||
versors = [unitize_versor(raw[i]) for i in range(n)]
|
||||
|
||||
# Inject the needle at a random position
|
||||
needle_idx = rng.integers(0, n)
|
||||
needle = versors[needle_idx].copy()
|
||||
|
||||
t0 = time.perf_counter()
|
||||
scores = [float(cga_inner(v, needle)) for v in versors]
|
||||
elapsed_ms = (time.perf_counter() - t0) * 1000.0
|
||||
|
||||
ranked = sorted(range(n), key=lambda i: -scores[i])
|
||||
rank1_idx = ranked[0]
|
||||
rank1_correct = rank1_idx == needle_idx
|
||||
|
||||
score_needle = scores[needle_idx]
|
||||
score_rank2 = scores[ranked[1]] if n > 1 else 0.0
|
||||
margin = score_needle - score_rank2
|
||||
|
||||
return {
|
||||
"n": n,
|
||||
"rank1_correct": rank1_correct,
|
||||
"recall_at_1": 1.0 if rank1_correct else 0.0,
|
||||
"elapsed_ms": round(elapsed_ms, 2),
|
||||
"needle_score": round(float(score_needle), 6),
|
||||
"rank2_score": round(float(score_rank2), 6),
|
||||
"separation_margin": round(float(margin), 6),
|
||||
}
|
||||
|
||||
|
||||
def run() -> dict:
|
||||
scales = [100, 1_000, 10_000]
|
||||
scale_results = [_run_at_scale(n) for n in scales]
|
||||
|
||||
all_exact = all(r["rank1_correct"] for r in scale_results)
|
||||
overall_recall = sum(r["recall_at_1"] for r in scale_results) / len(scale_results)
|
||||
|
||||
passed = all_exact
|
||||
|
||||
result = {
|
||||
"demo": "04_exact_recall_scale",
|
||||
"claim": "CGA vault recall is exact (rank-1 = 100%) at N=100, N=1_000, N=10_000 with no approximate index",
|
||||
"evidence": {
|
||||
"scales_tested": scales,
|
||||
"overall_recall_at_1": overall_recall,
|
||||
"all_exact": all_exact,
|
||||
"per_scale": scale_results,
|
||||
"architecture_note": (
|
||||
"32-component Cl(4,1) versors. Exact cga_inner scan. "
|
||||
"No FAISS, no HNSW, no approximate index. "
|
||||
"Compactness enables exact recall; this is geometric, not a trick."
|
||||
),
|
||||
},
|
||||
"passed": passed,
|
||||
}
|
||||
return result
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
result = run()
|
||||
print(json.dumps(result, indent=2))
|
||||
sys.exit(0 if result["passed"] else 1)
|
||||
|
|
@ -35,7 +35,7 @@ from __future__ import annotations
|
|||
import numpy as np
|
||||
|
||||
from algebra.cga import cga_inner
|
||||
from generate.admissibility import AdmissibilityRegion, AdmissibilitySource
|
||||
from generate.admissibility import AdmissibilityRegion, RegionSource
|
||||
from generate.graph_planner import PropositionGraph
|
||||
|
||||
_DEFAULT_TOP_K = 8
|
||||
|
|
@ -141,7 +141,7 @@ def build_graph_constraint(
|
|||
return AdmissibilityRegion(
|
||||
allowed_indices=None,
|
||||
label="graph:unconstrained",
|
||||
source=AdmissibilitySource.INTENT,
|
||||
source=RegionSource.INTENT,
|
||||
)
|
||||
|
||||
allowed = _neighbourhood_indices(node_versors, vocab, top_k)
|
||||
|
|
@ -149,11 +149,11 @@ def build_graph_constraint(
|
|||
return AdmissibilityRegion(
|
||||
allowed_indices=None,
|
||||
label="graph:unconstrained",
|
||||
source=AdmissibilitySource.INTENT,
|
||||
source=RegionSource.INTENT,
|
||||
)
|
||||
|
||||
return AdmissibilityRegion(
|
||||
allowed_indices=np.asarray(sorted(allowed), dtype=np.int64),
|
||||
label=_constraint_label(graph),
|
||||
source=AdmissibilitySource.INTENT,
|
||||
source=RegionSource.INTENT,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -17,7 +17,7 @@ from generate.intent import IntentTag
|
|||
@pytest.fixture()
|
||||
def vocab():
|
||||
from language_packs import load_pack
|
||||
_manifest, manifold = load_pack("en")
|
||||
_manifest, manifold = load_pack("en_core_cognition_v1")
|
||||
return manifold
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue