Commit graph

3 commits

Author SHA1 Message Date
Shay
e41a14f76c
chore(ratifier): calibrate default ratification threshold 0.0 → 0.5 (#86)
Closes audit Finding 3 (2026-05-20).

Pre-fix ``ratify_intent`` defaulted to ``threshold=0.0``, which admits
anything with non-negative ``cga_inner(prompt, anchor)`` — the field
gate (ADR-0022 §TBD-1) was structurally live but semantically
transparent.  RATIFIED was logged on essentially every turn because
the CGA inner product over conformal space is not sign-symmetric.

Measurement (``scripts/calibrate_ratification_threshold.py``):

  * Runs every cognition eval prompt (45 cases = 13 public + 13 dev +
    19 holdout) through a primed ``CognitiveTurnPipeline``.
  * Captures the actual ``cga_inner(prompt, anchor)`` score from the
    pipeline's own ``_ratify_intent`` via a temporary spy on the
    imported ``ratify_intent`` binding.

Observed distribution:

  * 34 RATIFIED:  min=+1.1039  p10=+1.1039  median=+2.6820  max=+5.7508
  * 11 PASSTHROUGH (no vocab-grounded anchor available; score=0.0)
  *  0 DEMOTED at any threshold ≤ 1.10

Threshold = 0.5 chosen as the calibrated default:

  * Well below the empirical floor of 1.10 — every currently-passing
    case stays RATIFIED, byte-identically.
  * Clearly non-trivially positive — random Cl(4,1) inner products
    fluctuate around zero, so 0.5 demands genuine correlation with
    the anchor rather than passive non-negativity.
  * Leaves headroom for the gate to actually demote weakly-aligned
    off-corpus / adversarial prompts to UNKNOWN and route them
    through the honest-refusal surface.

Verification:

  * ``core eval cognition`` — public 100/100/91.7/100, holdout
    100/100/83.3/100, dev 100/100/78.6/100 — byte-identical to
    MEMORY baselines.
  * ``core test --suite cognition`` — 120/0/1
  * ``core test --suite smoke`` — 67/0
  * ``core test --suite runtime`` — 19/0
  * 2 new tests in ``tests/test_ratification_threshold_default.py``
    pin both the constant and the signature default so a future
    change cannot silently regress to ``0.0``.
2026-05-20 19:59:25 -07:00
Shay
d7499c80b3
feat(intent): normalize confirmation-tag propositions (#45) 2026-05-19 22:55:28 -07:00
Shay
21c22b2201 feat(adr-0022): Forward Semantic Control — Accepted
Resolves all 5 TBDs and closes all 8 acceptance gates for ADR-0022.

TBD-1 (intent oracle): regex seed + field ratification —
generate/intent_ratifier.py. RATIFIED / DEMOTED / PASSTHROUGH
outcomes; DEMOTED routes through honest refusal.

TBD-2 (region intersection algebra): generate/admissibility.py.
Token-set composition via sorted set intersection; blade composition
via outer product with zero-blade as neutral element; rotor
composition via sandwich conjugation routed through
algebra.backend.versor_apply (Rust parity preserved by construction).
Empty intersections preserved — no silent relaxation.

Wiring: propose() and generate() accept an AdmissibilityRegion
(default None preserves legacy behavior); pipeline ratifies intent
at step 1b.i before graph construction.

Eval lane: evals/forward_semantic_control/ — both legs run against
CognitiveTurnPipeline (constrained) vs bare ChatRuntime.chat()
(unconstrained baseline). Dev (3 cases) and public/v1 (1 case) both
report overall_pass=true, causality_gap=1.0, coincidence_rate=0.0.
Chain-endpoint probe surfaces 'delta' only under forward semantic
control.

Bench cost (30 turns): -2.8% wall-clock (within +5% budget the ADR
set for the ratification gate on every turn). 138x cheaper than
Sonnet 4.5; main was 142x.

Tests: 33 new (25 admissibility + 8 ratifier). Full suite 912/913
pass — the single failure is pre-existing pack-size drift on main,
unrelated.
2026-05-17 12:10:20 -07:00