Commit graph

1 commit

Author SHA1 Message Date
Shay
e41a14f76c
chore(ratifier): calibrate default ratification threshold 0.0 → 0.5 (#86)
Closes audit Finding 3 (2026-05-20).

Pre-fix ``ratify_intent`` defaulted to ``threshold=0.0``, which admits
anything with non-negative ``cga_inner(prompt, anchor)`` — the field
gate (ADR-0022 §TBD-1) was structurally live but semantically
transparent.  RATIFIED was logged on essentially every turn because
the CGA inner product over conformal space is not sign-symmetric.

Measurement (``scripts/calibrate_ratification_threshold.py``):

  * Runs every cognition eval prompt (45 cases = 13 public + 13 dev +
    19 holdout) through a primed ``CognitiveTurnPipeline``.
  * Captures the actual ``cga_inner(prompt, anchor)`` score from the
    pipeline's own ``_ratify_intent`` via a temporary spy on the
    imported ``ratify_intent`` binding.

Observed distribution:

  * 34 RATIFIED:  min=+1.1039  p10=+1.1039  median=+2.6820  max=+5.7508
  * 11 PASSTHROUGH (no vocab-grounded anchor available; score=0.0)
  *  0 DEMOTED at any threshold ≤ 1.10

Threshold = 0.5 chosen as the calibrated default:

  * Well below the empirical floor of 1.10 — every currently-passing
    case stays RATIFIED, byte-identically.
  * Clearly non-trivially positive — random Cl(4,1) inner products
    fluctuate around zero, so 0.5 demands genuine correlation with
    the anchor rather than passive non-negativity.
  * Leaves headroom for the gate to actually demote weakly-aligned
    off-corpus / adversarial prompts to UNKNOWN and route them
    through the honest-refusal surface.

Verification:

  * ``core eval cognition`` — public 100/100/91.7/100, holdout
    100/100/83.3/100, dev 100/100/78.6/100 — byte-identical to
    MEMORY baselines.
  * ``core test --suite cognition`` — 120/0/1
  * ``core test --suite smoke`` — 67/0
  * ``core test --suite runtime`` — 19/0
  * 2 new tests in ``tests/test_ratification_threshold_default.py``
    pin both the constant and the signature default so a future
    change cannot silently regress to ``0.0``.
2026-05-20 19:59:25 -07:00