core/evals/results/phase2_pack_measurements.json
Shay 79f1678923 feat: ADR-0086 + ADR-0087 + 100-register catalog — cognition lane closure
Three load-bearing pieces:

1. ADR-0086 — UNKNOWN-intent pack-resident token surface
   New deterministic composer `pack_grounded_unknown_surface` in
   chat/pack_grounding.py.  When intent classification returns UNKNOWN
   but the prompt contains pack-resident lemmas (via cross-pack
   resolver), surface those lemmas with their semantic_domains
   instead of falling to the bare _UNKNOWN_DOMAIN_SURFACE.  Wired
   into chat/runtime.py::_maybe_pack_grounded_surface as the
   last typed-intent branch before the OOV fallback.  Null-lift
   invariant pinned: fully-OOV prompts still emit the universal
   disclosure byte-identically.  Closes four cognition-eval term
   misses: unknown_logos_019 (public), unknown_evidence_042 (dev),
   unknown_spirit_041 + unknown_word_018 (holdout).  Side effect:
   evals/results/phase2_pack_measurements.json refusal_rate drops
   from 0.25 → 0.125 across all three identity packs (no longer
   refusing on these prompts).

2. ADR-0087 — PROCEDURE selector + trailing-clause subject echo
   Two coupled changes in chat/pack_grounding.py:
   (a) Numeric-determiner downrank in _extract_procedure_topic_lemma:
       tokens whose primary semantic_domain starts with
       "quantitative.numeric." are demoted; non-numeric resident
       candidates always win.  So "compare two terms" anchors on
       `compare` not `two`.
   (b) Trailing clause echoes the full normalized subject_text
       rather than just the selected lemma, so OOV head nouns like
       "terms" reach the surface even when only the procedure verb
       is pack-resident.  Closes procedure_compare_011.

3. 100-register catalog
   New packs/register/_catalog.json — canonical machine-readable
   spec for all 100 registers (7 currently-ratified + 93 drafted)
   organized into 9 voice groups (depth/tone/stance/posture/domain/
   cultural/affective/functional/composite).  Each entry is a
   complete production input — realizer_overrides, marker palettes
   (openings/transitions/closings), depth_preference, description,
   author_notes.  All realizer_overrides use only legal keys per
   scripts/ratify_register_packs.py::_KNOWN_OVERRIDE_KEYS.
   Companion packs/register/CATALOG.md documents the production
   loop: materialize → widen REGISTER_IDS → ratify → smoke.

Cognition-eval lifts (all three splits):
  public:  term_capture 91.7% → 100.0%  (+8.3pp)
  holdout: term_capture 83.3% → 100.0%  (+16.7pp)
  dev:     term_capture 78.6% → 100.0%  (+21.4pp)
  surface_groundedness: 100% preserved on all splits
  intent_accuracy / versor_closure: 100% preserved on all splits

Tests:
  tests/test_pack_grounded_unknown.py     — 14 tests (composer
    direct + runtime engagement + null-lift invariant)
  tests/test_adr_0087_procedure_selector.py — 12 tests (selector
    numeric downrank + trailing-clause echo + regression guard)
  Existing test suites unaffected — cognition lane 120 passed / 1
  skipped both before and after.  Full lane net −3 failures vs
  pristine main (39 → 36 — none introduced).
2026-05-21 00:08:12 -07:00

91 lines
2 KiB
JSON

{
"all_claims_supported": true,
"claims_supported": {
"grounding_gate_pack_invariant": true,
"identity_load_bearing": true,
"no_fabrication_under_any_pack": true
},
"identity_divergence": {
"alignment_bands": [
0.2,
0.45,
0.6,
0.8,
0.95
],
"case_count": 10,
"load_bearing": true,
"packs": [
{
"bare_rate": 0.6,
"case_count": 10,
"hedge_rate": 0.4,
"pack_id": "default_general_v1",
"qualifier_rate": 0.0,
"surface_count": 50
},
{
"bare_rate": 0.2,
"case_count": 10,
"hedge_rate": 0.6,
"pack_id": "precision_first_v1",
"qualifier_rate": 0.2,
"surface_count": 50
},
{
"bare_rate": 0.8,
"case_count": 10,
"hedge_rate": 0.2,
"pack_id": "generosity_first_v1",
"qualifier_rate": 0.0,
"surface_count": 50
}
],
"pairwise_divergence": [
{
"distinct_rate": 0.8,
"pack_a": "default_general_v1",
"pack_b": "precision_first_v1"
},
{
"distinct_rate": 0.4,
"pack_a": "default_general_v1",
"pack_b": "generosity_first_v1"
},
{
"distinct_rate": 0.8,
"pack_a": "precision_first_v1",
"pack_b": "generosity_first_v1"
}
],
"schema_version": 1
},
"refusal_calibration": {
"case_count": 13,
"in_grounding_count": 5,
"out_of_grounding_count": 8,
"pack_invariant_gate": true,
"packs": [
{
"fabrication_rate": 0.0,
"out_of_grounding_count": 8,
"pack_id": "default_general_v1",
"refusal_rate": 0.125
},
{
"fabrication_rate": 0.0,
"out_of_grounding_count": 8,
"pack_id": "precision_first_v1",
"refusal_rate": 0.125
},
{
"fabrication_rate": 0.0,
"out_of_grounding_count": 8,
"pack_id": "generosity_first_v1",
"refusal_rate": 0.125
}
],
"schema_version": 1
},
"schema_version": 1
}