4ba1ef2da3
Converts the load-bearing claims of the ADR-0027→0042 pack-layer chain into CI-enforced numbers across the three ratified identity packs (default_general_v1, precision_first_v1, generosity_first_v1). Two new pack-driven runners + an orchestrator: - evals/identity_divergence/pack_runner.py — drives real SentenceAssembler + SurfaceContext (no mocks) across all three packs over 10 cases × 5 alignment bands; publishes per-pack bare/hedge/qualifier rates and pairwise distinct_rate. - evals/refusal_calibration/pack_runner.py — runs the existing grounding-refusal lane via RuntimeConfig(identity_pack=...); publishes per-pack refusal_rate/fabrication_rate and a pack_invariant_gate flag asserting byte-identical cold-start surfaces across packs. - scripts/publish_pack_measurements.py — combined publisher emitting evals/results/phase2_pack_measurements.json. Baseline numbers (2026-05-17): - precision_first hedge_rate=0.60, qualifier_rate=0.20 - generosity_first hedge_rate=0.20, qualifier_rate=0.00 - default_general hedge_rate=0.40, qualifier_rate=0.00 - pairwise distinct_rate ∈ [0.40, 0.80] - refusal_rate=1.00, fabrication_rate=0.00 for all three packs - pack_invariant_gate=True 6 tests in tests/test_pack_measurements_phase2.py lock the schema + load-bearing flags + the structural inequality precision.hedge_rate > generosity.hedge_rate. If identity packs get wired into the cognition gate, pack_invariant_gate flips and the suite fails. ADR-0043 documents the numbers, the extended marker rationale, and the trade-offs. README index updated with ADR-0043 row and chain title bumped to "ADR-0027 through ADR-0043". Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
90 lines
2 KiB
JSON
90 lines
2 KiB
JSON
{
|
|
"claims_supported": {
|
|
"grounding_gate_pack_invariant": true,
|
|
"identity_load_bearing": true,
|
|
"no_fabrication_under_any_pack": true
|
|
},
|
|
"identity_divergence": {
|
|
"alignment_bands": [
|
|
0.2,
|
|
0.45,
|
|
0.6,
|
|
0.8,
|
|
0.95
|
|
],
|
|
"case_count": 10,
|
|
"load_bearing": true,
|
|
"packs": [
|
|
{
|
|
"bare_rate": 0.6,
|
|
"case_count": 10,
|
|
"hedge_rate": 0.4,
|
|
"pack_id": "default_general_v1",
|
|
"qualifier_rate": 0.0,
|
|
"surface_count": 50
|
|
},
|
|
{
|
|
"bare_rate": 0.2,
|
|
"case_count": 10,
|
|
"hedge_rate": 0.6,
|
|
"pack_id": "precision_first_v1",
|
|
"qualifier_rate": 0.2,
|
|
"surface_count": 50
|
|
},
|
|
{
|
|
"bare_rate": 0.8,
|
|
"case_count": 10,
|
|
"hedge_rate": 0.2,
|
|
"pack_id": "generosity_first_v1",
|
|
"qualifier_rate": 0.0,
|
|
"surface_count": 50
|
|
}
|
|
],
|
|
"pairwise_divergence": [
|
|
{
|
|
"distinct_rate": 0.8,
|
|
"pack_a": "default_general_v1",
|
|
"pack_b": "precision_first_v1"
|
|
},
|
|
{
|
|
"distinct_rate": 0.4,
|
|
"pack_a": "default_general_v1",
|
|
"pack_b": "generosity_first_v1"
|
|
},
|
|
{
|
|
"distinct_rate": 0.8,
|
|
"pack_a": "precision_first_v1",
|
|
"pack_b": "generosity_first_v1"
|
|
}
|
|
],
|
|
"schema_version": 1
|
|
},
|
|
"refusal_calibration": {
|
|
"case_count": 13,
|
|
"in_grounding_count": 5,
|
|
"out_of_grounding_count": 8,
|
|
"pack_invariant_gate": true,
|
|
"packs": [
|
|
{
|
|
"fabrication_rate": 0.0,
|
|
"out_of_grounding_count": 8,
|
|
"pack_id": "default_general_v1",
|
|
"refusal_rate": 1.0
|
|
},
|
|
{
|
|
"fabrication_rate": 0.0,
|
|
"out_of_grounding_count": 8,
|
|
"pack_id": "precision_first_v1",
|
|
"refusal_rate": 1.0
|
|
},
|
|
{
|
|
"fabrication_rate": 0.0,
|
|
"out_of_grounding_count": 8,
|
|
"pack_id": "generosity_first_v1",
|
|
"refusal_rate": 1.0
|
|
}
|
|
],
|
|
"schema_version": 1
|
|
},
|
|
"schema_version": 1
|
|
}
|