docs: ADR-0123a — document all_three_pass_rate synonym in inference_shape
This commit is contained in:
parent
1e52d71b51
commit
a8d019d8a0
3 changed files with 124 additions and 0 deletions
65
docs/decisions/ADR-0123a-inference-shape-synonym.md
Normal file
65
docs/decisions/ADR-0123a-inference-shape-synonym.md
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
# ADR-0123a — `all_three_pass_rate` Synonym in `inference_shape` (ADR-0109 Amendment)
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-05-22
|
||||
**Author:** CORE agents + reviewers
|
||||
**Amends:** ADR-0109
|
||||
**Depends on:** ADR-0109, ADR-0123, ADR-0124
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
Under [ADR-0109](ADR-0109-lane-shape-aware-thresholds.md), threshold verification rules are lane-shape-aware. [ADR-0123](ADR-0123-symbolic-logic-shape-remap.md) re-mapped the `symbolic_logic` lane to `inference_shape` because it emits inference-style metrics.
|
||||
|
||||
However, the `inference_shape` checker (`_check_inference_shape`) was written expecting the exact metric key `all_pass_rate`. The `symbolic_logic` lane actually emits `all_three_pass_rate` instead of `all_pass_rate`. This metric mismatch caused evaluation to fail closed.
|
||||
|
||||
A widening bridge was implemented in ADR-0124 to accept `all_three_pass_rate` as a synonym for `all_pass_rate`. This amendment formally documents this contract widening and codifies the synonym precedence rules.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
Document and verify the synonym behavior within the `inference_shape` checker in `core/capability/expert_demo.py`.
|
||||
|
||||
The `inference_shape` checker accepts either `all_pass_rate` or `all_three_pass_rate` as the load-bearing pass-rate metric.
|
||||
|
||||
### Precedence Rules
|
||||
If both metric keys are present in the results payload:
|
||||
1. The primary key `all_pass_rate` takes precedence.
|
||||
2. The check evaluates the value of `all_pass_rate`, ignoring the value of `all_three_pass_rate`.
|
||||
|
||||
The threshold (`ALL_PASS_RATE_MIN = 0.95`) applies to whichever key is active.
|
||||
|
||||
---
|
||||
|
||||
## Invariants
|
||||
|
||||
### no_unregistered_synonyms
|
||||
Only `all_pass_rate` and `all_three_pass_rate` are recognized as pass-rate keys under `inference_shape`. Any third synonym key (e.g. `foo_bar_rate`) is refused, causing the gate to fail closed.
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Evidence
|
||||
|
||||
Accepted when:
|
||||
- A unit test class `TestInferenceShapeAcceptsSynonyms` in `tests/test_lane_shape_thresholds.py` asserts that:
|
||||
- `_check_inference_shape` successfully accepts `all_three_pass_rate` alone.
|
||||
- `_check_inference_shape` successfully accepts `all_pass_rate` alone.
|
||||
- Any third synonym is rejected.
|
||||
- When both are present, `all_pass_rate` has precedence (e.g., if `all_pass_rate` is below threshold and `all_three_pass_rate` is 1.0, the check fails).
|
||||
|
||||
---
|
||||
|
||||
## Consequences
|
||||
|
||||
- The `inference_shape` is now a 2-key synonym shape, making future lanes that produce either key eligible without further changes.
|
||||
- The gate remains secure against unregistered synonyms.
|
||||
|
||||
---
|
||||
|
||||
## Out of scope
|
||||
|
||||
- Introducing any new shapes or new checkers.
|
||||
- Changing threshold values.
|
||||
- Modifying threshold rules of other shapes.
|
||||
|
|
@ -43,6 +43,7 @@ ADRs record significant architectural decisions: what was decided, why, what alt
|
|||
| [ADR-0122](ADR-0122-systems-software-audit-passed-deferred.md) | `systems_software` Audit-Passed Promotion: Deferred | Accepted (2026-05-22) |
|
||||
| [ADR-0123](ADR-0123-symbolic-logic-shape-remap.md) | `symbolic_logic` Lane-Shape Remap (ADR-0109 Amendment) | Accepted (2026-05-22) |
|
||||
| [ADR-0124](ADR-0124-systems-software-audit-passed-promotion.md) | `systems_software` Audit-Passed Promotion | Accepted (2026-05-22) |
|
||||
| [ADR-0123a](ADR-0123a-inference-shape-synonym.md) | `all_three_pass_rate` Synonym in `inference_shape` (ADR-0109 Amendment) | Accepted (2026-05-22) |
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -81,6 +82,7 @@ The ADR-0091..0114 slate is fully accepted (0091..0113) plus one proposed-roadma
|
|||
- OOD Surface Generator (150 deterministic variants; discharges ADR-0114a obligation #2 for the GSM8K-style parser dev lane) — ADR-0118a
|
||||
- `symbolic_logic` Lane-Shape Remap (ADR-0109 amendment) — ADR-0123
|
||||
- `systems_software` Audit-Passed Promotion (third successful) — ADR-0124
|
||||
- `all_three_pass_rate` Synonym in `inference_shape` (ADR-0109 Amendment) — ADR-0123a
|
||||
|
||||
ADR-0080 has also landed: Contemplation Loop Phase 1 adds a read-only frontier-compare miner that emits `SPECULATIVE` findings only.
|
||||
|
||||
|
|
|
|||
|
|
@ -221,3 +221,60 @@ class TestSymbolicLogicShapeGate:
|
|||
assert verdict.passed is True
|
||||
assert verdict.reason == "all audit-passed predicates satisfied"
|
||||
|
||||
|
||||
class TestInferenceShapeAcceptsSynonyms:
|
||||
def test_accepts_all_three_pass_rate_alone(self) -> None:
|
||||
from core.capability.expert_demo import _check_inference_shape
|
||||
ok, reason = _check_inference_shape(
|
||||
"symbolic_logic",
|
||||
{
|
||||
"all_three_pass_rate": 1.0,
|
||||
"replay_determinism": 1.0,
|
||||
"overall_pass": True,
|
||||
},
|
||||
)
|
||||
assert ok is True
|
||||
assert reason == ""
|
||||
|
||||
def test_accepts_all_pass_rate_alone(self) -> None:
|
||||
from core.capability.expert_demo import _check_inference_shape
|
||||
ok, reason = _check_inference_shape(
|
||||
"inference_closure",
|
||||
{
|
||||
"all_pass_rate": 1.0,
|
||||
"replay_determinism": 1.0,
|
||||
"overall_pass": True,
|
||||
},
|
||||
)
|
||||
assert ok is True
|
||||
assert reason == ""
|
||||
|
||||
def test_rejects_third_synonym(self) -> None:
|
||||
from core.capability.expert_demo import _check_inference_shape
|
||||
ok, reason = _check_inference_shape(
|
||||
"some_lane",
|
||||
{
|
||||
"foo_bar_rate": 1.0,
|
||||
"replay_determinism": 1.0,
|
||||
"overall_pass": True,
|
||||
},
|
||||
)
|
||||
assert ok is False
|
||||
assert "missing required metric 'all_pass_rate'" in reason
|
||||
|
||||
def test_precedence_primary_key_wins(self) -> None:
|
||||
from core.capability.expert_demo import _check_inference_shape
|
||||
# Both keys are present: all_pass_rate is below threshold,
|
||||
# all_three_pass_rate is at threshold. The primary must win (fail).
|
||||
ok, reason = _check_inference_shape(
|
||||
"symbolic_logic",
|
||||
{
|
||||
"all_pass_rate": 0.90,
|
||||
"all_three_pass_rate": 1.0,
|
||||
"replay_determinism": 1.0,
|
||||
"overall_pass": True,
|
||||
},
|
||||
)
|
||||
assert ok is False
|
||||
assert "all_pass_rate=0.9 below threshold" in reason
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue