Bundles the three pieces needed to consummate the promotion after the reviewer signature lands: 1. Wire the expert tier in the capability ledger 2. Path-stability fix (digest filesystem-independence) 3. Reviewer-registry allow-list extension (regression fix for #194) Result: mathematics_logic is now the first expert-tier domain in the capability ledger. $ ledger_report() -> mathematics_logic row: status: "expert" predicates: { seeded, grounded, reasoning_capable, audit_passed, expert: True } expert_reason: "ADR-0120-math composer admitted" 1. Ledger wiring (core/capability/reporting.py): - _EXPERT_DOMAIN_STATUSES extends to 6 tiers with "expert" after "audit-passed" (strict super-tier). - New _EXPERT_COMPOSERS dict — per-domain registry of composer module names. Currently only mathematics_logic -> core.capability.expert_promotion_math. - New `expert` predicate computation gated on audit_passed; calls registered composer's evaluate_math_expert_promotion() and reads promote_admitted as the verdict. Fail-closed on exception or missing composer. - status = "expert" when predicate True. - predicates dict gains "expert" key; row gains expert_reason. 2. Path-stability fix (composite_math_gate.py + expert_promotion_math.py): - New _rel(path) helpers return repo-root-relative POSIX strings instead of str(absolute_path). - claim_digest now commits to relative paths, so operator A on ~/work/core and operator B on /srv/checkouts/core compute the SAME digest for identical evidence. - Without this fix no signature would ever match across filesystems — a real bug that would have blocked every signing attempt. 3. Allow-list regression fix (core/capability/reviewers.py): - ALLOWED_TOP_LEVEL_KEYS extended with "math_expert_claims". - PR #194 added the section to docs/reviewers.yaml but didn't extend the allow-list, silently breaking the audit_passed predicate for ALL 3 prior domains (loader rejected the file). This PR's test_allowed_top_level_keys_includes_math_expert_claims regression-pins the fix. Reviewer signature (operator-only action by shay-j) carried in docs/reviewers.yaml: math_expert_claims: - domain_id: mathematics_logic signed_by: shay-j claim_digest: "94149794e8c19896851e062cf1f921cfa9ba04770b674bc3b4c33023f7c7331b" The auto-mode safeguard correctly blocked the agent from self- signing during PR construction; the signature was performed by the reviewer directly and brought into this PR. Future signatures stay human-only. Tests: 12/12 new ledger-flip tests + 174/174 across full obligation auditor / composer / composite-gate / expert-demo / reviewer-registry regression. Updated #194's awaiting-state snapshot to reflect the new promote_admitted=True state on main. GSM8K (honest disclosure, not gating): still 0/50 admission, wrong=0, safety_rail_intact=True, substrate=candidate_graph. Probe lift is future work (bounded pronoun coref is the highest-leverage item — ~28% of refusals route through it). The promotion does not depend on GSM8K per ADR-0131.
330 lines
12 KiB
Python
330 lines
12 KiB
Python
"""Reviewer Registry v1 loader and validator (ADR-0092).
|
|
|
|
Parses ``docs/reviewers.yaml`` into a typed, immutable :class:`ReviewerRegistry`.
|
|
Schema rejection is loud: unknown keys, malformed entries, duplicate ids, and
|
|
out-of-scope wildcards all raise :class:`ReviewerRegistryError`.
|
|
|
|
The registry is consulted by the Domain Pack Contract v1 validator
|
|
(ADR-0093) to enforce ADR-0091 predicate #8 (reviewer resolution).
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from dataclasses import dataclass
|
|
from pathlib import Path
|
|
from typing import Any, Mapping
|
|
|
|
import yaml
|
|
|
|
|
|
ALLOWED_TOP_LEVEL_KEYS: frozenset[str] = frozenset(
|
|
{"schema_version", "reviewers", "audit_passed_claims", "math_expert_claims"}
|
|
)
|
|
ALLOWED_REVIEWER_KEYS: frozenset[str] = frozenset(
|
|
{"reviewer_id", "display_name", "role", "domains", "review_scope", "provenance"}
|
|
)
|
|
ALLOWED_EXPERT_DEMO_CLAIM_KEYS: frozenset[str] = frozenset(
|
|
{
|
|
"domain_id",
|
|
"evidence_lanes",
|
|
"evidence_revision",
|
|
"signed_by",
|
|
"claim_digest",
|
|
}
|
|
)
|
|
ALLOWED_ROLES: frozenset[str] = frozenset({"primary", "domain"})
|
|
ALLOWED_SCOPES: frozenset[str] = frozenset({"pack", "proposal", "chain", "eval"})
|
|
SCHEMA_VERSION: int = 1
|
|
WILDCARD_DOMAIN: str = "*"
|
|
|
|
|
|
class ReviewerRegistryError(ValueError):
|
|
"""Raised when ``docs/reviewers.yaml`` fails ADR-0092 v1 schema validation."""
|
|
|
|
|
|
@dataclass(frozen=True, slots=True)
|
|
class Reviewer:
|
|
reviewer_id: str
|
|
display_name: str
|
|
role: str
|
|
domains: tuple[str, ...]
|
|
review_scope: tuple[str, ...]
|
|
provenance: str
|
|
|
|
|
|
@dataclass(frozen=True, slots=True)
|
|
class ExpertDemoClaim:
|
|
"""Reviewer-signed audit-passed promotion claim (ADR-0106 / ADR-0113).
|
|
|
|
A row in ``audit_passed_claims`` asserts that ``signed_by`` has
|
|
inspected the evidence at ``evidence_revision`` for ``domain_id``
|
|
across ``evidence_lanes`` and that the canonical evidence-bundle
|
|
SHA-256 equals ``claim_digest``. The reporting layer re-derives the
|
|
digest from the lane results on disk; a mismatch demotes the row
|
|
back to ``reasoning-capable``.
|
|
|
|
The class is still named ``ExpertDemoClaim`` for backward
|
|
compatibility (ADR-0113 scope: semantics-only rename — the YAML key
|
|
and ledger status move, but internal Python identifiers stay).
|
|
"""
|
|
|
|
domain_id: str
|
|
evidence_lanes: tuple[str, ...]
|
|
evidence_revision: str
|
|
signed_by: str
|
|
claim_digest: str
|
|
|
|
|
|
@dataclass(frozen=True, slots=True)
|
|
class ReviewerRegistry:
|
|
schema_version: int
|
|
reviewers: tuple[Reviewer, ...]
|
|
expert_demo_claims: tuple[ExpertDemoClaim, ...] = ()
|
|
|
|
def expert_demo_claim_for(self, domain_id: str) -> ExpertDemoClaim | None:
|
|
for claim in self.expert_demo_claims:
|
|
if claim.domain_id == domain_id:
|
|
return claim
|
|
return None
|
|
|
|
def resolve(self, reviewer_id: str) -> Reviewer | None:
|
|
for reviewer in self.reviewers:
|
|
if reviewer.reviewer_id == reviewer_id:
|
|
return reviewer
|
|
return None
|
|
|
|
def can_review(self, reviewer_id: str, *, domain_id: str, scope: str) -> bool:
|
|
"""Return True iff this reviewer may ratify ``scope`` artifacts for ``domain_id``.
|
|
|
|
Implements ADR-0091 predicate #8 plus ADR-0092 rules 2, 3, and 4:
|
|
- rule 2: unknown ``reviewer_id`` → False
|
|
- rule 3: ``role: domain`` reviewer whose ``domains`` do not include
|
|
``domain_id`` → False
|
|
- rule 4: reviewer whose ``review_scope`` does not cover ``scope`` → False
|
|
"""
|
|
reviewer = self.resolve(reviewer_id)
|
|
if reviewer is None:
|
|
return False
|
|
if scope not in reviewer.review_scope:
|
|
return False
|
|
if reviewer.role == "primary":
|
|
return True
|
|
return domain_id in reviewer.domains
|
|
|
|
|
|
def load_reviewer_registry(path: Path) -> ReviewerRegistry:
|
|
"""Parse ``path`` as a Reviewer Registry v1 file.
|
|
|
|
Raises :class:`ReviewerRegistryError` on any schema violation. No
|
|
side effects beyond reading ``path``.
|
|
"""
|
|
if not path.exists():
|
|
raise ReviewerRegistryError(f"reviewer registry not found at {path}")
|
|
|
|
try:
|
|
raw = yaml.safe_load(path.read_text(encoding="utf-8"))
|
|
except yaml.YAMLError as exc:
|
|
raise ReviewerRegistryError(f"reviewer registry YAML parse error: {exc}") from exc
|
|
|
|
if not isinstance(raw, Mapping):
|
|
raise ReviewerRegistryError("reviewer registry root must be a mapping")
|
|
|
|
unknown_top = set(raw.keys()) - ALLOWED_TOP_LEVEL_KEYS
|
|
if unknown_top:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer registry has unknown top-level keys: {sorted(unknown_top)}"
|
|
)
|
|
|
|
schema_version = raw.get("schema_version")
|
|
if schema_version != SCHEMA_VERSION:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer registry schema_version must be {SCHEMA_VERSION}; got {schema_version!r}"
|
|
)
|
|
|
|
reviewers_raw = raw.get("reviewers")
|
|
if reviewers_raw is None:
|
|
raise ReviewerRegistryError("reviewer registry missing 'reviewers' field")
|
|
if not isinstance(reviewers_raw, list):
|
|
raise ReviewerRegistryError("reviewer registry 'reviewers' must be a list")
|
|
|
|
parsed: list[Reviewer] = []
|
|
seen_ids: set[str] = set()
|
|
for index, entry in enumerate(reviewers_raw):
|
|
reviewer = _parse_reviewer(entry, index=index)
|
|
if reviewer.reviewer_id in seen_ids:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer registry contains duplicate reviewer_id {reviewer.reviewer_id!r}"
|
|
)
|
|
seen_ids.add(reviewer.reviewer_id)
|
|
parsed.append(reviewer)
|
|
|
|
claims_raw = raw.get("audit_passed_claims", [])
|
|
if not isinstance(claims_raw, list):
|
|
raise ReviewerRegistryError(
|
|
"reviewer registry 'audit_passed_claims' must be a list when present"
|
|
)
|
|
parsed_claims: list[ExpertDemoClaim] = []
|
|
seen_domains: set[str] = set()
|
|
reviewer_ids = {reviewer.reviewer_id for reviewer in parsed}
|
|
for index, entry in enumerate(claims_raw):
|
|
claim = _parse_expert_demo_claim(entry, index=index, reviewer_ids=reviewer_ids)
|
|
if claim.domain_id in seen_domains:
|
|
raise ReviewerRegistryError(
|
|
f"audit_passed_claims contains duplicate domain_id"
|
|
f"{claim.domain_id!r}"
|
|
)
|
|
seen_domains.add(claim.domain_id)
|
|
parsed_claims.append(claim)
|
|
|
|
return ReviewerRegistry(
|
|
schema_version=SCHEMA_VERSION,
|
|
reviewers=tuple(parsed),
|
|
expert_demo_claims=tuple(parsed_claims),
|
|
)
|
|
|
|
|
|
def _parse_expert_demo_claim(
|
|
entry: Any, *, index: int, reviewer_ids: set[str]
|
|
) -> ExpertDemoClaim:
|
|
if not isinstance(entry, Mapping):
|
|
raise ReviewerRegistryError(
|
|
f"audit_passed_claims entry at index {index} must be a mapping"
|
|
)
|
|
unknown = set(entry.keys()) - ALLOWED_EXPERT_DEMO_CLAIM_KEYS
|
|
if unknown:
|
|
raise ReviewerRegistryError(
|
|
f"audit_passed_claims entry at index {index} has unknown fields: "
|
|
f"{sorted(unknown)}"
|
|
)
|
|
missing = ALLOWED_EXPERT_DEMO_CLAIM_KEYS - set(entry.keys())
|
|
if missing:
|
|
raise ReviewerRegistryError(
|
|
f"audit_passed_claims entry at index {index} missing required "
|
|
f"fields: {sorted(missing)}"
|
|
)
|
|
domain_id = _require_nonempty_str(
|
|
entry["domain_id"], field="domain_id", index=index
|
|
)
|
|
evidence_revision = _require_nonempty_str(
|
|
entry["evidence_revision"], field="evidence_revision", index=index
|
|
)
|
|
signed_by = _require_nonempty_str(
|
|
entry["signed_by"], field="signed_by", index=index
|
|
)
|
|
claim_digest = _require_nonempty_str(
|
|
entry["claim_digest"], field="claim_digest", index=index
|
|
)
|
|
if signed_by not in reviewer_ids:
|
|
raise ReviewerRegistryError(
|
|
f"audit_passed_claims entry at index {index} signed_by "
|
|
f"{signed_by!r} does not resolve to a registered reviewer"
|
|
)
|
|
if len(claim_digest) != 64 or any(
|
|
c not in "0123456789abcdef" for c in claim_digest
|
|
):
|
|
raise ReviewerRegistryError(
|
|
f"audit_passed_claims entry at index {index} claim_digest must be "
|
|
"a lowercase 64-char SHA-256 hex string"
|
|
)
|
|
evidence_lanes = _require_str_list(
|
|
entry["evidence_lanes"], field="evidence_lanes", index=index
|
|
)
|
|
if not evidence_lanes:
|
|
raise ReviewerRegistryError(
|
|
f"audit_passed_claims entry at index {index} has empty "
|
|
"'evidence_lanes' list"
|
|
)
|
|
return ExpertDemoClaim(
|
|
domain_id=domain_id,
|
|
evidence_lanes=evidence_lanes,
|
|
evidence_revision=evidence_revision,
|
|
signed_by=signed_by,
|
|
claim_digest=claim_digest,
|
|
)
|
|
|
|
|
|
def _parse_reviewer(entry: Any, *, index: int) -> Reviewer:
|
|
if not isinstance(entry, Mapping):
|
|
raise ReviewerRegistryError(f"reviewer entry at index {index} must be a mapping")
|
|
|
|
unknown = set(entry.keys()) - ALLOWED_REVIEWER_KEYS
|
|
if unknown:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} has unknown fields: {sorted(unknown)}"
|
|
)
|
|
|
|
missing = ALLOWED_REVIEWER_KEYS - set(entry.keys())
|
|
if missing:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} missing required fields: {sorted(missing)}"
|
|
)
|
|
|
|
reviewer_id = _require_nonempty_str(entry["reviewer_id"], field="reviewer_id", index=index)
|
|
display_name = _require_nonempty_str(entry["display_name"], field="display_name", index=index)
|
|
role = _require_nonempty_str(entry["role"], field="role", index=index)
|
|
provenance = _require_nonempty_str(entry["provenance"], field="provenance", index=index)
|
|
|
|
if role not in ALLOWED_ROLES:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} has invalid role {role!r}; "
|
|
f"must be one of {sorted(ALLOWED_ROLES)}"
|
|
)
|
|
|
|
domains = _require_str_list(entry["domains"], field="domains", index=index)
|
|
if not domains:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} has empty 'domains' list"
|
|
)
|
|
if role == "domain" and WILDCARD_DOMAIN in domains:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} has role='domain' but claims wildcard "
|
|
f"'{WILDCARD_DOMAIN}'; only role='primary' may claim wildcard domains"
|
|
)
|
|
if role == "primary" and domains != (WILDCARD_DOMAIN,):
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} has role='primary' but domains "
|
|
f"is not exactly [{WILDCARD_DOMAIN!r}]; primary reviewers must claim wildcard"
|
|
)
|
|
|
|
review_scope = _require_str_list(entry["review_scope"], field="review_scope", index=index)
|
|
if not review_scope:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} has empty 'review_scope' list"
|
|
)
|
|
unknown_scopes = set(review_scope) - ALLOWED_SCOPES
|
|
if unknown_scopes:
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} has unknown review_scope values: "
|
|
f"{sorted(unknown_scopes)}; allowed: {sorted(ALLOWED_SCOPES)}"
|
|
)
|
|
|
|
return Reviewer(
|
|
reviewer_id=reviewer_id,
|
|
display_name=display_name,
|
|
role=role,
|
|
domains=domains,
|
|
review_scope=review_scope,
|
|
provenance=provenance,
|
|
)
|
|
|
|
|
|
def _require_nonempty_str(value: Any, *, field: str, index: int) -> str:
|
|
if not isinstance(value, str) or not value.strip():
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} field {field!r} must be a non-empty string"
|
|
)
|
|
return value
|
|
|
|
|
|
def _require_str_list(value: Any, *, field: str, index: int) -> tuple[str, ...]:
|
|
if not isinstance(value, list):
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} field {field!r} must be a list of strings"
|
|
)
|
|
for item in value:
|
|
if not isinstance(item, str) or not item.strip():
|
|
raise ReviewerRegistryError(
|
|
f"reviewer entry at index {index} field {field!r} contains a "
|
|
"non-string or empty value"
|
|
)
|
|
return tuple(value)
|