core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
core/docs/decisions/ADR-0106-expert-demo-promotion-contract.md
Shay 0493808215
feat(capability): implement ADR-0106 expert-demo promotion contract (#113) 
Closes ADR-0106 acceptance evidence:

- ExpertDemoClaim dataclass + additive expert_demo_claims block on
  ReviewerRegistry (schema_version stays at 1; backward-compatible).
- New core/capability/expert_demo.py with derive_evidence_digest,
  evaluate_expert_demo, collect_domain_lanes, materialise_lane_results.
- core/capability/reporting.py: replaces the cognition-lane-only
  predicate (previous lines 418-433) with a domain-aware,
  reviewer-signed gate; ledger rows now also carry
  expert_demo_reason for operator legibility. Reviewer registry is
  fail-closed: an unloadable registry yields zero claims, so a broken
  registry never silently grants expert_demo=true.
- tests/test_expert_demo_contract.py covers all three ADR-0106
  invariants: requires_signature, domain_aware, replay_byte_equality;
  plus threshold + production-ledger-untouched gates. 12 new tests.
- tests/test_reviewer_registry.py extended with TestExpertDemoClaimsSchema
  covering omitted block, valid parse, unknown signer rejection,
  malformed digest rejection, duplicate domain rejection. 5 new tests.
- README index row + table preface updated to note expert_demo is
  contract-gated. Frontier list trimmed (ADR-0106 has landed).
- ADR-0106 Status flipped Proposed -> Accepted.

No domain row's expert_demo field flips by this PR -- only the contract
changes. Promotion of any ratified domain requires a follow-up ADR
(ADR-0107 reserved for mathematics_logic) plus a signed claim.
2026-05-22 11:39:09 -07:00

5.3 KiB
Raw Permalink Blame History

ADR-0106 — Expert-Demo Promotion Contract

Status: Accepted Date: 2026-05-22 Author: CORE agents + reviewers Depends on: ADR-0091, ADR-0092, ADR-0093, ADR-0096, ADR-0097, ADR-0100, ADR-0101, ADR-0102

Context

Four domains now sit at reasoning-capable in the ledger (mathematics_logic, physics, systems_software, hebrew_greek_textual_reasoning). The expert_demo field on each ledger row is false and currently has no domain-aware contract behind it.

core/capability/reporting.py:418 is the only site that can flip expert_demo to true, and it does so by reading the cognition lane's public/holdout metrics regardless of which domain is being scored. This is incoherent: a math ratification cannot be promoted to expert_demo by evidence from a cognition pack it never claimed against.

Without a domain-aware contract, reasoning-capable is the ceiling — and that ceiling is "contract predicates satisfied, no fabrication, operator coverage above floor," which is not the same claim as "the system can do real expert work in this domain." External readers cannot distinguish the two, so the ratification ledger overstates what the system has actually demonstrated.

ADR-0106 closes that gap.

Decision

Introduce expert_demo as a domain-scoped, reviewer-signed capability promotion, not a global threshold.

1. Promotion predicate (domain-aware)

expert_demo = true for a ratified domain D requires all of:

  1. D is already reasoning-capable (ADR-0093 contract still holds).
  2. Every lane attached to D's ratified packs reports public and holdout splits at:
    • surface_groundedness >= 0.95
    • term_capture_rate >= 0.85
    • intent_accuracy >= 0.95
    • versor_closure_rate == 1.0
  3. D's fabrication_control lane is passed=1.0 on both public and holdout.
  4. A signed expert-demo claim exists in the reviewer registry under a new expert_demo_claims block, naming:
    • domain_id
    • evidence_lanes (list of lane ids consulted)
    • evidence_revision (git sha at promotion time)
    • signed_by (reviewer id from ADR-0092 registry)
    • claim_digest (SHA-256 over the canonical evidence bundle)
  5. The promotion is replay-deterministic: re-running the consulted lanes at evidence_revision reproduces claim_digest byte-for-byte.

If any condition fails the row stays at reasoning-capable. Promotion is never silent — it requires both mechanical thresholds AND a reviewer signature.

2. Reporting changes

core/capability/reporting.py stops consulting the cognition lane for non-cognition domains. The expert-demo branch consults the domain's own attached lanes. The cognition lane keeps its existing thresholds only when the domain under evaluation is the cognition pack itself.

3. First worked promotion

This ADR does not promote any domain. It defines the contract.

A follow-up ADR (ADR-0107, expected to be mathematics_logic) will be the first worked expert-demo promotion against the contract defined here, including the reviewer-signed claim and the SHA-pinned evidence bundle.

Invariants

expert_demo_requires_signature

No domain row may carry expert_demo=true without a corresponding entry in expert_demo_claims whose claim_digest matches the re-derived evidence.

expert_demo_domain_aware

The reporting layer must consult only lanes attached to the domain's ratified packs when computing expert_demo. Cross-domain lane bleed (e.g. cognition lane metrics deciding a math promotion) is rejected by a test gate.

expert_demo_replay_byte_equality

Re-running every consulted lane at evidence_revision must reproduce the exact JSON bytes hashed into claim_digest. A drift here demotes the row to reasoning-capable until re-signed.

Acceptance evidence

Accepted when the following land together:

  • expert_demo_claims block added to the reviewer registry schema (ADR-0092 extension, additive only)
  • domain-aware expert_demo predicate in core/capability/reporting.py
  • new test tests/test_expert_demo_contract.py covering the three invariants above
  • updated docs/decisions/README.md index and "Accepted reasoning-capable domains" table noting the expert_demo column is contract-gated (not thresholded only)
  • no domain row's expert_demo field flips by this ADR — only the contract changes

Consequences

  • "Reasoning-capable" becomes an honest ceiling: it means the contract predicates hold, not that the system has demonstrated expert work.
  • Promoting a domain to expert_demo=true now requires deliberate reviewer action and replayable evidence — the same discipline that ADR-0092 imposed on reviewers themselves.
  • The ledger becomes externally legible: an outside reader can tell at a glance which domains are contract-passing versus demonstrated.
  • Opens the door to ADR-0107+ as worked promotions, starting with mathematics_logic as the smallest expert-demo proof surface.

Out of scope

  • This ADR does not change Domain Pack Contract v1 (ADR-0091).
  • This ADR does not introduce new eval lanes.
  • This ADR does not promote any existing domain — promotions are separate ADRs that consume this contract.
  • Multi-reviewer threshold signing (ADR-0105 candidate frontier) is orthogonal and remains future work.