57a0e38f0b
A narrow, auditable proof path for the hybrid-mode boundary: a Claude/Fable-style System 1 proposer submits an MCP-shaped tool call (core.semantic_derivation.verify); CORE re-derives via the ADR-0184 derivation lane, keeps sole acceptance authority, refuses or asks when it cannot honestly answer, and emits a deterministic replay/provenance trace for every decision. Boundary properties (each pinned by a failing-if-removed test): - the proposer has no commit path: additionalProperties:false schema, and the demo modules import nothing from generate.* (AST-scanned with a planted-bypass self-test) — the lane is consumed only through the ADR-0184 S4b audited trace facade; - 'verified' requires pool commit AND clean commit-law/faithfulness audits AND a gold-audited envelope.json entry AND byte-match to the pinned derivation trace (measured: 118 of the off-serving pool's 231 corpus commits disagree with lane gold, so a bare pool commit is never served — scenario s4 shows the refusal on a commit that even matches gold); - the ask leg is the real Q1 stack (router -> limitation -> Q1-D producer -> carried handle -> gated serving), dark by default, tamper-fail-closed on the content hash; - authority_path names only consulted authorities; responses are deterministic (double-run byte-identical) and pinned under expected/. No serving/runtime change; no CLAIMS/metrics/telemetry/lane-pin movement; no network or Anthropic API dependency (System 1 payloads are clearly-labeled static fixtures).
5 lines
4.2 KiB
JSON
5 lines
4.2 KiB
JSON
{"scenario_id": "s1-verified-grounded-chain", "title": "Verified: grounded accumulation chain inside the audited envelope", "system1": {"framing": "Claude/Fable-style System 1 proposal — static fixture payload; no model API was called and no network is used", "tool_call": {"name": "core.semantic_derivation.verify", "arguments": {"problem_text": "Sara has 32 dollars. Sara earns 15 dollars. She uses 21 dollars. How many dollars does Sara have left?", "domain_hint": "gsm8k_math", "request_id": "demo-s1", "return_trace": true}}}, "expected_status": "verified", "note": "Pool commits 26 dollars (gain+loss chain, every gate passes); the demo envelope authorizes it (gold-audited: evals/gsm8k_math/public/v1 case gma-141, expected 26)."}
|
||
{"scenario_id": "s2-refused-disagreement", "title": "Refused: two self-verifying readings disagree, so CORE will not guess", "system1": {"framing": "Claude/Fable-style System 1 proposal — static fixture payload; no model API was called and no network is used", "tool_call": {"name": "core.semantic_derivation.verify", "arguments": {"problem_text": "Tom’s cat is 8 years old. His rabbit is half the age of his cat. His dog is three times as old as his rabbit. How old is the dog?", "domain_hint": "gsm8k_math", "request_id": "demo-s2", "return_trace": true}}}, "expected_status": "refused", "note": "Two complete readings derive 24 and 36; both are wrong (gold 12, holdout case gsm8k-holdout-dev-v1-0005). The disagreement rule refuses BOTH — exactly the wrong=0 behavior the demo exists to show."}
|
||
{"scenario_id": "s3-ask-missing-total", "title": "Ask: CORE asks for the missing datum instead of inventing it", "system1": {"framing": "Claude/Fable-style System 1 proposal — static fixture payload; no model API was called and no network is used", "tool_call": {"name": "core.semantic_derivation.verify", "arguments": {"problem_text": "Each large bus holds 50 students and each small bus holds 30 students. The buses carry 260 students in total. How many large buses are there?", "domain_hint": "gsm8k_math", "request_id": "demo-s3", "return_trace": true}}}, "expected_status": "ask", "note": "The pool refuses (an additive mis-read of 340 is blocked by a disagreeing rival reading); the R2 constraint organ identifies the missing total bus count (missing_total_count, evals/constraint_oracle/r2_gold.jsonl case r2-011-missing-total-count); the Q1 stack renders and serves a grounded question through the carried-handle seam."}
|
||
{"scenario_id": "s4-refused-outside-envelope", "title": "Refused: a pool commit outside the audited envelope is not served", "system1": {"framing": "Claude/Fable-style System 1 proposal — static fixture payload; no model API was called and no network is used", "tool_call": {"name": "core.semantic_derivation.verify", "arguments": {"problem_text": "Jack has 17 toy cars. His dad buys him 8 more. How many toy cars does Jack have?", "domain_hint": "gsm8k_math", "request_id": "demo-s4", "return_trace": true}}}, "expected_status": "refused", "note": "The pool commits 25 (which happens to match gold: evals/gsm8k_math/practice/v1 case gsm8k-practice-v1-0022, answer 25), but this problem is deliberately NOT in the demo envelope — the demo still refuses, because the off-serving derivation lane is not a wrong=0 oracle over arbitrary text and an unaudited commit must not be served as verified. No false epistemic status."}
|
||
{"scenario_id": "s5-invalid-answer-smuggling", "title": "Invalid: the proposer tries to smuggle an answer through the payload", "system1": {"framing": "Claude/Fable-style System 1 proposal — static fixture payload (deliberately malformed); no model API was called and no network is used", "tool_call": {"name": "core.semantic_derivation.verify", "arguments": {"problem_text": "Each large bus holds 50 students and each small bus holds 30 students. The buses carry 260 students in total. How many large buses are there?", "proposed_answer": 340, "confidence": "high", "request_id": "demo-s5"}}}, "expected_status": "invalid", "note": "additionalProperties is false: there is no field through which the proposer can assert an answer, a derivation, or an acceptance decision. The payload is rejected at the typed boundary and no derivation is executed."}
|