core/docs/research
Shay 13a4b05d62 test(reliability): volume-honesty invariant + distinct-evidence audit
Phase B of the curriculum-license-loop arc. Stacked on the ADR-0264 branch.

The invariant: conservative_floor is a one-sided Wilson lower bound and Wilson
assumes INDEPENDENT trials. CORE's pipeline is deterministic, so replaying an
identical case is not a second trial -- it is the same trial with a guaranteed
outcome. A ledger's `committed` count is an upper bound on its evidence; the
defensible figure is its distinct-decision count.

Phase B was scoped as a precaution for a curriculum producer that does not
exist yet. Running the instrument against the producers that DO exist found it
already violated:

  21 of the 25 ratified deduction_serve bands do not clear theta_SERVE=0.99 on
  distinct evidence. Three inflate 28 distinct cases into 720 committed
  (honest floor 0.8084 vs claimed 0.9909). deduction_serving_enabled was
  ratified ON 2026-07-24 and is live.

The estimation producer (ADR-0175) is clean -- 660 committed, 660 distinct,
zero repeats, evidently chosen just above the 657 a perfect record needs. So
this is a regression from a standard already established in the repo, not an
architectural gap. `claimed` is identical for all 25 deduction bands because
every band commits 720 with a perfect record: the gate sees 25
indistinguishable passes while honest floors span 0.8084..0.9909.

NOT REPAIRED. The ledger is SHA-sealed, ratified, and gating a live flag, so
re-sealing it is Shay's ratification, not a test's. wrong=0 still holds -- no
band answered anything incorrectly. What is not established is reliability at
the volume claimed. Measured, pinned in both directions, written down.

Added:
- core/reliability_gate/evidence.py -- pure measurement, imported by no serving
  path. volume_for_theta() DERIVES 657 from conservative_floor rather than
  restating it, so a WILSON_Z change cannot leave a stale literal behind.
- tests/test_volume_honesty.py -- 13 tests. AUDIT_SOURCES is checked against
  CAPABILITY_LEDGERS, so a new licensed capability cannot ship unaudited; the
  curriculum entry is a declared None that fails the moment its ledger exists.
  That is Phase C's forcing function against repeating the pattern.
- docs/research/distinct-evidence-audit-2026-07-25.md -- full audit + the open
  outcome-mix ruling for Shay.
- ADR-0264 R9 amended with the measured exposure.

Decision key is case TEXT deliberately: a tighter key (normalized atom) would
collapse spelling variants and report MORE inflation, so text-identity
under-reports and every number is a floor on the real gap.

Outcome mix is deliberately NOT pinned. ClassTally has no verdict axis, the
deduction producer already balances 240/240/240 by construction, and imposing
a per-verdict-class floor would retroactively fail all 25 bands on a criterion
no ADR has ratified (smallest per-band class count is 120). Recorded for a
ruling with numbers instead.

[Verification]: uv sync --locked on canonical CPython 3.12.13; in-worktree
smoke 569 passed (556 baseline + 13), deductive 285 passed. Registered in the
`smoke` curated suite -- not left to `full`, which gates nothing. Mutation-
checked: doubling CASES_PER_BAND -> 2 failed; reporting committed as distinct
-> 4 failed; gating on `claimed` instead of `honest` -> 3 failed; tree restores
to 13 passed.
2026-07-25 16:03:10 -07:00
..
0148-first-conversion-proof.md docs(reader-arc): recalibration ADR-0251 + preserved knowledge + overfit inventory 2026-07-19 13:42:47 -07:00
adr-0241-0242-adversarial-and-fidelity-findings.md docs(research): W1 adversarial findings + 4-blueprint integration plan 2026-07-15 12:49:00 -07:00
adr-0241-0242-blueprint-integration-plan.md docs: close ADR-0242 memo fidelity slice + living-summary update + sanctioned-seam note (follow-up batch) 2026-07-15 15:10:31 -07:00
ADR-0242-deterministic-fibonacci-operators-and-evidence-gated-optimization.md docs(analysis): rejected Drive-draft re-implementation — assessment + sketch-defect pins 2026-07-16 10:13:04 -07:00
ADR-0243-wave-field-cognitive-lifecycle-comprehension-reasoning-and-resonant-learning.md docs(governance): weekly-audit rulings T1/T7/T8/T9 — INV-32/33/34 registration, ADR-0243 fold, residual escalation, path sweep, prune ledger 2026-07-22 19:18:59 -07:00
adr-status-governance-2026-07-25.md chore(governance): stamp nine merged ADRs Accepted + pin the invariant 2026-07-25 14:38:02 -07:00
architecture-assessment-verification-2026-07-25.md feat(provenance,teaching): close the lateral gaps the assessment actually found 2026-07-25 04:51:15 +00:00
cga-hot-path-measurement-2026-07-25.md feat(provenance,teaching): close the lateral gaps the assessment actually found 2026-07-25 04:51:15 +00:00
compare-increment-funnel-2026-07-18.md revert(reader-arc): hold frame-anchoring out of increment 1 — defer to rate-frame increment (#77 ruling) 2026-07-18 20:03:38 -07:00
compare-multiplicative-increment-plan-2026-07-18.md docs(reader-arc): fold PR #76 amendments into compare increment plan; X=15 ratified 2026-07-18 16:07:41 -07:00
conditional-membership-fusion-band-2026-07-23.md feat(deduction-serve): Band v4-CM — conditional-membership fusion, decided (ADR-0259) 2026-07-24 10:14:00 -07:00
CORE-ASI-Super-Blueprint-Third-Door-Horizon.md fix(third-door): align with perfected package contracts from Downloads artifacts (refs #10 #11 #12 #13) 2026-07-11 22:05:02 -07:00
core_labs_core_technical_gap_audit.pdf docs(research): intelligence-loop arc opening record — plan + Spark-audit adjudication 2026-07-18 08:14:29 -07:00
core_repository_systems_audit_and_gap_analysis.pdf docs(research): intelligence-loop arc opening record — plan + Spark-audit adjudication 2026-07-18 08:14:29 -07:00
curriculum-grounded-serving-2026-07-24.md feat(generalization): curriculum-grounded serving — exams answered from ratified curriculum (ADR-0262) 2026-07-24 14:38:08 -07:00
curriculum-premise-scope-2026-07-25.md docs(adr): ADR-0264 — negative curriculum, premise scope, and band ceilings 2026-07-25 15:44:25 -07:00
curriculum-volume-quantification-2026-07-24.md docs(generalization): Tier S evidence packet — ratification record + curriculum-volume quantification (S1, S6) 2026-07-24 16:44:41 -07:00
deduction-serve-arc-completion-and-scope-outs-2026-07-23.md feat(deduction-serve): Phase 5 — end-to-end REPL proof, telemetry checks, scope-out docs 2026-07-23 13:20:45 -07:00
deduction-serve-arc-phase0-baseline-2026-07-23.md docs(deduction-serve): Phase 0 baseline — organs verified wrong=0, serving path disconnected 2026-07-23 11:21:34 -07:00
deduction-serve-arc-phase1-turn-spine-2026-07-23.md feat(deduction-serve): Phase 1 — deduction turn spine (flag-gated, propositional Band v1) 2026-07-23 12:14:28 -07:00
deduction-serve-arc-phase2-eval-lane-2026-07-23.md feat(deduction-serve): Phase 2 — end-to-end eval lane, SHA-pinned wrong=0 gate 2026-07-23 12:36:59 -07:00
deduction-serve-arc-phase3-earned-license-2026-07-23.md feat(deduction-serve): Phase 3 — earned SERVE license via reliability gate (ADR-0256) 2026-07-23 12:59:50 -07:00
deduction-serve-arc-phase4-categorical-band-2026-07-23.md feat(deduction-serve): Phase 4 — Band v1b categorical/syllogism serving (ADR-0256) 2026-07-23 13:17:35 -07:00
deduction-serve-phase0-baseline.json docs(deduction-serve): Phase 0 baseline — organs verified wrong=0, serving path disconnected 2026-07-23 11:21:34 -07:00
dev2-frontier-measurement-spike-2026-07-18.md docs(reader-arc): fold PR #76 amendments into compare increment plan; X=15 ratified 2026-07-18 16:07:41 -07:00
distinct-evidence-audit-2026-07-25.md test(reliability): volume-honesty invariant + distinct-evidence audit 2026-07-25 16:03:10 -07:00
english-argument-band-2026-07-23.md feat(deduction-serve): Band v2-EN — natural-English argument serving, earned licenses (ADR-0257) 2026-07-23 14:37:50 -07:00
epistemic_state_math_audit_findings.md docs(epistemic): align audit terminology with runtime enum 2026-05-28 04:17:19 -07:00
existential-witness-band-2026-07-24.md feat(deduction-serve): Band v6-EX — existential witnesses, decided (ADR-0261) 2026-07-24 14:14:37 -07:00
field-reasoner-wedge-selection.md docs(research): select quantitative field reasoner wedge 2026-06-04 17:23:57 -07:00
hitl-proposal-queue-cli-2026-07-24.md feat(teaching): HITL proposal-queue CLI — list/review contemplation-idle sinks (S4) 2026-07-24 17:05:26 -07:00
increment-1-band-plan-2026-07-19.md docs(reader-arc): band-solve=0 outcome + minimal-convertible-band measurement 2026-07-18 21:37:16 -07:00
independent-comprehension-agreement-gate.md docs(research): define independent comprehension agreement gate 2026-06-04 17:23:20 -07:00
intelligence-loop-homestretch-plan-2026-07-18.md docs(research): plan record — mark EXECUTED/merged, add missed Phase-4 DONE marker 2026-07-18 11:27:02 -07:00
lane-drift-investigation-2026-07-24.md fix(ci): re-pin public_demo lane — real content drift, not the env flake (S2) 2026-07-24 16:58:05 -07:00
math-phase-4-2-case-first-traces-2026-07-25.md feat(provenance,teaching): close the lateral gaps the assessment actually found 2026-07-25 04:51:15 +00:00
math-reader-phase-4-1-status-2026-07-24.md docs(math-reader): Phase 4.1 measured status — seeding injection is built, converts 0 (null result) 2026-07-24 14:46:55 -07:00
member-chain-band-2026-07-23.md feat(deduction-serve): Band v3-MEM — member-chain band, Socrates syllogism decided (ADR-0258) 2026-07-23 19:21:41 -07:00
paradigm-deadcode-audit-2026-07-19.md docs(paradigm): dead-code audit — zero removals, all candidates traced and verified 2026-07-19 15:15:52 -07:00
public-demo-lane-drift-2026-07-24.md fix(ci): re-pin public_demo lane — real content drift, not the env flake (S2) 2026-07-24 16:58:05 -07:00
ratification-packet-2026-07-24.md docs(generalization): Tier S evidence packet — ratification record + curriculum-volume quantification (S1, S6) 2026-07-24 16:44:41 -07:00
reader-arc-overfit-inventory-2026-07-19.md docs(reader-arc): recalibration ADR-0251 + preserved knowledge + overfit inventory 2026-07-19 13:42:47 -07:00
reader-arc-recalibration-preserved-knowledge-2026-07-19.md docs(reader-arc): recalibration ADR-0251 + preserved knowledge + overfit inventory 2026-07-19 13:42:47 -07:00
reader-hamiltonian-compiler-spike-2026-07-18.md feat(adr-0249): P1 conformal quantity kernel 2026-07-18 12:04:28 -07:00
register-matrix-truth-path-fix-2026-07-25.md docs(register-matrix): 801 passed, 0 failed — clears the DO NOT MERGE on bf17f42 2026-07-25 14:45:51 +00:00
robdd_proof_chain_design_note.md Add proof-chain corpus fixtures 2026-06-02 19:26:46 -07:00
spark-audit-adjudication-2026-07-18.md docs(research): intelligence-loop arc opening record — plan + Spark-audit adjudication 2026-07-18 08:14:29 -07:00
third-door-blueprint-fidelity.md docs(governance): RATIFY — ADR-0241 and ADR-0242 → Accepted (Joshua Shay, 2026-07-15) 2026-07-15 15:25:47 -07:00
THIRD-DOOR-LANDING-PACKAGE-README.md fix(third-door): align with perfected package contracts from Downloads artifacts (refs #10 #11 #12 #13) 2026-07-11 22:05:02 -07:00
tier-s-housekeeping-2026-07-24.md chore(generalization): Tier S housekeeping — smoke promotion, capability-index entries, promotion sweep (S5) 2026-07-24 17:13:19 -07:00
tier2-multi-entity-arithmetic-spike-2026-07-18.md docs(research): Tier-2 multi-entity arithmetic design spike 2026-07-18 14:01:15 -07:00
trackb-inc2-generalization-2026-07-20.md fix(trackb): pure-S1 extract refuse multi-clause overmatch (0361 wrong=0) 2026-07-19 20:04:27 -07:00
trackb-s1-slice-2026-07-20.md fix(trackb): pure-S1 gate — refuse total supersets and extra contains 2026-07-19 19:10:05 -07:00
verb-predicate-band-2026-07-24.md feat(deduction-serve): Band v5-VP — verb-predicate arguments, decided (ADR-0260) 2026-07-24 12:51:36 -07:00
vocab-trigger-instrument-2026-07-24.md feat(generalization): vocab-trigger instrument — mechanism-vs-coverage refusal histogram (S3) 2026-07-24 16:54:13 -07:00