Commit graph

202 commits

Author SHA1 Message Date
Shay
c21068ed3e feat: ADR-0119.4 — frontier-baseline comparison (ADR-0114a Obligation #7) 2026-05-22 17:33:28 -07:00
Shay
1a37f97c4f feat: ADR-0119.2 — author 200 grade-school math problems for the GSM8K eval lane (dev + public) 2026-05-22 17:28:00 -07:00
Shay
f9dd650df0 Merge remote-tracking branch 'origin/main' into feat/adr-0119.1-sealed-holdout-fabrication-control
# Conflicts:
#	docs/decisions/README.md
2026-05-22 17:24:32 -07:00
Shay
32c0a90ad9 feat: ADR-0119.1 — seal fabrication_control holdout with age encryption (Obligation #1) 2026-05-22 17:22:46 -07:00
Shay
93a149fc45
Merge pull request #142 from AssetOverflow/feat/adr-0119-gsm8k-eval-lane-roadmap
docs: ADR-0119 — GSM8K eval lane roadmap (Phase 5 decomposition)
2026-05-22 17:22:23 -07:00
Shay
98eb4d9f75
Merge pull request #141 from AssetOverflow/feat/adr-0118-stepped-realizer
feat: ADR-0118 — stepped realizer (SolutionTrace → show-your-work prose)
2026-05-22 17:20:31 -07:00
Shay
5ac3e3ff2f docs: ADR-0119 — GSM8K eval lane roadmap (Phase 5 decomposition)
Umbrella ADR for Phase 5 of ADR-0114. Decomposes the eval-lane work
into eight sub-phases (5.1..5.8) that ship under their own ADRs.

Sub-phase scope:
  5.1 — sealed-holdout encryption (one lane, fabrication_control)
  5.2 — CORE-original GSM8K-style corpus (dev + public, 200 cases)
  5.3 — lane runner (parser → solver → verifier → realizer)
  5.4 — frontier-baseline comparison (Obligation #7)
  5.5 — adversarial generation; misparse rate zero (Obligation #8)
  5.6 — depth-curve measurement harness (Obligation #6, measurement)
  5.7 — sealed GSM8K test as the holdout (Obligation #1, lane-side)
  5.8 — overall lane gate; new gsm8k_capability_shape

Roadmap only — no code lands with this ADR. Each sub-phase ships
independently and discharges a specific ADR-0114a obligation.

Critical invariant: dev + public splits are CORE-original; the real
GSM8K test set enters only via the encrypted holdout under 5.7.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:18:19 -07:00
Shay
c1d726179a feat: add ADR-0125 perturbation suite 2026-05-22 17:12:33 -07:00
Shay
7ad3f72cb4 feat: ADR-0118 — stepped realizer (SolutionTrace → show-your-work prose)
Phase 4 of the ADR-0114 GSM8K-math roadmap. Consumes a SolutionTrace
and emits one sentence per step plus setup + answer sentences. Pure
function; same trace → byte-equal RealizedTrace.

What landed

generate/math_realizer.py
  - realize(initial_state, trace) -> RealizedTrace
  - Frozen RealizedTrace dataclass with canonical_bytes() + as_prose()
  - Per-kind sentence rules (add / subtract / transfer / multiply×2 /
    multiply×3 / multiply-general / divide)
  - Singular/plural surface rule matches parser canonicalization
  - Typed RealizerError on unrecognized step kinds

tests/test_math_realizer.py — 60 cases pinning five invariants:
  1. All 50 dev-set cases realize without error
  2. Determinism: byte-equal RealizedTrace across two calls
  3. Setup sentence count == initial_state count
  4. Step sentence count == operation count
  5. Answer sentence contains the resolved value + unit

ADR-0114a obligation discharge update

ADR-0118 hardens determinism (#9) across a third layer (realizer)
and makes #3 / #10 human-inspectable via the prose surface. No
obligation is directly newly discharged by ADR-0118; it's substrate
for ADR-0119 GSM8K eval lane.

Round-trippability of the prose through the parser is explicitly
out of scope for this phase. The trace is the verifiable artifact
(ADR-0117); the prose is human-readable documentation.

Tests: 60 new realizer cases; 546 total green across realizer +
parser + solver + verifier + OOD; 67/67 smoke green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 17:11:10 -07:00
Shay
a8d019d8a0 docs: ADR-0123a — document all_three_pass_rate synonym in inference_shape 2026-05-22 17:00:51 -07:00
Shay
1a929a4e83 feat: ADR-0124 — systems_software audit-passed promotion (third successful) 2026-05-22 16:55:41 -07:00
Shay
9d2a5f22e3 feat: ADR-0118a OOD surface generator 2026-05-22 16:49:40 -07:00
Shay
5e52cd4547
Merge pull request #136 from AssetOverflow/feat/adr-0123-lane-shape-symbolic-logic-fix
feat: ADR-0123 — re-map symbolic_logic to inference_shape (unblocks ADR-0122)
2026-05-22 16:45:52 -07:00
Shay
4336490731 feat: ADR-0117 — SolutionTrace verifier (solver-independent)
Phase 3 of the ADR-0114 expert-capability roadmap. Re-applies every
step of a SolutionTrace from the input graph's initial state and
asserts byte-equal reproduction of answer_value. Pure function; same
(graph, trace) → byte-equal VerifierVerdict.

Why this is distinct from the solver

ADR-0116's solver enforces correctness at construction. ADR-0117's
verifier is a SECOND, INDEPENDENT implementation that re-derives
every value the trace claims. The verifier does NOT call solve(). It
re-implements the operation semantics from ADR-0116 directly inside
_verify_step. If the solver had a bug or was tampered with after the
fact, the verifier catches it.

Six checks per verdict (named, ordered, audit-logged):
  1. graph_canonical_hash_matches
  2. pack_id_matches
  3. pack_lemmas_resolve
  4. step_pack_lemma_ids_match_bindings
  5. step_replay_matches_before_after
  6. answer_value_reproduces

Seven named tamper classes all caught:
  - mutated before_value / after_value / operand of any step
  - mutated pack_lemma_id of any step
  - mutated graph_canonical_hash
  - mutated answer_value
  - mutated pack_id
  - mutated target_before / target_after of transfer step

ADR-0114a obligation update

  #3 Replay-equal trace — now discharged at VERIFIER FIDELITY
     (was solver-only under ADR-0116). A third party with only
     (graph, trace, pack) can reproduce the answer byte-equal.

Five of ten obligations now load-bearing: #3, #4, #9, #10 plus
in-flight #2 (Codex's ADR-0118a OOD generator).

Tests: 62/62 verifier suite green; 67/67 smoke green; existing
solver + parser + schema suites unaffected.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 16:40:38 -07:00
Shay
71321a5058 feat: ADR-0123 — re-map symbolic_logic to inference_shape (unblocks ADR-0122) 2026-05-22 16:39:53 -07:00
Shay
a0e9833851 feat: ADR-0122 systems_software audit-passed deferred (lane-shape mismatch) 2026-05-22 16:31:59 -07:00
Shay
d2f5607167 feat: ADR-0116 — deterministic solver + en_arithmetic_v1 operator pack
Phase 2 of the ADR-0114 expert-capability roadmap. Consumes the
MathProblemGraph from Phase 1 and emits a SolutionTrace — ordered
operation applications ending at a numeric answer, byte-deterministic
across runs, with each step's operation bound to a pack-resolved
lemma identifier.

What landed

generate/math_solver.py
  - solve(graph) -> SolutionTrace; pure function, no I/O, no globals
  - SolutionStep dataclass with before/after values per step (for
    verifier replay; ADR-0117 hardens)
  - SolutionTrace with canonical_bytes() byte-deterministic JSON
  - SolveError typed refusal: missing pack, division by zero,
    unknown-references-nothing

language_packs/data/en_arithmetic_v1/
  - 5 operator lemmas: add / subtract / multiply / divide / transfer
  - role=operational_base (vocabulary-only; no domain claim)
  - SHA-256-anchored lexicon + glosses; manifest carries
    provenance=adr-0116:operator_seed:2026-05-22

tests/test_math_solver.py — 109 cases pinning five invariants:
  1. Phase 2 exit criterion: ≥ 0.80 on parser-correct dev set
     (current: 50/50 = 1.00)
  2. Determinism: two solves produce byte-equal trace
  3. Trace replay reproduces answer_value (verifier rehearsal)
  4. Typed refusal on under-determined inputs
  5. Every step.pack_lemma_id resolves to a real lexicon entry
     in en_arithmetic_v1

ADR-0114a obligation discharge

Four of ten anti-overfitting obligations now have load-bearing
implementations in code:

  #3  replay-equal trace                 — discharged (solver-layer)
  #4  typed refusal                      — discharged (solver-layer)
  #9  determinism                        — discharged (solver-layer)
  #10 operation provenance via pack      — DISCHARGED IN FULL

Removing the en_arithmetic_v1 pack now breaks every solve loudly.
The "operations bind to concepts, not hardcoded strings" claim is
architecturally true, not rhetorical.

Tests: 109/109 green on solver suite; 67/67 smoke suite green;
parser + schema suites still green from prior phases.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 16:28:04 -07:00
Shay
46ded6b287 docs: ADR-0114a — anti-overfitting proof obligations for expert promotion
Documentation-only amendment to ADR-0114. Locks in the 10-point
falsifiable proof framework that any future `expert` ledger-tier
promotion (ADR-0120+) MUST require.

The obligations:

  1.  Sealed-holdout discipline
  2.  OOD surface variation ≥ 0.95 of public
  3.  Every correct answer ships with replay-equal trace
  4.  Refusal is first-class; misparse rate zero; zero `wrong` answers
  5.  Reasoning-isolation perturbation suite (invariance + predictable change)
  6.  Compositional-depth curve flat within documented ε
  7.  Frontier-baseline comparison on identical items, published
  8.  Adversarial generation; misparse rate zero
  9.  Determinism across release boundaries
  10. Operation provenance via the pack (not hardcoded strings)

Each obligation is load-bearing and falsifiable: a domain that
cannot satisfy any one stays at `audit-passed`. ADR-0114a binds
ADR-0116..ADR-0120 to carry the obligations into implementation;
ADR-0120 finally invokes all ten as hard gates.

The audit-passed tier (ADR-0106/0109/0113) is unaffected. The two
tiers measure orthogonal properties: audit-passed verifies CORE
claim-shape compliance (transformer-unreachable invariants); expert
verifies capability with anti-overfitting proof.

No code change. Pure forward contract for the next phase of work.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 16:16:52 -07:00
Shay
57b257ca1d feat: ADR-0115 Phase 1.1 — math problem graph schema + 5 seed cases
First Phase of ADR-0114's expert-capability roadmap. Decomposed into four
sub-phases so each lands as its own auditable step:

  1.1  schema + 5 seed cases + invariants   ← this commit
  1.2  45 more dev-set cases                 ← delegated (Codex)
  1.3  the parser itself                     ← exit: ≥0.90 on dev set
  1.4  runtime binding                       ← if non-trivial

What landed

- generate/math_problem_graph.py — typed dataclasses (Quantity,
  InitialPossession, Operation, Unknown, MathProblemGraph) + frozen
  validation + canonical_bytes() byte-deterministic serialization +
  graph_from_dict roundtrip.

- evals/gsm8k_parser_dev/cases.jsonl — 5 seed cases (gpd-001..005)
  covering single-add, single-subtract, multi-step, two-entity
  transfer, and multi-entity sum constructions. Every case carries a
  ground_truth_graph and the documented patterns it exercises.

- evals/gsm8k_parser_dev/README.md — authoring contract: schema,
  pattern registry, canonicalization rules, Phase 1.1 scope boundary,
  hand-solving rubric, distribution target for the remaining 45
  cases. This is the spec Phase 1.2 authors work against.

- tests/test_math_problem_graph.py — 26 cases pinning four invariants:
  round-trip byte equality, canonical_bytes() determinism, schema
  rejection of malformed graphs, and ground_truth_graph ↔
  expected_answer agreement (a hand-solver inside the test module
  falsifies mis-authored cases).

Why this is sticky

The Phase 1.1 schema is load-bearing for Phase 1.2 (the 45 authored
cases will be written against it) AND Phase 1.3 (the parser will be
graded byte-equal against ground-truth graphs in this schema). Changing
the schema after Phase 1.2 lands requires an amendment ADR + rewriting
authored cases. The schema choices here are intentionally conservative.

Tests: 26/26 new; 67/67 smoke green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 15:50:34 -07:00
Shay
696f62abdd feat: ADR-0113 rename expert-demoaudit-passed; reserve expert namespace (ADR-0114 GSM8K roadmap)
The word "expert" in the previous status name implied raw-capability parity
with frontier LLMs on the same benchmark — which the gate does NOT verify.
What the gate actually verifies is CORE *claim-shape compliance*:

  * signed digest (replay-reproducible from on-disk lane results)
  * replay determinism (same inputs → byte-equal trace_hash)
  * typed refusal (fabrication refused, not paraphrased)
  * exact recall (no ANN, no cosine, no attention bottleneck)
  * grounding-source provenance

These are claim shapes a transformer LLM cannot structurally produce
regardless of raw accuracy. A frontier LLM might score higher on the
same benchmark but cannot pass this contract.

Rename scope (semantics only, per ADR-0113):

  status string         "expert-demo"        → "audit-passed"
  predicate key         predicates.expert_demo → predicates.audit_passed
  reason key            expert_demo_reason   → audit_passed_reason
  YAML key              expert_demo_claims   → audit_passed_claims
  CLI command           core demo expert     → core demo audit-passed
  output dir            evals/expert_demos/  → evals/audit_passed/
  artifact filenames    expert_demo.{json,html} → audit_passed.{json,html}
  HTML title            CORE Expert-Demo: X  → CORE Audit-Passed: X

Internal Python identifiers (module/file/function/class names like
`expert_demo.py`, `evaluate_expert_demo`, `ExpertDemoClaim`,
`expert_demo_claim_for`) are deliberately kept to minimize churn. ADR
file titles (ADR-0106..0112) preserved as historical record.

`expert` namespace reserved for ADR-0114+: an actual capability tier
above `audit-passed` backed by a public benchmark with a stated
threshold. ADR-0114 proposes the first such target — GSM8K-math —
laying out a falsifiable 7-phase arc (parser → solver → verifier →
stepped-realizer → eval lane → first `expert` ledger tier promotion).

Tests: 184 directly-affected tests green (140 capability/expert-demo
suite + 34 demo/audit-tour + 10 correction-cue). Smoke suite 67/67.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 15:36:10 -07:00
Shay
bd7005c786 feat: ADR-0112 runnable expert-demo showcase (core demo expert --domain <id>)
Closes the asymmetry between the `expert-demo` ledger status (audit
artifact only) and the actual `core demo` surface (runnable
walkthroughs producing HTML + JSON). Until this commit the word
"demo" in `expert-demo` was aspirational; now it corresponds to
something a reader can open.

What it does

- Reads the signed expert_demo_claims entry from docs/reviewers.yaml
- Loads latest on-disk result files for each attached lane × split
- Re-derives the evidence-bundle digest and asserts byte-for-byte
  match against the signed claim_digest — this is the load-bearing
  audit step, now exercised at two independent enforcement points
  (ledger gate + showcase)
- Runs each lane's metrics through the ADR-0109 lane-shape registry
  and surfaces the verdict
- Picks the first three cases from each split verbatim (deterministic
  by file order) and renders them as HTML for inspection
- Emits expert_demo.json (canonical bytes, deterministic) + expert_demo.html

Surface

  core demo expert --domain mathematics_logic
  core demo expert --domain physics
  # → evals/expert_demos/<domain>/latest/expert_demo.{json,html}

Read-only by construction: cannot mutate docs/reviewers.yaml or any
lane result file. Tested. Unpromoted domains raise ValueError —
no silent fallback, no "preview" mode that fakes a showcase.

Generated artifacts are gitignored — the inputs they derive from are
already committed, so duplicating the renders would just churn the
tree.

Tests: 16 new cases pinning all five ADR-0112 invariants. Smoke suite
still 67/67 green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 14:59:27 -07:00
Shay
6a0905103b docs: backfill physics expert-demo (ADR-0111) across narrative docs
ADR-0111 landed via PR #123 (merge commit 4b5e6b7). The structural docs
(README, ADR index, reviewers.yaml, CLAIMS.md) were updated in that PR.
The narrative docs that reference the worked-promotion arc by ordinal
("first promotion / one demonstrated domain") needed a follow-up sweep:

- docs/PROGRESS.md — ADR-0100 footnote, ADR-0111 row, ledger table,
  Open-within-Phase-6 changed from "second" to "third" promotion
- docs/runtime_contracts.md — narrative line on contract demonstration
- docs/Whitepaper.md §XIII — chain range, demonstration paragraph,
  "two domains demonstrated" line, full-chain pointer
- docs/capability_roadmap.md — Phase 6 worked-promotion narrative step 5
  and exit-criteria checklist (5th item now ☑, 6th open as "third")

docs/Yellowpaper.md is unchanged: its expert-demo section is the formal
contract spec, domain-agnostic. docs/eval_methodology.md unchanged for
the same reason. docs/sessions/SESSION-2026-05-22-contract-layer-arc.md
is left as the historical snapshot of the original session.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 14:46:33 -07:00
Shay
45272a7bb2 feat: ADR-0111 physics expert-demo promotion (second successful)
Second worked promotion exercising the ADR-0106 + ADR-0109 contract
on a domain distinct from mathematics_logic. No contract change.

Evidence:
- foundational_physics_ood: accuracy=1.0 (117/117 public, 39/39 holdout)
- inference_closure: all_pass_rate=1.0 (shared with math, distinct digest via domain_id)
- fabrication_control: refused=n, fabricated=0 across all classes (shared)

Signed claim digest: a104cad136f3219df05dc7ce6a78437c02f7b5827cd3cdce568db3acda6a43ed

Bridge landed: cases_plaintext.jsonl dev-mode fallback for
foundational_physics_ood (matches ADR-0105 convention; analogous to the
math/inference bridges in ADR-0110). One small file, not a contract change.

Tests:
- tests/test_adr_0111_physics_expert_demo.py — 4 invariants, 6 cases
- tests/test_adr_0110_math_expert_demo.py — relaxed "only math promoted"
  to "math stays promoted" (load-bearing for ADR-0110 is persistence)
- tests/test_capability_reports.py — physics row now expert-demo

Retires the "first promotion was math-specific" objection: the bridges
ADR-0110 landed were correctly scoped, and the contract holds across
two distinct domains using shared lane infrastructure with distinct
digests.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 14:37:36 -07:00
Shay
bc90622e53
docs: runtime_contracts + eval_methodology + papers + session log (audit #3) (#122)
Closes the documentation audit. All four lower-priority surfaces from
the earlier scan now reflect the post-ADR-0110 reality.

New:
- docs/sessions/SESSION-2026-05-22-contract-layer-arc.md: full
  narrative of the ADR-0103 -> ADR-0110 arc, including the
  refused/amended/succeeded narrative, infrastructure bridges, and
  ledger state at session close. Pairs with the 2026-05-21
  articulation-arc session log.

Extended (additive, no rewrites):
- docs/runtime_contracts.md: new 'Expert-Demo Promotion Contract
  (ADR-0106 + ADR-0109)' section. Surfaces ledger_report() shape,
  reviewer-yaml schema, lane-shape registry, replay invariant,
  fail-closed registry behavior, and trust boundary.
- docs/eval_methodology.md: new 'Lane-shape registry (ADR-0109)'
  section. Documents what to do when adding a new eval lane:
  pick a shape or amend ADR-0109. Notes the holdout-runner gating
  (cases_plaintext.jsonl vs sealed). Header bumped to 2026-05-22.
- docs/Whitepaper.md: new section XIII 'Evidence-Governed Domain
  Layer' between XII (Forward Semantic Control) and the
  Extensions/closing. Narrates the contract chain at the
  philosophical / external-reader level. The original §XIII
  becomes §XIV.
- docs/Yellowpaper.md: new section XII 'Ratification Contract
  (ADR-0091 + ADR-0106 + ADR-0109)'. Formal specification of the
  nine predicates, the promotion predicate (with explicit logical
  form), the digest function, and the lane-shape registry table.
  The original §XII becomes §XIII.

No code, no tests, no contract changes.
2026-05-22 13:28:03 -07:00
Shay
95d3b4a45a
docs: backfill PROGRESS.md + capability_roadmap.md with Phase 6 + ADR-0027..0110 (audit #2) (#121)
PROGRESS.md was last updated mid-Phase-5 (~ADR-0021). The pack-layer
chain (ADR-0027..0045), forward-graph+surface composer chain
(ADR-0046..0089), and evidence-governed domain layer (ADR-0091..0110)
were all absent.

capability_roadmap.md was last updated 2026-05-17 and predated the
expert-demo promotion contract entirely.

Added to both:

- Phase 6 — Evidence-Governed Domain Layer section covering ADR-0091
  contract chain through ADR-0110 expert-demo arc, including the
  refused-amended-succeeded narrative and the current ledger state
  (math at expert-demo, four others at reasoning-capable).
- Backfill subsections for ADR-0027..0045 (pack-layer chain) and
  ADR-0046..0089 (forward-graph + surface-composer + register +
  anchor-lens + contemplation-loop work) marked retrospective.

capability_roadmap.md also gains a '2026-05-22' status update at the
top alongside the existing 2026-05-17 entry. The full Phase 0-5
exit-criteria rewrite remains queued separately.

No code, no tests, no claims changes.
2026-05-22 13:21:22 -07:00
Shay
95e5e93d02
docs: README fixes + ADR index cleanup post-ADR-0110 (audit #1) (#120)
- README.md: fix broken evals/CLAIMS.md link to root CLAIMS.md;
  add 'Evidence-Governed Domain Layer' section pointing to the
  ADR-0091/0096/0106/0109 contract chain and current ledger state
  (math at expert-demo, four other domains at reasoning-capable).
- docs/decisions/README.md: extend the 'fully accepted' slate from
  0091..0105 to 0091..0110 with ADR-0106 through 0110 entries; extend
  the 'Evidence-governed domain chain' chain-notes section to reflect
  the expert-demo arc (refused-amended-succeeded narrative).
2026-05-22 13:14:19 -07:00
Shay
5f149340cc
feat(contemplation): land ADR-0080 phase 1 (#119) 2026-05-22 13:10:03 -07:00
Shay
5cad0a4b72
feat(capability): ADR-0110 promote mathematics_logic to expert_demo (#118)
First worked expert-demo promotion under the ADR-0106 + ADR-0109
contract. Math is now the first domain at expert_demo=true.

Signed claim (docs/reviewers.yaml):
  domain_id: mathematics_logic
  evidence_lanes: [elementary_mathematics_ood, inference_closure,
                   fabrication_control]
  evidence_revision: adr-0110:reviewed:2026-05-22
  signed_by: shay-j
  claim_digest: 94d74781e103854230c1a71590e4df2287f5d2e87832f1c29b8ec4618853c04b

Evidence (all three lanes, public + holdout):
  elementary_mathematics_ood: accuracy=1.0 (117/117 public, 39/39 holdout)
  inference_closure: all_pass_rate=1.0, replay_determinism=1.0,
                     overall_pass=True (20 public, 12 holdout)
  fabrication_control: by-class refusals 3/3/3, fabricated=0
                       (9 public, 9 holdout)

Infrastructure bridges (not contract changes):
- cases_plaintext.jsonl dev-mode fallback files for
  elementary_mathematics_ood + inference_closure (ADR-0105 pattern)
- 9 new holdout cases for fabrication_control across all three
  refusal classes (phantom_endpoint / cross_pack_non_bridge /
  sibling_collapse)
- core/capability/reporting.py: _fetch_lane_split folds top-level
  by_class into metrics so refusal_shape sees a canonical layout

Tests:
- tests/test_adr_0110_math_expert_demo.py: 4 invariant tests
  (math_expert_demo_holds, signed_claim_present, replay_digest_
  byte_equality, other_domains_unaffected)
- tests/test_adr_0107_deferral.py retired (deferral resolved)
- tests/test_expert_demo_contract.py: production-ledger test
  rewritten as 'every promoted domain has signed claim' (load-
  bearing invariant preserved)
- tests/test_capability_reports.py: math row asserted at
  expert-demo (was reasoning-capable)

Ledger state:
  systems_software: reasoning-capable
  mathematics_logic: EXPERT-DEMO   <- new
  physics: reasoning-capable
  hebrew_greek_textual_reasoning: reasoning-capable
  philosophy_theology: reasoning-capable

README updated. ADR-0107 referenced as resolved by this ADR.
CLAIMS.md regenerated. ADR-0106 / ADR-0109 contract unchanged.
2026-05-22 12:59:23 -07:00
Shay
36053317be
feat(capability): implement ADR-0109 lane-shape-aware thresholds (#116)
Replaces the cognition-shape-uniform threshold dispatch in
core/capability/expert_demo.py with an explicit LANE_SHAPE_REGISTRY
mapping 8 ratified lane ids to 5 shapes:

  cognition           -> cognition_shape
  elementary_math_ood -> accuracy_shape
  foundational_physics_ood -> accuracy_shape
  symbolic_logic      -> symbolic_logic_shape
  hebrew_fluency      -> accuracy_shape
  koine_greek_fluency -> accuracy_shape
  inference_closure   -> inference_shape
  fabrication_control -> refusal_shape

Each shape has a documented threshold checker. Unknown lane ids
fail-closed with a named reason. ADR-0106 \xc2\xa71.1/\xc2\xa71.3/\xc2\xa71.4/\xc2\xa71.5
unchanged; only \xc2\xa71.2 (threshold rules) dispatches by shape.

tests/test_lane_shape_thresholds.py pins all four ADR-0109 invariants
plus dead-shape and threshold-value gates (13 new tests).
tests/test_expert_demo_contract.py fixtures updated to provide
shape-appropriate metrics (no semantic change to those tests; same
12 cases still pin the ADR-0106 contract).

ADR-0109 status: Proposed -> Accepted. README sequencing updated
(ADR-0110 now only blocked by inference_closure, not by metric-shape
amendment).

Ledger: all five domains remain reasoning-capable, expert_demo=false.
2026-05-22 12:11:58 -07:00
Shay
c2dd69611b
docs: propose ADR-0109 lane-shape-aware threshold amendment (#115)
Amends ADR-0106 \xc2\xa71.2 to dispatch threshold rules by lane shape rather
than imposing cognition-pack-shape metrics uniformly. ADR-0107
surfaced that every non-cognition lane was failing the gate by
absence-of-key, not by substance.

Status: Proposed. Ships five shapes covering every lane currently
attached to a ratified pack: cognition_shape, accuracy_shape,
inference_shape, refusal_shape, symbolic_logic_shape. Four invariants
pinned. Unknown lanes fail closed; new shapes require ADR amendment.

\xc2\xa71.1 (reasoning-capable prereq), \xc2\xa71.3 (signature scoping), \xc2\xa71.4
(domain-aware), \xc2\xa71.5 (replay byte-equality) all preserved. ADR-0106
status remains Accepted.
2026-05-22 12:00:59 -07:00
Shay
7cc2f7b422
feat(adr): ADR-0107 mathematics_logic expert-demo promotion deferred (#114)
The ADR-0106 contract correctly refused promotion. ADR-0107 records the
deferral and reserves two follow-up ADRs:

- ADR-0109 (lane-shape-aware threshold amendment): ADR-0106 \xc2\xa71.2
  prescribes cognition-pack-shape metrics uniformly, but math /
  physics / systems / hebrew-greek lanes carry native shapes
  (accuracy, passed_rate, all_pass_rate). Prerequisite for any future
  expert-demo promotion.
- ADR-0110 (math re-attempt): conditional on ADR-0109 landing and
  inference_closure substantively passing (currently all_pass_rate=0.4
  on public).

tests/test_adr_0107_deferral.py pins adr_0107_no_silent_promotion: math
stays at reasoning-capable, has no expert_demo_claims entry, and the
ledger row carries a named refusal reason.

No change to core/capability/expert_demo.py or reporting.py -- the
contract is honored, not amended. README sequencing updated to reflect
ADR-0107 acceptance and the new ADR-0109/0110 prerequisites.
2026-05-22 11:49:37 -07:00
Shay
0493808215
feat(capability): implement ADR-0106 expert-demo promotion contract (#113)
Closes ADR-0106 acceptance evidence:

- ExpertDemoClaim dataclass + additive expert_demo_claims block on
  ReviewerRegistry (schema_version stays at 1; backward-compatible).
- New core/capability/expert_demo.py with derive_evidence_digest,
  evaluate_expert_demo, collect_domain_lanes, materialise_lane_results.
- core/capability/reporting.py: replaces the cognition-lane-only
  predicate (previous lines 418-433) with a domain-aware,
  reviewer-signed gate; ledger rows now also carry
  expert_demo_reason for operator legibility. Reviewer registry is
  fail-closed: an unloadable registry yields zero claims, so a broken
  registry never silently grants expert_demo=true.
- tests/test_expert_demo_contract.py covers all three ADR-0106
  invariants: requires_signature, domain_aware, replay_byte_equality;
  plus threshold + production-ledger-untouched gates. 12 new tests.
- tests/test_reviewer_registry.py extended with TestExpertDemoClaimsSchema
  covering omitted block, valid parse, unknown signer rejection,
  malformed digest rejection, duplicate domain rejection. 5 new tests.
- README index row + table preface updated to note expert_demo is
  contract-gated. Frontier list trimmed (ADR-0106 has landed).
- ADR-0106 Status flipped Proposed -> Accepted.

No domain row's expert_demo field flips by this PR -- only the contract
changes. Promotion of any ratified domain requires a follow-up ADR
(ADR-0107 reserved for mathematics_logic) plus a signed claim.
2026-05-22 11:39:09 -07:00
Shay
7bb8334fc5
docs: accept ADR-0108 and rewrite Current frontier per its acceptance evidence (#112)
- ADR-0108 Status: Proposed -> Accepted
- README index row updated to Accepted
- 'Current frontier' rewritten with the ranked Proposed-ADR list mandated
  by ADR-0108 \xc2\xa7Decision; removes the now-false 'No ADR currently sits in
  a "Proposed but unimplemented" state' sentence
- Open candidate directions (no-ADR-yet) section retained for the
  multi-reviewer governance frontier item from ADR-0105
2026-05-22 11:21:20 -07:00
Shay
7e50738034
docs: propose ADR-0108 sequencing for currently-Proposed ADRs (#111)
Makes the post-ADR-0105 sequencing of ADR-0080 / 0084 / 0087 / 0106
explicit, durable, and revisable. Status: Proposed. No content of the
four sequenced ADRs is modified — sequencing is meta, not content.
2026-05-22 10:52:44 -07:00
Shay
8e749546b2
docs: propose ADR-0106 expert-demo promotion contract (#109)
Defines a domain-aware, reviewer-signed expert_demo promotion gate to
replace the current cognition-lane-only predicate in
core/capability/reporting.py:418. Status: Proposed. This ADR does not
promote any domain — it defines the contract that a follow-up ADR (likely
mathematics_logic as ADR-0107) will consume as the first worked
promotion.
2026-05-22 10:44:36 -07:00
Shay
257fd4503d
feat(evals): ADR-0105 — sealed holdout encryption via age (#108)
* feat(evals): add pyrage dependency

* feat(evals): add sealed holdout path resolution

* feat(evals): implement sealed holdout decryption

* feat(evals): add sealed holdout CLI

* test(evals): add sealed holdout encryption tests

* docs(decisions): add ADR-0105 sealed holdout encryption

* feat(evals): route holdout split through sealed decryptor

* docs(decisions): add ADR-0105 index entry

* chore: restore project description

* fix(evals): use pyrage Identity.from_str and pin curriculum SHA

- holdout_runner: pyrage exposes Identity.from_str, not from_file; parse
  identity file by line and pass list[Identity] into decrypt(). Restores
  PR 108's sealed-holdout test suite to green.
- verify_lane_shas: realign curriculum_loop_closure pin with the actual
  deterministic runner output (carryover from PR 107).
2026-05-22 10:09:43 -07:00
Shay
f7680e96ea
feat(teaching): ADR-0104 — curriculum-sourced teaching proposals (#107)
* feat(teaching): add curriculum-sourced proposal builder

* test(teaching): cover curriculum proposal construction

* test(evals): add curriculum loop closure contract

* test(evals): add curriculum loop closure runner

* test(evals): add canonical curriculum loop closure report

* ci(lanes): pin curriculum loop closure lane

* docs(adr): add ADR-0104 curriculum sourced proposals

* docs(adr): register ADR-0104 and seven pinned lanes

* docs(teaching): mark curriculum source activation

* fix(ci): pin curriculum_loop_closure SHA to runner output

* fix(ci): register curriculum_loop_closure in CLAIMS.md generator
2026-05-22 10:05:14 -07:00
Shay
1395ec1354
feat(packs): ADR-0103 — attach hebrew_fluency + koine_greek_fluency lanes to ADR-0102 (#106)
* feat(evals): add Hebrew fluency holdout cases

* feat(evals): add Koine Greek fluency holdout cases

* feat(packs): attach fluency lanes to he_core_cognition_v1

* feat(packs): attach fluency lanes to he_logos_micro_v1

* feat(packs): attach fluency lanes to grc_logos_cognition_v1

* feat(packs): ADR-0103 fluency lane attachment

* test(packs): expect ADR-0103 fluency lanes on Hebrew Greek contracts

* docs(evals): add Hebrew fluency holdout split note

* docs(evals): add Koine Greek fluency holdout split note

* docs(evals): note Hebrew holdout attachment

* docs(evals): note Koine Greek holdout attachment

* docs: add ADR 0103 placeholder

* docs(adr): expand ADR-0103 fluency lane attachment

* docs: index ADR-0103 and refresh frontier
2026-05-22 09:43:46 -07:00
Shay
57fa078aaa docs: mark ADR-0094/0095/0098/0099 accepted
Sibling reconciliation PR to #104. The four ADRs explicitly called out as
the 'current implementation frontier' in PR #104 are already implemented
to the same evidence bar as the eight ADRs that PR accepted:

- ADR-0094: teaching/source.py + proposal schema widening + migration
  script; tests/test_proposal_source.py green
- ADR-0095: teaching/from_miner.py + miner_loop_closure lane;
  SHA-pinned in scripts/verify_lane_shas.py; tests/test_miner_proposals.py
  green
- ADR-0098: core/demos/contract.py + adapter surface + demo_composition
  lane; SHA-pinned; tests/test_demo_composition.py green
- ADR-0099: core/demos/showcase.py + public_demo lane;
  SHA-pinned; tests/test_public_showcase.py green

Three of four lanes are SHA-pinned in CI (a stricter bar than several
already-accepted ADRs). Local pytest run: 85/85 passed across the four
tests/test_*.py files in 17s.

Also refreshes docs/decisions/README.md:
- flips the four table rows to Accepted (2026-05-22)
- rewrites the 'Current frontier' section now that no ADR-0091..0102
  entry is unimplemented
- enumerates candidate next directions (curriculum proposals,
  language-specific holdout splits, expert-demo ratification)

Docs-only change; no runtime code touched.
2026-05-22 07:09:14 -07:00
Shay
fe8d6db224 docs: update ADR index and current frontier 2026-05-22 06:47:25 -07:00
Shay
f21b1a9ba6 docs: mark ADR 0102 accepted 2026-05-22 06:42:27 -07:00
Shay
dde928859c docs: mark ADR 0101 accepted 2026-05-22 06:34:35 -07:00
Shay
2bc3e67474 docs: mark ADR 0100 accepted 2026-05-22 06:30:53 -07:00
Shay
d0d1a96669 docs: mark ADR 0097 accepted 2026-05-22 06:28:51 -07:00
Shay
852585a83d docs: mark ADR 0096 accepted 2026-05-22 06:27:25 -07:00
Shay
32d6ff6576 docs: mark ADR 0093 accepted 2026-05-22 06:25:38 -07:00
Shay
cf8ad47c53 docs: mark ADR 0092 accepted 2026-05-22 06:24:03 -07:00
Shay
87762e34fa docs: mark ADR 0091 accepted 2026-05-22 06:21:59 -07:00
Shay
b9a6f2ddb5 feat(packs): ADR-0100/0101/0102 — three sibling domain ratifications
Ratifies the remaining three sibling domains as reasoning-capable
under ADR-0091's Domain Pack Contract v1, using the template
ADR-0097 established for mathematics_logic. The capability ledger
now has four reasoning-capable rows backed by validated contracts.

ADR-0100 physics (en_physics_v1):
  domain_id: physics
  claimed_operators: causal, modal
  teaching_chains: [physics_chains_v1]
  eval_lanes: foundational_physics_ood, inference_closure,
    fabrication_control
  9/9 predicates pass

ADR-0101 systems_software (en_systems_software_v1):
  domain_id: systems_software
  claimed_operators: transitive, causal
  teaching_chains: [systems_software_chains_v1]
  eval_lanes: symbolic_logic, inference_closure, fabrication_control
  9/9 predicates pass

ADR-0102 hebrew_greek_textual_reasoning (FIRST MULTI-PACK ratification):
  domain_id: hebrew_greek_textual_reasoning
  claimed_operators: causal, contradiction
  teaching_chains: [hebrew_greek_textual_reasoning_chains_v1]
  eval_lanes: inference_closure, fabrication_control
    (universal lanes only — language-specific fluency lanes lack
    holdout splits; a separate ADR adds those when holdouts ship)
  packs: grc_logos_micro_v1, grc_logos_cognition_v1,
    he_logos_micro_v1, he_core_cognition_v1
  all four pack contracts identical (uniformity invariant pinned);
  all four 9/9 predicates pass
  pre-existing gap: hebrew/greek manifests lacked a provenance field
  entirely; ratification fills that uniformly across the four packs

44 new ratification tests in test_adr_0100_0102_sibling_ratifications.py:
- 6 parametrized 9-predicate validation tests (one per pack)
- 21 per-domain ledger status assertions (status, reasoning_capable,
  expert_demo gated, no_open_gaps, provenance points at correct ADR,
  operator_chain_coverage, intent_shapes minimum) — 7 cases × 3 domains
- 15 per-domain contract field shape assertions (teaching_chains,
  eval_lanes, splits coverage, axioms/rules null, primary reviewer) —
  5 cases × 3 domains
- 2 ADR-0102 multi-pack uniformity invariants (all four packs carry
  the contract; contracts identical across packs)

Capability ledger after ratification:
  systems_software           : reasoning-capable
  mathematics_logic          : reasoning-capable
  physics                    : reasoning-capable
  hebrew_greek_textual_reasoning : reasoning-capable
  philosophy_theology        : reasoning-capable (no contract; pre-existing)

Lane SHA pin update:
- public_demo pin refreshed (21751aaf.. → 71090323..) — the
  ratification adds new manifest fields (provenance,
  domain_contract_*) that surface in pack-related demo paths;
  intentional ADR-tracked change per the verifier doctrine

Smoke 67/67, packs 6/6, sibling ratifications 44/44, cognition eval
byte-identical 100/100/100/100; all 6 lanes match pinned SHAs:
  reviewer_registry            681a2aab..
  miner_loop_closure           9f071733..
  domain_contract_validation   f9c06cde..
  fabrication_control_summary  01e1b6b7..
  demo_composition             27d83824..
  public_demo                  71090323..
2026-05-21 20:25:48 -07:00
Shay
afdd2ee413 feat(capability): implement ADR-0092 — Reviewer Registry v1
Closes the load-bearing gap blocking every reasoning-capable claim
under ADR-0091: docs/reviewers.yaml was previously `reviewers: []` and
unparsed. Now schema-validated at v1, with a bootstrap shay-j entry
self-sealed via provenance.

- new core.capability.reviewers module: frozen Reviewer/ReviewerRegistry
  dataclasses, strict load_reviewer_registry parser, ReviewerRegistryError
- enforces ADR-0092 schema rules: schema_version==1, no unknown
  top-level keys, no unknown reviewer fields, role∈{primary,domain},
  primary must claim ["*"], domain must NOT claim "*", review_scope
  subset of {pack,proposal,chain,eval}, no duplicate reviewer_ids
- can_review(reviewer_id, domain_id, scope) helper implements
  ADR-0092 rules 2-4 for downstream use by ADR-0093 validator
- docs/reviewers.yaml updated to v1 schema with shay-j bootstrap
- ledger_report() evidence_counts now exposes structured
  reviewer_registry status (valid, schema_version, reviewer_count,
  reviewer_ids, error) alongside the legacy reviewers_present bool
- new evals/reviewer_registry/ lane: 6 cases (2 positive + 4 negative)
  covering empty-registry, wrong-version, domain-wildcard rejection,
  and unknown-field rejection
- runner emits deterministic JSON report; two runs produce byte-identical
  output (sha256 verified)
- 26 unit tests in tests/test_reviewer_registry.py
- capability ledger test extended to assert new reviewer_registry block
- smoke suite green (67/67); lane passes 6/6

The pre-existing test_flag_report_tracks_default_off_flags failure is
unrelated (discourse_planner flag default) and not introduced here.
2026-05-21 18:01:24 -07:00