Commit graph

529 commits

Author SHA1 Message Date
Shay
a8d019d8a0 docs: ADR-0123a — document all_three_pass_rate synonym in inference_shape 2026-05-22 17:00:51 -07:00
Shay
1e52d71b51
Merge pull request #138 from AssetOverflow/feat/adr-0124-systems-software-audit-passed
feat: ADR-0124 — systems_software audit-passed promotion (third successful)
2026-05-22 16:59:17 -07:00
Shay
1a929a4e83 feat: ADR-0124 — systems_software audit-passed promotion (third successful) 2026-05-22 16:55:41 -07:00
Shay
f5323edb39
Merge pull request #137 from AssetOverflow/feat/adr-0118a-ood-surface-generator
feat: ADR-0118a — OOD surface generator (ADR-0114a Obligation #2)
2026-05-22 16:54:50 -07:00
Shay
9d2a5f22e3 feat: ADR-0118a OOD surface generator 2026-05-22 16:49:40 -07:00
Shay
5e52cd4547
Merge pull request #136 from AssetOverflow/feat/adr-0123-lane-shape-symbolic-logic-fix
feat: ADR-0123 — re-map symbolic_logic to inference_shape (unblocks ADR-0122)
2026-05-22 16:45:52 -07:00
Shay
3e8546c56b
Merge pull request #135 from AssetOverflow/feat/adr-0117-verifier
feat: ADR-0117 — SolutionTrace verifier (solver-independent)
2026-05-22 16:43:53 -07:00
Shay
4336490731 feat: ADR-0117 — SolutionTrace verifier (solver-independent)
Phase 3 of the ADR-0114 expert-capability roadmap. Re-applies every
step of a SolutionTrace from the input graph's initial state and
asserts byte-equal reproduction of answer_value. Pure function; same
(graph, trace) → byte-equal VerifierVerdict.

Why this is distinct from the solver

ADR-0116's solver enforces correctness at construction. ADR-0117's
verifier is a SECOND, INDEPENDENT implementation that re-derives
every value the trace claims. The verifier does NOT call solve(). It
re-implements the operation semantics from ADR-0116 directly inside
_verify_step. If the solver had a bug or was tampered with after the
fact, the verifier catches it.

Six checks per verdict (named, ordered, audit-logged):
  1. graph_canonical_hash_matches
  2. pack_id_matches
  3. pack_lemmas_resolve
  4. step_pack_lemma_ids_match_bindings
  5. step_replay_matches_before_after
  6. answer_value_reproduces

Seven named tamper classes all caught:
  - mutated before_value / after_value / operand of any step
  - mutated pack_lemma_id of any step
  - mutated graph_canonical_hash
  - mutated answer_value
  - mutated pack_id
  - mutated target_before / target_after of transfer step

ADR-0114a obligation update

  #3 Replay-equal trace — now discharged at VERIFIER FIDELITY
     (was solver-only under ADR-0116). A third party with only
     (graph, trace, pack) can reproduce the answer byte-equal.

Five of ten obligations now load-bearing: #3, #4, #9, #10 plus
in-flight #2 (Codex's ADR-0118a OOD generator).

Tests: 62/62 verifier suite green; 67/67 smoke green; existing
solver + parser + schema suites unaffected.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 16:40:38 -07:00
Shay
71321a5058 feat: ADR-0123 — re-map symbolic_logic to inference_shape (unblocks ADR-0122) 2026-05-22 16:39:53 -07:00
Shay
83a3824d65
Merge pull request #133 from AssetOverflow/feat/adr-0122-systems-software-audit-passed
feat: ADR-0122 systems_software audit-passed deferred (lane-shape mismatch)
2026-05-22 16:35:22 -07:00
Shay
a0e9833851 feat: ADR-0122 systems_software audit-passed deferred (lane-shape mismatch) 2026-05-22 16:31:59 -07:00
Shay
cdf0e99abc
Merge pull request #134 from AssetOverflow/feat/adr-0116-math-solver
feat: ADR-0116 — deterministic solver + en_arithmetic_v1 operator pack
2026-05-22 16:30:55 -07:00
Shay
d2f5607167 feat: ADR-0116 — deterministic solver + en_arithmetic_v1 operator pack
Phase 2 of the ADR-0114 expert-capability roadmap. Consumes the
MathProblemGraph from Phase 1 and emits a SolutionTrace — ordered
operation applications ending at a numeric answer, byte-deterministic
across runs, with each step's operation bound to a pack-resolved
lemma identifier.

What landed

generate/math_solver.py
  - solve(graph) -> SolutionTrace; pure function, no I/O, no globals
  - SolutionStep dataclass with before/after values per step (for
    verifier replay; ADR-0117 hardens)
  - SolutionTrace with canonical_bytes() byte-deterministic JSON
  - SolveError typed refusal: missing pack, division by zero,
    unknown-references-nothing

language_packs/data/en_arithmetic_v1/
  - 5 operator lemmas: add / subtract / multiply / divide / transfer
  - role=operational_base (vocabulary-only; no domain claim)
  - SHA-256-anchored lexicon + glosses; manifest carries
    provenance=adr-0116:operator_seed:2026-05-22

tests/test_math_solver.py — 109 cases pinning five invariants:
  1. Phase 2 exit criterion: ≥ 0.80 on parser-correct dev set
     (current: 50/50 = 1.00)
  2. Determinism: two solves produce byte-equal trace
  3. Trace replay reproduces answer_value (verifier rehearsal)
  4. Typed refusal on under-determined inputs
  5. Every step.pack_lemma_id resolves to a real lexicon entry
     in en_arithmetic_v1

ADR-0114a obligation discharge

Four of ten anti-overfitting obligations now have load-bearing
implementations in code:

  #3  replay-equal trace                 — discharged (solver-layer)
  #4  typed refusal                      — discharged (solver-layer)
  #9  determinism                        — discharged (solver-layer)
  #10 operation provenance via pack      — DISCHARGED IN FULL

Removing the en_arithmetic_v1 pack now breaks every solve loudly.
The "operations bind to concepts, not hardcoded strings" claim is
architecturally true, not rhetorical.

Tests: 109/109 green on solver suite; 67/67 smoke suite green;
parser + schema suites still green from prior phases.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 16:28:04 -07:00
Shay
bf267a3172
Merge pull request #132 from AssetOverflow/feat/adr-0114a-anti-overfitting-obligations
docs: ADR-0114a — anti-overfitting proof obligations for `expert` promotion
2026-05-22 16:20:53 -07:00
Shay
46ded6b287 docs: ADR-0114a — anti-overfitting proof obligations for expert promotion
Documentation-only amendment to ADR-0114. Locks in the 10-point
falsifiable proof framework that any future `expert` ledger-tier
promotion (ADR-0120+) MUST require.

The obligations:

  1.  Sealed-holdout discipline
  2.  OOD surface variation ≥ 0.95 of public
  3.  Every correct answer ships with replay-equal trace
  4.  Refusal is first-class; misparse rate zero; zero `wrong` answers
  5.  Reasoning-isolation perturbation suite (invariance + predictable change)
  6.  Compositional-depth curve flat within documented ε
  7.  Frontier-baseline comparison on identical items, published
  8.  Adversarial generation; misparse rate zero
  9.  Determinism across release boundaries
  10. Operation provenance via the pack (not hardcoded strings)

Each obligation is load-bearing and falsifiable: a domain that
cannot satisfy any one stays at `audit-passed`. ADR-0114a binds
ADR-0116..ADR-0120 to carry the obligations into implementation;
ADR-0120 finally invokes all ten as hard gates.

The audit-passed tier (ADR-0106/0109/0113) is unaffected. The two
tiers measure orthogonal properties: audit-passed verifies CORE
claim-shape compliance (transformer-unreachable invariants); expert
verifies capability with anti-overfitting proof.

No code change. Pure forward contract for the next phase of work.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 16:16:52 -07:00
Shay
b17dc397e7
Merge pull request #131 from AssetOverflow/feat/adr-0115-phase-1.2-dev-set-expand
feat: ADR-0115 Phase 1.2 — 45 additional dev-set cases (gpd-006 .. gpd-050)
2026-05-22 16:14:14 -07:00
Shay
dc0988416e feat: ADR-0115 Phase 1.2 — rewrite gpd-021 to drop metaphor / mixed units (parser hits 50/50) 2026-05-22 16:11:22 -07:00
Shay
2141bab3c7
Merge pull request #130 from AssetOverflow/feat/adr-0115-phase-1.3-parser
feat: ADR-0115 Phase 1.3 — deterministic math word-problem parser
2026-05-22 16:08:45 -07:00
Shay
1e7f5a2dbf
Merge pull request #127 from AssetOverflow/feat/adr-0115-math-problem-parser
feat: ADR-0115 Phase 1.1 — math problem graph schema + 5 seed cases
2026-05-22 16:04:52 -07:00
Shay
18503f3d6e feat: ADR-0115 Phase 1.3 — deterministic math word-problem parser
Closes Phase 1.3 of the ADR-0114 expert-capability roadmap. Turns a
grade-school word problem into a typed MathProblemGraph deterministically
(no LLM, no sampling). Same input string always produces the same
graph; unsupported constructions raise ParseError rather than guessing.

What the parser handles

  Initial possession:    "<E> has <N> <unit>."
  Add verbs:             buys, gets, finds, receives, earns, adds
                         (+ "<N> more" / unit elision via state.last_unit)
  Subtract verbs:        eats, loses, sells, donates, uses, spends, drops, removes
  Transfer verbs:        gives, sends, hands, passes, mails  (with target)
  Multiply (scalar):     "X doubles <obj>" / "X triples <obj>"
  Divide (split):        "X splits {them|his Y|N Y} evenly into M groups [and keeps one]"

  Compound sentences:    "X buys 5, then donates 3."
  Sentence opener:       "Then X eats 1."  (inherits subject + unit)
  Pronoun anaphora:      he/she/it → last-introduced singular subject
  Object pronoun:        them/these/those → state.last_unit
  Trailing PP:           "finds 7 buttons on the floor" — discarded
  Singular→plural:       "Iris has 1 coin" → canonical unit "coins"

  Questions:
    "How many <unit> does <E> have [left|now|in total|altogether]?"
    "How many <unit> do they have [in total|altogether|left|now]?"

What it explicitly rejects

  - Conditional / time-modal ("If X had ...")
  - Compound questions (two unknowns)
  - Multiple "?" sentences
  - Questions referencing entities never introduced
  - Empty / whitespace-only input

Verification

  - tests/test_math_parser.py: 20 cases (5 byte-equal parametrized
    + 5 determinism parametrized + 1 exit-criterion gate + 6 typed-
    refusal + 2 purity + 1 type check)
  - tests/test_math_problem_graph.py: 26 schema cases still green
  - On the 5 seed cases:  5/5 = 100% byte-equal
  - On Codex's PR #128 50-case dev set (locally tested):
    49/50 = 98% byte-equal. Single failure (gpd-021) is a case-
    quality issue, not a parser limit; feedback filed on #128 to
    rewrite (mixed units + metaphor not in pattern registry).
  - Phase 1.3 exit criterion (≥ 0.90): met.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 16:03:31 -07:00
Shay
61d3cf8095 feat: ADR-0115 Phase 1.2 — 45 additional dev-set cases (gpd-006 .. gpd-050) 2026-05-22 15:56:46 -07:00
Shay
57b257ca1d feat: ADR-0115 Phase 1.1 — math problem graph schema + 5 seed cases
First Phase of ADR-0114's expert-capability roadmap. Decomposed into four
sub-phases so each lands as its own auditable step:

  1.1  schema + 5 seed cases + invariants   ← this commit
  1.2  45 more dev-set cases                 ← delegated (Codex)
  1.3  the parser itself                     ← exit: ≥0.90 on dev set
  1.4  runtime binding                       ← if non-trivial

What landed

- generate/math_problem_graph.py — typed dataclasses (Quantity,
  InitialPossession, Operation, Unknown, MathProblemGraph) + frozen
  validation + canonical_bytes() byte-deterministic serialization +
  graph_from_dict roundtrip.

- evals/gsm8k_parser_dev/cases.jsonl — 5 seed cases (gpd-001..005)
  covering single-add, single-subtract, multi-step, two-entity
  transfer, and multi-entity sum constructions. Every case carries a
  ground_truth_graph and the documented patterns it exercises.

- evals/gsm8k_parser_dev/README.md — authoring contract: schema,
  pattern registry, canonicalization rules, Phase 1.1 scope boundary,
  hand-solving rubric, distribution target for the remaining 45
  cases. This is the spec Phase 1.2 authors work against.

- tests/test_math_problem_graph.py — 26 cases pinning four invariants:
  round-trip byte equality, canonical_bytes() determinism, schema
  rejection of malformed graphs, and ground_truth_graph ↔
  expected_answer agreement (a hand-solver inside the test module
  falsifies mis-authored cases).

Why this is sticky

The Phase 1.1 schema is load-bearing for Phase 1.2 (the 45 authored
cases will be written against it) AND Phase 1.3 (the parser will be
graded byte-equal against ground-truth graphs in this schema). Changing
the schema after Phase 1.2 lands requires an amendment ADR + rewriting
authored cases. The schema choices here are intentionally conservative.

Tests: 26/26 new; 67/67 smoke green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 15:50:34 -07:00
Shay
e8dfa8460c
Merge pull request #126 from AssetOverflow/feat/rename-expert-demo-to-audit-passed
feat: ADR-0113 rename `expert-demo` → `audit-passed` + ADR-0114 GSM8K-math expert roadmap
2026-05-22 15:39:13 -07:00
Shay
696f62abdd feat: ADR-0113 rename expert-demoaudit-passed; reserve expert namespace (ADR-0114 GSM8K roadmap)
The word "expert" in the previous status name implied raw-capability parity
with frontier LLMs on the same benchmark — which the gate does NOT verify.
What the gate actually verifies is CORE *claim-shape compliance*:

  * signed digest (replay-reproducible from on-disk lane results)
  * replay determinism (same inputs → byte-equal trace_hash)
  * typed refusal (fabrication refused, not paraphrased)
  * exact recall (no ANN, no cosine, no attention bottleneck)
  * grounding-source provenance

These are claim shapes a transformer LLM cannot structurally produce
regardless of raw accuracy. A frontier LLM might score higher on the
same benchmark but cannot pass this contract.

Rename scope (semantics only, per ADR-0113):

  status string         "expert-demo"        → "audit-passed"
  predicate key         predicates.expert_demo → predicates.audit_passed
  reason key            expert_demo_reason   → audit_passed_reason
  YAML key              expert_demo_claims   → audit_passed_claims
  CLI command           core demo expert     → core demo audit-passed
  output dir            evals/expert_demos/  → evals/audit_passed/
  artifact filenames    expert_demo.{json,html} → audit_passed.{json,html}
  HTML title            CORE Expert-Demo: X  → CORE Audit-Passed: X

Internal Python identifiers (module/file/function/class names like
`expert_demo.py`, `evaluate_expert_demo`, `ExpertDemoClaim`,
`expert_demo_claim_for`) are deliberately kept to minimize churn. ADR
file titles (ADR-0106..0112) preserved as historical record.

`expert` namespace reserved for ADR-0114+: an actual capability tier
above `audit-passed` backed by a public benchmark with a stated
threshold. ADR-0114 proposes the first such target — GSM8K-math —
laying out a falsifiable 7-phase arc (parser → solver → verifier →
stepped-realizer → eval lane → first `expert` ledger tier promotion).

Tests: 184 directly-affected tests green (140 capability/expert-demo
suite + 34 demo/audit-tour + 10 correction-cue). Smoke suite 67/67.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 15:36:10 -07:00
Shay
12e69eaa78
Merge pull request #125 from AssetOverflow/feat/expert-demo-runnable
feat: ADR-0112 — runnable expert-demo showcase (`core demo expert --domain <id>`)
2026-05-22 15:02:10 -07:00
Shay
bd7005c786 feat: ADR-0112 runnable expert-demo showcase (core demo expert --domain <id>)
Closes the asymmetry between the `expert-demo` ledger status (audit
artifact only) and the actual `core demo` surface (runnable
walkthroughs producing HTML + JSON). Until this commit the word
"demo" in `expert-demo` was aspirational; now it corresponds to
something a reader can open.

What it does

- Reads the signed expert_demo_claims entry from docs/reviewers.yaml
- Loads latest on-disk result files for each attached lane × split
- Re-derives the evidence-bundle digest and asserts byte-for-byte
  match against the signed claim_digest — this is the load-bearing
  audit step, now exercised at two independent enforcement points
  (ledger gate + showcase)
- Runs each lane's metrics through the ADR-0109 lane-shape registry
  and surfaces the verdict
- Picks the first three cases from each split verbatim (deterministic
  by file order) and renders them as HTML for inspection
- Emits expert_demo.json (canonical bytes, deterministic) + expert_demo.html

Surface

  core demo expert --domain mathematics_logic
  core demo expert --domain physics
  # → evals/expert_demos/<domain>/latest/expert_demo.{json,html}

Read-only by construction: cannot mutate docs/reviewers.yaml or any
lane result file. Tested. Unpromoted domains raise ValueError —
no silent fallback, no "preview" mode that fakes a showcase.

Generated artifacts are gitignored — the inputs they derive from are
already committed, so duplicating the renders would just churn the
tree.

Tests: 16 new cases pinning all five ADR-0112 invariants. Smoke suite
still 67/67 green.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 14:59:27 -07:00
Shay
4e66af644a
Merge pull request #124 from AssetOverflow/docs/adr-0111-followup
docs: backfill physics expert-demo (ADR-0111) across narrative docs
2026-05-22 14:49:31 -07:00
Shay
6a0905103b docs: backfill physics expert-demo (ADR-0111) across narrative docs
ADR-0111 landed via PR #123 (merge commit 4b5e6b7). The structural docs
(README, ADR index, reviewers.yaml, CLAIMS.md) were updated in that PR.
The narrative docs that reference the worked-promotion arc by ordinal
("first promotion / one demonstrated domain") needed a follow-up sweep:

- docs/PROGRESS.md — ADR-0100 footnote, ADR-0111 row, ledger table,
  Open-within-Phase-6 changed from "second" to "third" promotion
- docs/runtime_contracts.md — narrative line on contract demonstration
- docs/Whitepaper.md §XIII — chain range, demonstration paragraph,
  "two domains demonstrated" line, full-chain pointer
- docs/capability_roadmap.md — Phase 6 worked-promotion narrative step 5
  and exit-criteria checklist (5th item now ☑, 6th open as "third")

docs/Yellowpaper.md is unchanged: its expert-demo section is the formal
contract spec, domain-agnostic. docs/eval_methodology.md unchanged for
the same reason. docs/sessions/SESSION-2026-05-22-contract-layer-arc.md
is left as the historical snapshot of the original session.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 14:46:33 -07:00
Shay
4b5e6b762e
Merge pull request #123 from AssetOverflow/adr/0111-physics-expert-demo
ADR-0111: physics expert-demo promotion (second successful)
2026-05-22 14:44:45 -07:00
Shay
0c448a475c chore: regenerate CLAIMS.md for physics expert-demo
Picked up by the lane-shas workflow's CLAIMS.md freshness check.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 14:42:16 -07:00
Shay
45272a7bb2 feat: ADR-0111 physics expert-demo promotion (second successful)
Second worked promotion exercising the ADR-0106 + ADR-0109 contract
on a domain distinct from mathematics_logic. No contract change.

Evidence:
- foundational_physics_ood: accuracy=1.0 (117/117 public, 39/39 holdout)
- inference_closure: all_pass_rate=1.0 (shared with math, distinct digest via domain_id)
- fabrication_control: refused=n, fabricated=0 across all classes (shared)

Signed claim digest: a104cad136f3219df05dc7ce6a78437c02f7b5827cd3cdce568db3acda6a43ed

Bridge landed: cases_plaintext.jsonl dev-mode fallback for
foundational_physics_ood (matches ADR-0105 convention; analogous to the
math/inference bridges in ADR-0110). One small file, not a contract change.

Tests:
- tests/test_adr_0111_physics_expert_demo.py — 4 invariants, 6 cases
- tests/test_adr_0110_math_expert_demo.py — relaxed "only math promoted"
  to "math stays promoted" (load-bearing for ADR-0110 is persistence)
- tests/test_capability_reports.py — physics row now expert-demo

Retires the "first promotion was math-specific" objection: the bridges
ADR-0110 landed were correctly scoped, and the contract holds across
two distinct domains using shared lane infrastructure with distinct
digests.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 14:37:36 -07:00
Shay
bc90622e53
docs: runtime_contracts + eval_methodology + papers + session log (audit #3) (#122)
Closes the documentation audit. All four lower-priority surfaces from
the earlier scan now reflect the post-ADR-0110 reality.

New:
- docs/sessions/SESSION-2026-05-22-contract-layer-arc.md: full
  narrative of the ADR-0103 -> ADR-0110 arc, including the
  refused/amended/succeeded narrative, infrastructure bridges, and
  ledger state at session close. Pairs with the 2026-05-21
  articulation-arc session log.

Extended (additive, no rewrites):
- docs/runtime_contracts.md: new 'Expert-Demo Promotion Contract
  (ADR-0106 + ADR-0109)' section. Surfaces ledger_report() shape,
  reviewer-yaml schema, lane-shape registry, replay invariant,
  fail-closed registry behavior, and trust boundary.
- docs/eval_methodology.md: new 'Lane-shape registry (ADR-0109)'
  section. Documents what to do when adding a new eval lane:
  pick a shape or amend ADR-0109. Notes the holdout-runner gating
  (cases_plaintext.jsonl vs sealed). Header bumped to 2026-05-22.
- docs/Whitepaper.md: new section XIII 'Evidence-Governed Domain
  Layer' between XII (Forward Semantic Control) and the
  Extensions/closing. Narrates the contract chain at the
  philosophical / external-reader level. The original §XIII
  becomes §XIV.
- docs/Yellowpaper.md: new section XII 'Ratification Contract
  (ADR-0091 + ADR-0106 + ADR-0109)'. Formal specification of the
  nine predicates, the promotion predicate (with explicit logical
  form), the digest function, and the lane-shape registry table.
  The original §XII becomes §XIII.

No code, no tests, no contract changes.
2026-05-22 13:28:03 -07:00
Shay
95d3b4a45a
docs: backfill PROGRESS.md + capability_roadmap.md with Phase 6 + ADR-0027..0110 (audit #2) (#121)
PROGRESS.md was last updated mid-Phase-5 (~ADR-0021). The pack-layer
chain (ADR-0027..0045), forward-graph+surface composer chain
(ADR-0046..0089), and evidence-governed domain layer (ADR-0091..0110)
were all absent.

capability_roadmap.md was last updated 2026-05-17 and predated the
expert-demo promotion contract entirely.

Added to both:

- Phase 6 — Evidence-Governed Domain Layer section covering ADR-0091
  contract chain through ADR-0110 expert-demo arc, including the
  refused-amended-succeeded narrative and the current ledger state
  (math at expert-demo, four others at reasoning-capable).
- Backfill subsections for ADR-0027..0045 (pack-layer chain) and
  ADR-0046..0089 (forward-graph + surface-composer + register +
  anchor-lens + contemplation-loop work) marked retrospective.

capability_roadmap.md also gains a '2026-05-22' status update at the
top alongside the existing 2026-05-17 entry. The full Phase 0-5
exit-criteria rewrite remains queued separately.

No code, no tests, no claims changes.
2026-05-22 13:21:22 -07:00
Shay
95e5e93d02
docs: README fixes + ADR index cleanup post-ADR-0110 (audit #1) (#120)
- README.md: fix broken evals/CLAIMS.md link to root CLAIMS.md;
  add 'Evidence-Governed Domain Layer' section pointing to the
  ADR-0091/0096/0106/0109 contract chain and current ledger state
  (math at expert-demo, four other domains at reasoning-capable).
- docs/decisions/README.md: extend the 'fully accepted' slate from
  0091..0105 to 0091..0110 with ADR-0106 through 0110 entries; extend
  the 'Evidence-governed domain chain' chain-notes section to reflect
  the expert-demo arc (refused-amended-succeeded narrative).
2026-05-22 13:14:19 -07:00
Shay
5f149340cc
feat(contemplation): land ADR-0080 phase 1 (#119) 2026-05-22 13:10:03 -07:00
Shay
5cad0a4b72
feat(capability): ADR-0110 promote mathematics_logic to expert_demo (#118)
First worked expert-demo promotion under the ADR-0106 + ADR-0109
contract. Math is now the first domain at expert_demo=true.

Signed claim (docs/reviewers.yaml):
  domain_id: mathematics_logic
  evidence_lanes: [elementary_mathematics_ood, inference_closure,
                   fabrication_control]
  evidence_revision: adr-0110:reviewed:2026-05-22
  signed_by: shay-j
  claim_digest: 94d74781e103854230c1a71590e4df2287f5d2e87832f1c29b8ec4618853c04b

Evidence (all three lanes, public + holdout):
  elementary_mathematics_ood: accuracy=1.0 (117/117 public, 39/39 holdout)
  inference_closure: all_pass_rate=1.0, replay_determinism=1.0,
                     overall_pass=True (20 public, 12 holdout)
  fabrication_control: by-class refusals 3/3/3, fabricated=0
                       (9 public, 9 holdout)

Infrastructure bridges (not contract changes):
- cases_plaintext.jsonl dev-mode fallback files for
  elementary_mathematics_ood + inference_closure (ADR-0105 pattern)
- 9 new holdout cases for fabrication_control across all three
  refusal classes (phantom_endpoint / cross_pack_non_bridge /
  sibling_collapse)
- core/capability/reporting.py: _fetch_lane_split folds top-level
  by_class into metrics so refusal_shape sees a canonical layout

Tests:
- tests/test_adr_0110_math_expert_demo.py: 4 invariant tests
  (math_expert_demo_holds, signed_claim_present, replay_digest_
  byte_equality, other_domains_unaffected)
- tests/test_adr_0107_deferral.py retired (deferral resolved)
- tests/test_expert_demo_contract.py: production-ledger test
  rewritten as 'every promoted domain has signed claim' (load-
  bearing invariant preserved)
- tests/test_capability_reports.py: math row asserted at
  expert-demo (was reasoning-capable)

Ledger state:
  systems_software: reasoning-capable
  mathematics_logic: EXPERT-DEMO   <- new
  physics: reasoning-capable
  hebrew_greek_textual_reasoning: reasoning-capable
  philosophy_theology: reasoning-capable

README updated. ADR-0107 referenced as resolved by this ADR.
CLAIMS.md regenerated. ADR-0106 / ADR-0109 contract unchanged.
2026-05-22 12:59:23 -07:00
Shay
360905db4d
fix(intent): route 'Actually X R Y' premises to CORRECTION (inference_closure) (#117)
Between 2026-05-17 and 2026-05-22 the inference_closure lane regressed
from all_pass_rate=1.0 to 0.4 on public. Root cause: the
_DECLARATIVE_RELATION_RE branch in generate/intent.py runs ahead of the
_RULES loop and swallowed sentences beginning with 'Actually' into the
subject phrase, routing them to VERIFICATION. The lane's premise emit
path is gated on CORRECTION intent, so PackMutationProposal records
stopped being emitted for any non-'is' relation (precedes / grounds /
causes / reveals). Only the four transitive_is cases passed because
'is' is not in the declarative-relation verb list.

Fix: _CORRECTION_CUE_PREFIX_RE guard. When the text begins with a
correction cue ('Actually', 'Incorrect, ', 'No, ', 'Correction'), the
declarative-match branch is skipped and the sentence falls through to
the _RULES CORRECTION rule. Plain declarative-relation assertions still
route to VERIFICATION unchanged.

Lane on 2026-05-22 post-fix:
  dev/v1:    all_pass_rate=1.0, overall_pass=True (5 cases)
  public/v1: all_pass_rate=1.0, overall_pass=True (20 cases)

- tests/test_correction_cue_prefix_routing.py pins both halves of the
  guard (10 new tests).
- evals/inference_closure/gaps.md documents the regression + fix in a
  new section, preserving the 2026-05-17 resolution narrative.
- evals/inference_closure/results/ now carries canonical v1_dev and
  v1_public reports (the lane had no checked-in results before; ADR-0110
  will reference these).

This unblocks the second of ADR-0107's two named blockers. ADR-0110
(math expert-demo re-attempt) now becomes feasible once the math
domain's three lanes have signed-and-digested evidence.
2026-05-22 12:33:56 -07:00
Shay
36053317be
feat(capability): implement ADR-0109 lane-shape-aware thresholds (#116)
Replaces the cognition-shape-uniform threshold dispatch in
core/capability/expert_demo.py with an explicit LANE_SHAPE_REGISTRY
mapping 8 ratified lane ids to 5 shapes:

  cognition           -> cognition_shape
  elementary_math_ood -> accuracy_shape
  foundational_physics_ood -> accuracy_shape
  symbolic_logic      -> symbolic_logic_shape
  hebrew_fluency      -> accuracy_shape
  koine_greek_fluency -> accuracy_shape
  inference_closure   -> inference_shape
  fabrication_control -> refusal_shape

Each shape has a documented threshold checker. Unknown lane ids
fail-closed with a named reason. ADR-0106 \xc2\xa71.1/\xc2\xa71.3/\xc2\xa71.4/\xc2\xa71.5
unchanged; only \xc2\xa71.2 (threshold rules) dispatches by shape.

tests/test_lane_shape_thresholds.py pins all four ADR-0109 invariants
plus dead-shape and threshold-value gates (13 new tests).
tests/test_expert_demo_contract.py fixtures updated to provide
shape-appropriate metrics (no semantic change to those tests; same
12 cases still pin the ADR-0106 contract).

ADR-0109 status: Proposed -> Accepted. README sequencing updated
(ADR-0110 now only blocked by inference_closure, not by metric-shape
amendment).

Ledger: all five domains remain reasoning-capable, expert_demo=false.
2026-05-22 12:11:58 -07:00
Shay
c2dd69611b
docs: propose ADR-0109 lane-shape-aware threshold amendment (#115)
Amends ADR-0106 \xc2\xa71.2 to dispatch threshold rules by lane shape rather
than imposing cognition-pack-shape metrics uniformly. ADR-0107
surfaced that every non-cognition lane was failing the gate by
absence-of-key, not by substance.

Status: Proposed. Ships five shapes covering every lane currently
attached to a ratified pack: cognition_shape, accuracy_shape,
inference_shape, refusal_shape, symbolic_logic_shape. Four invariants
pinned. Unknown lanes fail closed; new shapes require ADR amendment.

\xc2\xa71.1 (reasoning-capable prereq), \xc2\xa71.3 (signature scoping), \xc2\xa71.4
(domain-aware), \xc2\xa71.5 (replay byte-equality) all preserved. ADR-0106
status remains Accepted.
2026-05-22 12:00:59 -07:00
Shay
7cc2f7b422
feat(adr): ADR-0107 mathematics_logic expert-demo promotion deferred (#114)
The ADR-0106 contract correctly refused promotion. ADR-0107 records the
deferral and reserves two follow-up ADRs:

- ADR-0109 (lane-shape-aware threshold amendment): ADR-0106 \xc2\xa71.2
  prescribes cognition-pack-shape metrics uniformly, but math /
  physics / systems / hebrew-greek lanes carry native shapes
  (accuracy, passed_rate, all_pass_rate). Prerequisite for any future
  expert-demo promotion.
- ADR-0110 (math re-attempt): conditional on ADR-0109 landing and
  inference_closure substantively passing (currently all_pass_rate=0.4
  on public).

tests/test_adr_0107_deferral.py pins adr_0107_no_silent_promotion: math
stays at reasoning-capable, has no expert_demo_claims entry, and the
ledger row carries a named refusal reason.

No change to core/capability/expert_demo.py or reporting.py -- the
contract is honored, not amended. README sequencing updated to reflect
ADR-0107 acceptance and the new ADR-0109/0110 prerequisites.
2026-05-22 11:49:37 -07:00
Shay
0493808215
feat(capability): implement ADR-0106 expert-demo promotion contract (#113)
Closes ADR-0106 acceptance evidence:

- ExpertDemoClaim dataclass + additive expert_demo_claims block on
  ReviewerRegistry (schema_version stays at 1; backward-compatible).
- New core/capability/expert_demo.py with derive_evidence_digest,
  evaluate_expert_demo, collect_domain_lanes, materialise_lane_results.
- core/capability/reporting.py: replaces the cognition-lane-only
  predicate (previous lines 418-433) with a domain-aware,
  reviewer-signed gate; ledger rows now also carry
  expert_demo_reason for operator legibility. Reviewer registry is
  fail-closed: an unloadable registry yields zero claims, so a broken
  registry never silently grants expert_demo=true.
- tests/test_expert_demo_contract.py covers all three ADR-0106
  invariants: requires_signature, domain_aware, replay_byte_equality;
  plus threshold + production-ledger-untouched gates. 12 new tests.
- tests/test_reviewer_registry.py extended with TestExpertDemoClaimsSchema
  covering omitted block, valid parse, unknown signer rejection,
  malformed digest rejection, duplicate domain rejection. 5 new tests.
- README index row + table preface updated to note expert_demo is
  contract-gated. Frontier list trimmed (ADR-0106 has landed).
- ADR-0106 Status flipped Proposed -> Accepted.

No domain row's expert_demo field flips by this PR -- only the contract
changes. Promotion of any ratified domain requires a follow-up ADR
(ADR-0107 reserved for mathematics_logic) plus a signed claim.
2026-05-22 11:39:09 -07:00
Shay
7bb8334fc5
docs: accept ADR-0108 and rewrite Current frontier per its acceptance evidence (#112)
- ADR-0108 Status: Proposed -> Accepted
- README index row updated to Accepted
- 'Current frontier' rewritten with the ranked Proposed-ADR list mandated
  by ADR-0108 \xc2\xa7Decision; removes the now-false 'No ADR currently sits in
  a "Proposed but unimplemented" state' sentence
- Open candidate directions (no-ADR-yet) section retained for the
  multi-reviewer governance frontier item from ADR-0105
2026-05-22 11:21:20 -07:00
Shay
7e50738034
docs: propose ADR-0108 sequencing for currently-Proposed ADRs (#111)
Makes the post-ADR-0105 sequencing of ADR-0080 / 0084 / 0087 / 0106
explicit, durable, and revisable. Status: Proposed. No content of the
four sequenced ADRs is modified — sequencing is meta, not content.
2026-05-22 10:52:44 -07:00
Shay
470ebae38e
chore: ignore .claude/ (Claude Code local session artifacts) (#110) 2026-05-22 10:44:39 -07:00
Shay
8e749546b2
docs: propose ADR-0106 expert-demo promotion contract (#109)
Defines a domain-aware, reviewer-signed expert_demo promotion gate to
replace the current cognition-lane-only predicate in
core/capability/reporting.py:418. Status: Proposed. This ADR does not
promote any domain — it defines the contract that a follow-up ADR (likely
mathematics_logic as ADR-0107) will consume as the first worked
promotion.
2026-05-22 10:44:36 -07:00
Shay
257fd4503d
feat(evals): ADR-0105 — sealed holdout encryption via age (#108)
* feat(evals): add pyrage dependency

* feat(evals): add sealed holdout path resolution

* feat(evals): implement sealed holdout decryption

* feat(evals): add sealed holdout CLI

* test(evals): add sealed holdout encryption tests

* docs(decisions): add ADR-0105 sealed holdout encryption

* feat(evals): route holdout split through sealed decryptor

* docs(decisions): add ADR-0105 index entry

* chore: restore project description

* fix(evals): use pyrage Identity.from_str and pin curriculum SHA

- holdout_runner: pyrage exposes Identity.from_str, not from_file; parse
  identity file by line and pass list[Identity] into decrypt(). Restores
  PR 108's sealed-holdout test suite to green.
- verify_lane_shas: realign curriculum_loop_closure pin with the actual
  deterministic runner output (carryover from PR 107).
2026-05-22 10:09:43 -07:00
Shay
f7680e96ea
feat(teaching): ADR-0104 — curriculum-sourced teaching proposals (#107)
* feat(teaching): add curriculum-sourced proposal builder

* test(teaching): cover curriculum proposal construction

* test(evals): add curriculum loop closure contract

* test(evals): add curriculum loop closure runner

* test(evals): add canonical curriculum loop closure report

* ci(lanes): pin curriculum loop closure lane

* docs(adr): add ADR-0104 curriculum sourced proposals

* docs(adr): register ADR-0104 and seven pinned lanes

* docs(teaching): mark curriculum source activation

* fix(ci): pin curriculum_loop_closure SHA to runner output

* fix(ci): register curriculum_loop_closure in CLAIMS.md generator
2026-05-22 10:05:14 -07:00
Shay
1395ec1354
feat(packs): ADR-0103 — attach hebrew_fluency + koine_greek_fluency lanes to ADR-0102 (#106)
* feat(evals): add Hebrew fluency holdout cases

* feat(evals): add Koine Greek fluency holdout cases

* feat(packs): attach fluency lanes to he_core_cognition_v1

* feat(packs): attach fluency lanes to he_logos_micro_v1

* feat(packs): attach fluency lanes to grc_logos_cognition_v1

* feat(packs): ADR-0103 fluency lane attachment

* test(packs): expect ADR-0103 fluency lanes on Hebrew Greek contracts

* docs(evals): add Hebrew fluency holdout split note

* docs(evals): add Koine Greek fluency holdout split note

* docs(evals): note Hebrew holdout attachment

* docs(evals): note Koine Greek holdout attachment

* docs: add ADR 0103 placeholder

* docs(adr): expand ADR-0103 fluency lane attachment

* docs: index ADR-0103 and refresh frontier
2026-05-22 09:43:46 -07:00
Shay
c07b967454
merge: ADR status reconciliation — mark 0094/0095/0098/0099 Accepted
docs: mark ADR-0094/0095/0098/0099 accepted (sibling reconciliation to #104)
2026-05-22 07:12:30 -07:00
Shay
57fa078aaa docs: mark ADR-0094/0095/0098/0099 accepted
Sibling reconciliation PR to #104. The four ADRs explicitly called out as
the 'current implementation frontier' in PR #104 are already implemented
to the same evidence bar as the eight ADRs that PR accepted:

- ADR-0094: teaching/source.py + proposal schema widening + migration
  script; tests/test_proposal_source.py green
- ADR-0095: teaching/from_miner.py + miner_loop_closure lane;
  SHA-pinned in scripts/verify_lane_shas.py; tests/test_miner_proposals.py
  green
- ADR-0098: core/demos/contract.py + adapter surface + demo_composition
  lane; SHA-pinned; tests/test_demo_composition.py green
- ADR-0099: core/demos/showcase.py + public_demo lane;
  SHA-pinned; tests/test_public_showcase.py green

Three of four lanes are SHA-pinned in CI (a stricter bar than several
already-accepted ADRs). Local pytest run: 85/85 passed across the four
tests/test_*.py files in 17s.

Also refreshes docs/decisions/README.md:
- flips the four table rows to Accepted (2026-05-22)
- rewrites the 'Current frontier' section now that no ADR-0091..0102
  entry is unimplemented
- enumerates candidate next directions (curriculum proposals,
  language-specific holdout splits, expert-demo ratification)

Docs-only change; no runtime code touched.
2026-05-22 07:09:14 -07:00