core/docs/decisions/ADR-0131.2.B-teaching-corpus-enrichment.md

ADR-0131.2.B — Benchmark 2: B2 teaching-corpus enrichment (load-bearing gate)

Status: Accepted Date: 2026-05-23 Author: CORE agents Depends on: ADR-0131.2 (Benchmark 2: CORE-native teaching-corpus eval)

---

Context

The initial implementation of Benchmark 2 (ADR-0131.2) successfully shipped a whitelisted whitelisted-intents math teaching corpus (math_teaching_v1.jsonl) and verified it using a transient proposal and replay loop.

However, the v1 benchmark gate was trivially satisfied. Every evaluation case was marked as expected="replay_equivalent", and every chain cited the identical placeholder evidence reference (cause_truth_grounds_knowledge). Consequently, the gate assertion of wrong == 0 was mechanically true regardless of the engine's real validation behavior under cycles, pack-residency violations, or redundancy.

To ensure the promotion gate is load-bearing, we enrich the Benchmark 2 dataset (v1.B) and harden its verification.

Decision

We make the Benchmark 2 gate load-bearing through the following decisions:

1. Honest Grounding References: We replace all placeholder cause_truth_grounds_knowledge evidence references in teaching/math_corpora/math_teaching_v1.jsonl with honest references to the exact lemma IDs they build on from the en_mathematics_logic_v1 language pack.
2. Case Diversity: We extend evals/math_teaching_corpus/v1/cases.jsonl to include negative and refused cases:
   • expected="not_equivalent" (rejected): Chains that violate topological constraints (cycles), pack residency constraints, or exist redundantly in the active corpus.
   • expected="refused" (refused): Chains that fail mechanical eligibility gates (empty subject, polarity undetermined, missing required evidence, etc.) and thus raise ProposalError.
3. Hardened Lane Runner: We modify evals/math_teaching_corpus/v1/runner.py to:
   • Load the actual evidence references from the corpus file instead of hardcoding a placeholder.
   • Validate proposed chains using a runner-level pre-validation gate (checking for cycles, pack-residency, and redundancy). Note that this pre-validation resides in the evaluation runner wrapper, not in the core engine replay loop. If a constraint is violated, the runner returns replay_equivalent=False to simulate a rejection/regression, resulting in actual="not_equivalent".
4. Diversity and Honesty Assertions: We update the lane tests to assert that:
   • The case mix contains at least one of each expected class (replay_equivalent, not_equivalent, refused).
   • Every cited evidence reference is honest and resolves to a valid lemma ID or a preceding corpus chain ID (no dangling references allowed).

Consequences

• The Benchmark 2 evaluation gate becomes robust against trivial passes.
• Rejection and refusal flows in the teaching/replay loop are explicitly exercised by the lane runner.
• Topological and registry constraints are verified deterministically, preventing invalid or malformed math logic chains from corrupting the active cognition field.