core/docs/decisions/ADR-0130-pre-articulation-calibration-deferred.md

ADR-0130 — Pre-Articulation Calibration Logging (Deferred Proposal)

Status: Proposed — Deferred (backlog item; no implementation scheduled until the GSM8K-math substrate arc through ADR-0127 / ADR-0128 resolves Path-A vs Path-B) Date: 2026-05-23 Author: CORE agents + reviewers Depends on: ADR-0035 (turn-loop verdicts), ADR-0036 (safety refusal), ADR-0040 (telemetry sink), ADR-0043 (pack measurements phase 2), ADR-0059 (correction-pass telemetry) Supersedes: none

---

Context

The same research review that motivated ADR-0129 (docs/sessions/SESSION-2026-05-23-pedagogy-research-and-teaching-loop-pivot.md) surfaced a second pedagogy finding with strong empirical support and a clean structural mapping: metacognitive calibration via prediction-outcome comparison.

In human pedagogy: learners predict their performance before a task, compare prediction to outcome, and use the gap to recalibrate their judgments of learning. Repeated calibration cycles shrink the gap (Bjork, Dunlosky, Koriat, and successors). The mechanism is that uncalibrated confidence is the central failure mode of self-regulated learning — over-confidence leads to under-study; under-confidence to over-study; both waste capacity.

CORE has analogs at the runtime layer:

• ADR-0035 end-of-turn safety + ethics verdicts
• ADR-0036 typed safety refusal
• ADR-0040 structured telemetry sink

But CORE has no analog at the teaching layer. Before a reviewed correction lands, no event captures the answer CORE would have produced on the case-pre-correction; after correction, no event captures the gap.

This is a genuine information loss. Unlike a human learner, CORE can do this without subjective bias: the pre-correction answer is deterministic and exactly recordable. The gap is a real measurement, not a self-report.

---

Decision (proposed; deferred)

Add a deterministic "pre-correction prediction" capture step to the teaching subsystem. When a correction is proposed (before review), record CORE's current pipeline output on the case as a prediction event. When the correction is accepted (review passes, correction lands in store), emit a paired calibration event recording the delta between prediction and the corrected outcome.

Proposed shape (non-binding sketch — implementation defers)

• Pre-correction prediction event (emitted at correction proposal):

  {
    "type": "pre_correction_prediction",
    "correction_proposal_id": "...",
    "input_digest": "<sha256-of-input>",
    "predicted_output_digest": "<sha256>",
    "predicted_trace_hash": "...",
    "predicted_verdict": "correct | wrong | refused",
    "current_runtime_state_digest": "<sha256-of-pack-versions+correction-store>"
  }
  

• Post-correction calibration event (emitted at correction acceptance):

  {
    "type": "post_correction_calibration",
    "correction_id": "...",
    "linked_prediction_id": "...",
    "predicted_output_digest": "<sha256>",
    "corrected_output_digest": "<sha256>",
    "delta_class":
      "no_change | answer_value | answer_unit | trace_only | refused_to_correct | correct_to_refused",
    "pack_provenance_diff": [...]
  }
  

• Aggregation (offline, periodic): a calibration_report.json in the teaching subsystem's reports directory, summarizing:
  • rate at which predictions matched corrections (no-change),
  • distribution of delta classes,
  • per-pack-version cohort comparisons (does calibration improve after pack ratifications?).
• No runtime gating. The prediction is observational. It does NOT alter what the operator can or cannot do; it does NOT veto any correction. It's measurement, not control.

Invariants

Invariant	Status
wrong == 0	Preserved — prediction is observational
Trace determinism	Preserved — prediction uses standard pipeline
No unreviewed mutation	Preserved — prediction does not write to correction store
Reviewed teaching only	Preserved — calibration emits only at proposal AND acceptance, both of which are operator-mediated
Telemetry redaction defaults	Preserved — input digests, not raw input
versor_condition(F) < 1e-6	Untouched

What this enables that's not currently possible

1. Empirical answer to "is CORE actually getting better?" Per-pack- version calibration trends would show whether ratifications improve pre-correction accuracy or just shift the surface.
2. Audit story strengthens. Today operators see that corrections happen; they don't see how often CORE was already right before the correction. The calibration gap is exactly that signal.
3. Misconfigured-pack detection. A pack version that suddenly spikes pre-correction error rate (vs the prior pack's rate) is a flag worth surfacing automatically.
4. Honest framing of operator workload. If the calibration shows pre-correction prediction matches the eventual correction 95% of the time, the operator review can be lighter-touch on that pack; if 5%, heavier-touch is warranted.

Why this is deferred, not accepted

1. Path-B uncertainty (same as ADR-0129): the GSM8K-math arc may produce a different correction-store population structure that changes the right calibration cohorts.
2. No measured calibration problem. We don't currently have evidence that pre-correction accuracy is misaligned with post-correction. The proposal is "measure to find out" — but the cost of building the measurement infrastructure should match the prior of finding something. We don't have a strong prior.
3. Telemetry already substantial. ADR-0040 / ADR-0042 / ADR-0043 ship significant telemetry. Adding two new event classes raises operator-noise floor; should only do so if the signal proves worth it.
4. Operator workload concern. Even though prediction is observational, a calibration report is a thing the operator has to read. More artifacts means more attention budget; only worth it if the artifacts surface decisions.

Exit criteria for un-deferral

This ADR becomes a candidate for acceptance if any of:

1. An incident occurs where a correction was applied unnecessarily (CORE was already producing the right answer on that input) AND the wasted review effort would have been visible to a calibration-event sequence.
2. A pack ratification produces unexpected behavior whose detection would have been faster via per-pack calibration cohort comparison.
3. The teaching corpus grows to where operator review bandwidth becomes a bottleneck and routing reviews by calibration confidence would help triage.
4. The GSM8K-math arc resolves and ADR-0129 (spaced replay) is un-deferred — at which point these two capabilities should compose, since spaced replay events naturally produce calibration evidence and they should share infrastructure.

Alternatives considered

A. Build calibration logging now, defer reporting.

Considered. Logging without reporting still costs telemetry volume; without a report nobody reads the events; without reading the events the log is decoration. Rejected per CLAUDE.md "no decoration without integration."

B. Sample-based calibration (log a random 10% of proposals).

Considered. Determinism doctrine pushes against sampling — same correction proposal should always produce same calibration event, or none at all. Could be acceptable if sampling is content-keyed (hash of input → bucket) so it's deterministic, but adds complexity. Defer for now.

C. Manual calibration audit on demand.

The CLI could provide core teaching calibration --window N that re-runs the last N corrections through the prediction path offline and produces a one-shot calibration report. Lower implementation cost than continuous logging; could be a useful half-step. Worth its own short ADR if any of the exit criteria above fire.

Composition with ADR-0129

If both ADRs are eventually un-deferred, they should share infrastructure:

• Spaced-replay events (ADR-0129) naturally yield calibration evidence: each replay produces a prediction against the original correction's expected outcome. The two event streams should merge into a single calibration report.
• A correction whose spaced-replay events show repeated divergence is a stronger signal than either system alone would catch.

This composition is itself an argument for un-deferring both together if either is un-deferred.

PR checklist (if/when proposed for acceptance)

What capability did this add?
  → Deterministic measurement of pre-correction prediction
    accuracy; empirical signal for "is CORE getting better."
What invariant proves the field remains valid?
  → wrong == 0 (prediction is observational); trace determinism
    (standard pipeline); no unreviewed mutation (calibration
    writes events, not corrections).
Which CLI suite/eval proves the lane?
  → New `core test --suite teaching-calibration` lane; fixture
    correction-proposal sequence asserts deterministic event
    pairs and report aggregation.
Did this avoid hidden normalization, stochastic fallback,
approximate recall, unreviewed mutation?
  → Yes. Pure observational, deterministic pipeline call.
If it touches user input, what trust boundary was enforced?
  → Telemetry emits input digests (SHA-256), not raw input,
    consistent with ADR-0040's redact-by-default policy.