core/teaching/promotion.py
Shay 84e74eede8 feat(teaching): discovery gaps aggregator + auto-promotion queue (Phase 1.1+1.2)
Closes the corpus flywheel.  ADR-0055 Phase B emits DiscoveryCandidate
JSONL to the discovery sink, but until now there was no operator-facing
view: candidates accumulated to disk, no one grepped them, the system's
"I would have grounded this if I had a chain" signal went into a void.

P1.1 — Discovery aggregator (teaching/gaps.py).

  Pure reader over the discovery-sink monthly-rollover layout
  (<root>/<YYYY>/<YYYY-MM>.jsonl).  aggregate_gaps(root, since,
  sample_limit) groups emitted candidates by (subject, intent) cell
  and returns a deterministic ranked tuple of Gap records.

  - count: total emissions
  - boundary_clean_count: subset whose boundary_clean flag held
    (refusal/hedge-tainted emissions split out so operators can filter)
  - sample_candidate_ids: up to N retained ids per cell, sorted
  - months_seen: every month token where the cell appeared

  --since YYYY-MM filters by file naming convention (no timestamp
  dependency).  Malformed lines silently skipped.  Default root:
  teaching/discovery_log.

  CLI: core teaching gaps [--root PATH] [--since YYYY-MM] [--top N]
                          [--sample-limit N] [--json]

P1.2 — Auto-promotion queue (teaching/promotion.py).

  promote_gaps(gaps, threshold, include_tainted) lifts cells whose
  effective count meets the threshold into GapPromotion records.

  - Default mode: boundary_clean_count gates promotion.  Tainted-only
    cells (count > 0 but all emissions refusal/hedge-tainted) do not
    auto-promote — those may indicate the prompt hit a safety axis,
    not a curriculum gap.
  - include_tainted=True counts every emission (operator override).
  - Threshold must be >= 1 (zero threshold defeats the queue).
  - queue_id is stable + deterministic (gap:<intent>:<subject>@<N>).
  - No content synthesis — promotion never invents connective or
    object; only an operator can author a complete chain via the
    propose/replay/accept pipeline.

  CLI: core teaching queue [--threshold N] [--include-tainted]
                           [--root PATH] [--since YYYY-MM] [--json]

Operator workflow (closed loop):

  operator → core chat                            # asks question
           ← cold turn emits DiscoveryCandidate
  operator → core teaching gaps --top 10          # ranked gaps
  operator → core teaching queue --threshold 3    # auto-promoted
  operator → authors candidate JSONL
  operator → core teaching propose <path>         # replay gate runs
  operator → core teaching review <id> --accept   # corpus mutates

24 new tests (13 gaps + 11 promotion), all pure / no I/O dependencies,
fast (<1s combined).  Full lane: 1933 passed, 2 skipped.
2026-05-18 16:04:39 -07:00

132 lines
4.8 KiB
Python

"""teaching/promotion.py — Phase 1.2: auto-promote high-frequency
discovery cells into an operator-visible review queue.
The discovery aggregator (:mod:`teaching.gaps`) ranks
``(subject, intent)`` cells by how many DiscoveryCandidate emissions
they accumulated. ADR-0055 emits structured evidence; the
aggregator surfaces frequency; **this module closes the loop**:
when a cell's emission count crosses a threshold, it becomes a
:class:`GapPromotion` — an explicit "author a chain for me" signal
that operators can act on without grepping raw aggregation tables.
Design constraints:
- **Pure function of the aggregator output.** Promotion is derived
state — no separate persistent queue, no double-bookkeeping.
Re-running ``promote_gaps`` on the same sink contents produces
the same result deterministically.
- **Threshold is explicit.** No magic defaults. Operators pick
a threshold appropriate to their workload (3 is the v1 baseline:
"three independent cold-start prompts hit this cell; author the
chain").
- **No content synthesis.** A promotion records that authorship is
*needed*; it never invents connective or object. Only an
operator can author a complete chain — the trust boundary that
prevents stochastic chain-generation is preserved.
- **Boundary-clean filter on by default.** A gap whose only
contributing candidates carry ``boundary_clean=False`` (refusal-
or hedge-tainted) does not promote, since those signals may
indicate the prompt itself violated a safety/ethics axis rather
than a curriculum gap. The operator can opt in to including
tainted cells via ``include_tainted=True``.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Iterable
from teaching.gaps import Gap
@dataclass(frozen=True, slots=True)
class GapPromotion:
"""A ``(subject, intent)`` cell whose emission count met the
auto-promotion threshold.
Fields mirror :class:`teaching.gaps.Gap` for traceability, plus
``threshold`` so the operator can see *why* this cell promoted
(a count of 7 at threshold 3 reads differently than 7 at 7).
"""
subject: str
intent: str
count: int
boundary_clean_count: int
sample_candidate_ids: tuple[str, ...]
months_seen: tuple[str, ...]
threshold: int
@property
def queue_id(self) -> str:
"""Stable, deterministic identifier for this cell promotion.
The same cell at the same threshold always produces the same
``queue_id`` so operators can diff queue states across
invocations.
"""
return f"gap:{self.intent}:{self.subject}@{self.threshold}"
def as_dict(self) -> dict[str, object]:
return {
"queue_id": self.queue_id,
"subject": self.subject,
"intent": self.intent,
"count": self.count,
"boundary_clean_count": self.boundary_clean_count,
"sample_candidate_ids": list(self.sample_candidate_ids),
"months_seen": list(self.months_seen),
"threshold": self.threshold,
}
def promote_gaps(
gaps: Iterable[Gap],
*,
threshold: int = 3,
include_tainted: bool = False,
) -> tuple[GapPromotion, ...]:
"""Return the subset of *gaps* whose effective count meets *threshold*.
Effective count:
- When ``include_tainted=True`` (default False), the comparison
uses :attr:`Gap.count` (every emission counts).
- When ``include_tainted=False`` (default), the comparison uses
:attr:`Gap.boundary_clean_count` (refusal/hedge-tainted
emissions are excluded from the promotion decision).
Threshold must be ``>= 1``; a threshold of zero would promote
every observed cell and defeats the purpose of a priority queue.
"""
if threshold < 1:
raise ValueError(f"threshold must be >= 1 (got {threshold!r})")
promoted: list[GapPromotion] = []
for gap in gaps:
effective_count = gap.count if include_tainted else gap.boundary_clean_count
if effective_count < threshold:
continue
promoted.append(
GapPromotion(
subject=gap.subject,
intent=gap.intent,
count=gap.count,
boundary_clean_count=gap.boundary_clean_count,
sample_candidate_ids=gap.sample_candidate_ids,
months_seen=gap.months_seen,
threshold=threshold,
)
)
# Deterministic order: highest effective count first, ties broken
# by subject then intent — mirrors :func:`aggregate_gaps`.
promoted.sort(
key=lambda p: (
-(p.count if include_tainted else p.boundary_clean_count),
p.subject,
p.intent,
)
)
return tuple(promoted)
__all__ = ["GapPromotion", "promote_gaps"]