Mirrors the chain-gap pipeline (Phase 1.1+1.2) for vocabulary gaps:
the OOV invitation surface (P2.1) now emits structured signals that
operators can aggregate, rank, and auto-promote into reviewed
PackMutationProposal candidates — closing the OOV loop the same way
Phase 1 closed the chain loop.
Three new modules + two new CLI surfaces:
teaching/oov_sink.py.
OOVCandidate dataclass mirroring teaching.discovery.DiscoveryCandidate.
OOVBufferSink (in-memory) + OOVMonthlyFileSink (append-only JSONL
under <root>/<YYYY>/<YYYY-MM>.jsonl — same layout as discovery sink
so the aggregator reuses the file-walk machinery).
hash_oov_candidate_id(token, intent, trace_hash) — deterministic
32-char hex id matching DiscoveryCandidate's replay invariant.
format_oov_candidate_jsonl — sorted-keys compact JSONL line.
teaching/oov_gaps.py.
aggregate_oov_gaps(root, since, sample_limit) groups emitted
candidates by token, tracks intent-shape union (a token asked under
multiple intents is a stronger curriculum signal), splits
boundary_clean from boundary_tainted counts, supports --since
YYYY-MM filtering via the sink's file naming convention.
Pure reader; never mutates the sink. Deterministic ordering:
(count desc, token asc).
teaching/oov_promotion.py.
promote_oov_gaps(gaps, threshold, include_tainted, suggested_packs)
lifts threshold-crossing tokens to OOVPromotion records.
- boundary_clean_count gates promotion by default (tainted-only
tokens may indicate the prompt hit a safety axis rather than a
vocab gap).
- --include-tainted flag for operator override.
- threshold < 1 raises.
- queue_id deterministic: ``oov:<token>@<threshold>`` — diffable
across runs.
- suggested_packs lists mounted packs but does NOT recommend one
— domain inference is out of scope (would require a stochastic
classifier). Operator picks the destination.
Runtime wiring:
ChatRuntime.attach_oov_sink(sink) mirrors attach_discovery_sink.
Runtime emits one OOVCandidate JSONL line per turn whose
grounding_source == "oov", no-op when no sink is attached.
Intent classifier is now invoked when EITHER sink is attached
(was: only discovery sink) — both downstream paths need it.
CLI:
core teaching oov-gaps [--top N] [--since YYYY-MM] [--root PATH]
[--sample-limit N] [--json]
core teaching oov-queue [--threshold N] [--include-tainted]
[--root PATH] [--since YYYY-MM] [--json]
ADR-0065 documents the full design (five-tier honesty gradient,
P2.1-P2.4 architecture). README.md updated with the ADR-0065
index entry.
Verification:
tests/test_oov_pipeline.py 24 passed
Operator workflow round-trip verified live:
> rt.attach_oov_sink(sink); rt.chat("What is photosynthesis?")
→ sink receives:
{"boundary_clean":true,"candidate_id":"f51bf8...",
"intent":"definition","token":"photosynthesis","trigger":"unresolved_subject",
"source_turn_trace":"","review_state":"unreviewed"}
> core teaching oov-gaps --root /tmp/oov_demo
→ ranked table by count, intent-set per token
> core teaching oov-queue --root /tmp/oov_demo --threshold 2
→ promoted tokens + suggested mounted packs
Full lane: 2005 passed, 2 skipped, 0 failed in 2:34 (xdist).
Shay