1bff5689db
Round 1 of ADR-0163 Phase B: hand-author seed exemplars for the top three refusal shape categories surfaced by the Phase A histogram. These corpora are INPUT to the Phase C contemplation runner, which will derive DerivedRecognizer proposals from them; this PR ships no recognizer logic, no proposal logging, and no runtime change. Per-category breakdown: - descriptive_setup_no_quantity_v1.jsonl — 20 exemplars (5 train + 12 novel + 3 edge) - temporal_aggregation_v1.jsonl — 20 exemplars (4 train + 13 novel + 3 edge) - rate_with_currency_v1.jsonl — 20 exemplars (3 train + 14 novel + 3 edge) Train-sample citations resolve against evals/gsm8k_math/train_sample/v1/report.json (the 50-case sample only; public/holdout/full splits NOT mined per ADR-0163 §Constraints). Each file is sorted by exemplar_id, byte-canonical, and disjoint from the others. Statements are surface-preserved verbatim from the train sample where cited. Validation: - tests/test_admissibility_exemplars.py: 20/20 passed (schema, enum binding, per-category quantity_anchor dispatch, cross-file disjointness, >=3 train-sample citations per category, sort/byte-canonical determinism, read-only import invariant) - tests/test_adr_0131_*.py: 224 passed / 3 skipped — capability axes G1..G5 + S1 remain wrong=0 - core test --suite smoke: 67 passed - core eval refusal_taxonomy: case_digest unchanged (d030f826cb0f4088771d90c52c8be2ff75054ab27c7d47eae8dbfe1225b2eea1) - Phase A categorize() agrees with the file's category for all 60 statements (sanity check; not pinned in tests since the rules-only categorizer is coarser than the recognizer Phase C will derive) Author notes on quantity_anchor annotation calls flagged for operator review are embedded in provenance.author_note where ambiguous (notably: 'in N minutes' / 'over N hours' window framings collapsed to window_quantifier='per', 'every other day' approximated as 'every', day-of-week labels not captured in the schema, 'for one X' / slash-form per-unit framings, non-USD currencies, and discrete-occurrence per_unit values like 'event' and 'session'). Refs: ADR-0163 §Phase B; depends on the Phase A lane shipped in #297. Cross-refs: ADR-0057 (proposal review), ADR-0149/0154 (recognizer pipeline), ADR-0161 (HITL queue), [[thesis-decoding-not-generating]].
57 lines
2.6 KiB
Markdown
57 lines
2.6 KiB
Markdown
# Admissibility Exemplars (ADR-0163 Phase B)
|
|
|
|
Operator-facing overview of the hand-authored exemplar corpora that feed the
|
|
Phase C contemplation runner. For the full schema, sourcing rules, and
|
|
forward reference, see
|
|
[`teaching/admissibility_exemplars/contract.md`](../teaching/admissibility_exemplars/contract.md).
|
|
|
|
## What this is
|
|
|
|
For each shape category surfaced by the Phase A refusal-taxonomy lane, the
|
|
operator hand-authors a small JSONL corpus of canonical exemplars. Phase C
|
|
(contemplation runner) ingests these corpora and emits recognizer proposals;
|
|
Phase D ratifies; Phase E re-baselines GSM8K.
|
|
|
|
Phase B is the **only** phase where the engine learns from operator-authored
|
|
statements. Every dimension of "what shape did the operator think was
|
|
canonical?" propagates into the recognizer Phase C derives and the gates
|
|
Phase D ratifies. Therefore: canonical over comprehensive, surface
|
|
preservation over normalization, distinguishing over similar.
|
|
|
|
## Round 1 — categories and counts
|
|
|
|
The Phase A histogram (`evals/refusal_taxonomy/v1/report.json`) selected
|
|
these three categories:
|
|
|
|
| Rank | Category | Phase A count | Exemplars (this round) |
|
|
|---|---|---|---|
|
|
| 1 | `descriptive_setup_no_quantity` | 17 | 20 |
|
|
| 2 | `temporal_aggregation` | 4 | 20 |
|
|
| 3 | `rate_with_currency` | 3 | 20 |
|
|
|
|
Total: **60 hand-authored exemplars** across three files in
|
|
`teaching/admissibility_exemplars/`.
|
|
|
|
Per-category breakdown of sourcing:
|
|
|
|
| Category | Train-sample citations | Novel (operator-authored) | Edge cases |
|
|
|---|---|---|---|
|
|
| `descriptive_setup_no_quantity` | 5 | 12 | 3 |
|
|
| `temporal_aggregation` | 4 | 13 | 3 |
|
|
| `rate_with_currency` | 3 | 14 | 3 |
|
|
|
|
(Edge cases overlap with novel; counts above split them out.)
|
|
|
|
## Files
|
|
|
|
- [`teaching/admissibility_exemplars/descriptive_setup_no_quantity_v1.jsonl`](../teaching/admissibility_exemplars/descriptive_setup_no_quantity_v1.jsonl)
|
|
- [`teaching/admissibility_exemplars/temporal_aggregation_v1.jsonl`](../teaching/admissibility_exemplars/temporal_aggregation_v1.jsonl)
|
|
- [`teaching/admissibility_exemplars/rate_with_currency_v1.jsonl`](../teaching/admissibility_exemplars/rate_with_currency_v1.jsonl)
|
|
- [`teaching/admissibility_exemplars/contract.md`](../teaching/admissibility_exemplars/contract.md)
|
|
|
|
## Cross-references
|
|
|
|
- [ADR-0163 — Path to GSM8K mastery](decisions/ADR-0163-gsm8k-path-to-mastery.md)
|
|
- [Phase A refusal taxonomy contract](../evals/refusal_taxonomy/contract.md)
|
|
- [ADR-0057 — Proposal review + replay-equivalence](decisions/ADR-0057-teaching-chain-proposal-review.md)
|
|
- [ADR-0161 — HITL async queue](decisions/ADR-0161-hitl-async-queue.md)
|