feat(ADR-0163.B.2): four new exemplar corpora — discrete_count_statement, multiplicative_aggregation, currency_amount, plus temporal_aggregation v2 widening (#306 )

Phase B round 2.  Categorizing the post-#304 GSM8K train_sample's
still-refused 47 set surfaced three coherent sub-shapes in the previously
UNCATEGORIZED tail plus five ratified-but-narrowness-blocked temporal
cases; this PR ships the operator-authored exemplar seeds + Phase A
categorizer extension that prove the corridor scales beyond round 1.

Exemplar corpora (70 new exemplars across 4 files):
- discrete_count_statement_v1.jsonl (20)
- multiplicative_aggregation_v1.jsonl (20)
- currency_amount_v1.jsonl (20)
- temporal_aggregation_v2.jsonl (10, widening)

Each corpus carries ≥3 verbatim train-sample citations, ≥12 (≥5 for v2)
novel operator-authored statements, and ≥1–3 edge cases.  Statements are
disjoint across all 7 round-1 + round-2 corpora; tests enforce.

Phase A categorizer (evals/refusal_taxonomy/shape_categories.py)
extends ShapeCategory with three new members and inserts their rule
predicates AFTER the existing more-specific categories:
- rate_with_currency before currency_amount
- multiplicative_aggregation before discrete_count_statement
Each new rule predicate cites ≥3 train_sample case_ids in its docstring
(ADR-0163 §Risks).  No LLM, no embedding, no learned classifier.

Refusal-taxonomy histogram empirical signal (public 50 sample):
- pre-round-2: 14 UNCATEGORIZED (categorized_rate 0.72)
- post-round-2: 1 UNCATEGORIZED (categorized_rate 0.98)

The single residual is case 0044 ("10% simple interest" — percentage
without change verb), an honest tail outside the three round-2 shapes.

wrong=0 holds on capability axes G1..G5 + S1; no runtime code shipped.
Smoke suite green (67/67).

Cross-refs: ADR-0163, #297 (Phase A), #298 (Phase B round 1),
#301 (Phase C), #302 (Phase D), #304 (round-1 ratify), #305 (session
recap).

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

2026-05-26 14:36:59 -07:00

4.3 KiB

Raw Permalink Blame History

Admissibility Exemplars (ADR-0163 Phase B)

Operator-facing overview of the hand-authored exemplar corpora that feed the Phase C contemplation runner. For the full schema, sourcing rules, and forward reference, see teaching/admissibility_exemplars/contract.md.

What this is

For each shape category surfaced by the Phase A refusal-taxonomy lane, the operator hand-authors a small JSONL corpus of canonical exemplars. Phase C (contemplation runner) ingests these corpora and emits recognizer proposals; Phase D ratifies; Phase E re-baselines GSM8K.

Phase B is the only phase where the engine learns from operator-authored statements. Every dimension of "what shape did the operator think was canonical?" propagates into the recognizer Phase C derives and the gates Phase D ratifies. Therefore: canonical over comprehensive, surface preservation over normalization, distinguishing over similar.

Round 1 — categories and counts

The original Phase A histogram (pre-round-2-extension) selected these three categories:

Rank	Category	Phase A count	Exemplars
1	`descriptive_setup_no_quantity`	17	20
2	`temporal_aggregation`	4	20
3	`rate_with_currency`	3	20

Round 1 sourcing breakdown:

Category	Train-sample citations	Novel (operator-authored)	Edge cases
`descriptive_setup_no_quantity`	5	12	3
`temporal_aggregation`	4	13	3
`rate_with_currency`	3	14	3

Round 2 — categories and counts

Round 2 was driven by categorizing the post-#304 GSM8K train_sample still-refused 47 set: 23 had been UNCATEGORIZED under the round-1 categorizer; the categorization sweep surfaced three coherent sub-shapes, plus five ratified-but-unmatched temporal cases that called for a v2 widening.

Rank	Category	Phase A round-2 count (public 50)	Exemplars
—	`discrete_count_statement` (new)	10	20
—	`multiplicative_aggregation` (new)	2	20
—	`currency_amount` (new)	1	20
—	`temporal_aggregation` v2 (widening)	—	10

The "Phase A round-2 count" column is the number of the public 50-case sample now categorized into the new category by the extended Phase A categorizer; it is the empirical signal the categorization actually worked. Pre-round-2 the public sample carried 14 UNCATEGORIZED cases; post-round-2 only 1 remains (case 0044, "10% simple interest" with no change verb — an honest residual outside the three sub-shapes).

Round 2 sourcing breakdown:

Category	Train-sample citations	Novel (operator-authored)	Edge cases
`discrete_count_statement`	6	11	3
`multiplicative_aggregation`	3	13	4
`currency_amount`	3	14	3
`temporal_aggregation` v2	4	5	1

Files

Round 1:

Round 2:

Contract:

teaching/admissibility_exemplars/contract.md

4.3 KiB Raw Permalink Blame History