core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
core/docs/admissibility-exemplars.md
Shay 1bff5689db
feat(ADR-0163.B.1): exemplar corpora — descriptive_setup_no_quantity, temporal_aggregation, rate_with_currency (#298) 
Round 1 of ADR-0163 Phase B: hand-author seed exemplars for the top three
refusal shape categories surfaced by the Phase A histogram. These corpora
are INPUT to the Phase C contemplation runner, which will derive
DerivedRecognizer proposals from them; this PR ships no recognizer logic,
no proposal logging, and no runtime change.

Per-category breakdown:
- descriptive_setup_no_quantity_v1.jsonl — 20 exemplars (5 train + 12 novel + 3 edge)
- temporal_aggregation_v1.jsonl          — 20 exemplars (4 train + 13 novel + 3 edge)
- rate_with_currency_v1.jsonl            — 20 exemplars (3 train + 14 novel + 3 edge)

Train-sample citations resolve against
evals/gsm8k_math/train_sample/v1/report.json (the 50-case sample only;
public/holdout/full splits NOT mined per ADR-0163 §Constraints).

Each file is sorted by exemplar_id, byte-canonical, and disjoint from the
others. Statements are surface-preserved verbatim from the train sample
where cited.

Validation:
- tests/test_admissibility_exemplars.py: 20/20 passed (schema, enum
  binding, per-category quantity_anchor dispatch, cross-file disjointness,
  >=3 train-sample citations per category, sort/byte-canonical determinism,
  read-only import invariant)
- tests/test_adr_0131_*.py: 224 passed / 3 skipped — capability axes
  G1..G5 + S1 remain wrong=0
- core test --suite smoke: 67 passed
- core eval refusal_taxonomy: case_digest unchanged
  (d030f826cb0f4088771d90c52c8be2ff75054ab27c7d47eae8dbfe1225b2eea1)
- Phase A categorize() agrees with the file's category for all 60
  statements (sanity check; not pinned in tests since the rules-only
  categorizer is coarser than the recognizer Phase C will derive)

Author notes on quantity_anchor annotation calls flagged for operator
review are embedded in provenance.author_note where ambiguous (notably:
'in N minutes' / 'over N hours' window framings collapsed to
window_quantifier='per', 'every other day' approximated as 'every',
day-of-week labels not captured in the schema, 'for one X' / slash-form
per-unit framings, non-USD currencies, and discrete-occurrence per_unit
values like 'event' and 'session').

Refs: ADR-0163 §Phase B; depends on the Phase A lane shipped in #297.
Cross-refs: ADR-0057 (proposal review), ADR-0149/0154 (recognizer
pipeline), ADR-0161 (HITL queue), [[thesis-decoding-not-generating]].
2026-05-26 11:52:23 -07:00

2.6 KiB
Raw Blame History

Admissibility Exemplars (ADR-0163 Phase B)

Operator-facing overview of the hand-authored exemplar corpora that feed the Phase C contemplation runner. For the full schema, sourcing rules, and forward reference, see teaching/admissibility_exemplars/contract.md.

What this is

For each shape category surfaced by the Phase A refusal-taxonomy lane, the operator hand-authors a small JSONL corpus of canonical exemplars. Phase C (contemplation runner) ingests these corpora and emits recognizer proposals; Phase D ratifies; Phase E re-baselines GSM8K.

Phase B is the only phase where the engine learns from operator-authored statements. Every dimension of "what shape did the operator think was canonical?" propagates into the recognizer Phase C derives and the gates Phase D ratifies. Therefore: canonical over comprehensive, surface preservation over normalization, distinguishing over similar.

Round 1 — categories and counts

The Phase A histogram (evals/refusal_taxonomy/v1/report.json) selected these three categories:

Rank Category Phase A count Exemplars (this round)
1 descriptive_setup_no_quantity 17 20
2 temporal_aggregation 4 20
3 rate_with_currency 3 20

Total: 60 hand-authored exemplars across three files in teaching/admissibility_exemplars/.

Per-category breakdown of sourcing:

Category Train-sample citations Novel (operator-authored) Edge cases
descriptive_setup_no_quantity 5 12 3
temporal_aggregation 4 13 3
rate_with_currency 3 14 3

(Edge cases overlap with novel; counts above split them out.)

Files

Cross-references