docs(ADR-0164.2): pronoun/entity resolution policy (#319)
Proposed sub-ADR under ADR-0164 resolving Open question #3. - Reviews existing _resolve_question_entity heuristic in generate/math_candidate_parser.py: refuse-on-ambiguity is correct, but flat-document whitelist scan misses recency, kinship entities, group antecedents from conjunction, and names absent from the closed name lists. - Specifies EntityRegistry as a field on ProblemReadingState (ADR-0164.3 companion): append-only entries with canonical name, inferred gender + source, mention positions, and relational anchor for kinship entities. - Two refusal-first ambiguity rules: ambiguous_pronoun_referent (R1, recency tiebreaker within RECENCY_GAP_MIN refuses) and unresolved_pronoun (R2). - Worked walk-through on five GSM8K train_sample cases (0001 Tina, 0010 Yun/Marion, 0027 Malcolm, 0017 Jason/Eric, 0033 Rachel + kin). - Three policy-vs-heuristic disagreements (D1 Jason/Eric him; D2 Georgie he via single-salient back-fill; D3 Aaron/Carson they via GROUP entry) all turn refusals into correct resolutions, plus one counter-direction D4 where new policy is principled-conservative. - Preserves wrong = 0 by construction at every branch.
This commit is contained in:
parent
3b8f441ae0
commit
2fcd22c319
1 changed files with 573 additions and 0 deletions
573
docs/decisions/ADR-0164.2-pronoun-entity-resolution.md
Normal file
573
docs/decisions/ADR-0164.2-pronoun-entity-resolution.md
Normal file
|
|
@ -0,0 +1,573 @@
|
|||
# ADR-0164.2 — Pronoun / Entity Resolution Policy
|
||||
|
||||
**Status:** Proposed
|
||||
**Date:** 2026-05-26
|
||||
**Author:** Shay
|
||||
**Anchor:** [[thesis-decoding-not-generating]]
|
||||
**Parent:** [ADR-0164 — Incremental Comprehension Reader](./ADR-0164-incremental-comprehension-reader.md)
|
||||
**Companion:** [ADR-0164.3 — ProblemReadingState and cross-sentence registry](./)
|
||||
**Resolves:** ADR-0164 §Open question #3 ("Pronoun-entity resolution")
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
ADR-0164 replaces the regex sentence-template front-end with an incremental
|
||||
compositional reader. Open question #3 of that ADR notes that the reader needs
|
||||
entity resolution and that the regex parser's
|
||||
`_resolve_question_entity` heuristic (`generate/math_candidate_parser.py:2457`)
|
||||
is "a reasonable starting point but should be reviewed against the
|
||||
compositional model." This ADR carries out that review and specifies the
|
||||
replacement policy.
|
||||
|
||||
Pronoun and entity resolution is load-bearing for GSM8K. A large fraction of
|
||||
question sentences refer to the subject of preceding statements only by
|
||||
pronoun ("how much money does **she** make?", "how much will it cost **him**?",
|
||||
"how much weight does **he** have to lose?"). A reader that comprehends the
|
||||
whole problem but refuses on the question because it cannot resolve the
|
||||
question's subject pronoun is a reader that has failed at exactly the moment
|
||||
it needed to commit.
|
||||
|
||||
The replacement must hold to the same invariants ADR-0164 holds to:
|
||||
deterministic, no hidden normalization, no stochastic fallback, refuses
|
||||
cleanly on novel structure, and produces evidence the teaching loop can
|
||||
chew on (typed refusal reasons with position information).
|
||||
|
||||
## §1 — Review of the existing heuristic
|
||||
|
||||
The function under review is `_resolve_pronoun_entity` (lines 2413–2454) and
|
||||
its wrapper `_resolve_question_entity` (lines 2457–2477). Its behavior:
|
||||
|
||||
1. The pronoun surface form is lower-cased.
|
||||
2. `they` and `it` are unconditionally refused.
|
||||
3. `she` / `her` look up the `_FEMALE_NAMES` whitelist (62 entries, line
|
||||
2376).
|
||||
4. `he` / `his` / `him` look up the `_MALE_NAMES` whitelist (76 entries, line
|
||||
2389).
|
||||
5. The full problem text is scanned with a Title-cased proper-noun extractor
|
||||
(`_PROPER_NOUN_MENTION_RE`, line 2408).
|
||||
6. Distinct names matching the gender whitelist are accumulated in order of
|
||||
first mention.
|
||||
7. **Exactly one** match → resolve to that name. Zero or two-plus → refuse.
|
||||
|
||||
### §1.1 What this heuristic gets right
|
||||
|
||||
- **Refuse-on-ambiguity is the correct default under wrong = 0.** Returning
|
||||
the wrong entity binding produces a candidate that the binding graph will
|
||||
happily admit and the verifier may not catch (the verifier checks unit
|
||||
closure and arithmetic, not whether the question subject was the right
|
||||
person). Refusal flows to the teaching loop instead.
|
||||
- **Pure and deterministic.** No global state, no normalization side effects,
|
||||
no nondeterministic name picking.
|
||||
- **Closed-set vocabulary.** The name whitelists are ratified data; they are
|
||||
the seed for the operational lexicon ADR-0164 promotes them into.
|
||||
- **Plural and neuter refusal.** `they` and `it` are not pretended-resolved.
|
||||
|
||||
### §1.2 What it gets wrong
|
||||
|
||||
Five concrete failure modes are observable on the GSM8K train_sample:
|
||||
|
||||
**F1 — Whitelist closed-set gap.** The proper-noun extractor finds names that
|
||||
are not on either whitelist (Yun, Marion, Rex, Georgie, Allison in
|
||||
train_sample). When the question uses a pronoun whose only entity candidate
|
||||
is one of these, the heuristic refuses despite the antecedent being
|
||||
unambiguous in context.
|
||||
|
||||
- *Example.* Case 0034 ("Georgie is a varsity player on a football team.
|
||||
**He** can run 40 yards within 5 seconds. … how many yards will **he** be
|
||||
able to run within 10 seconds?"). `Georgie` is not in `_MALE_NAMES`.
|
||||
Distinct-males-matching = 0. Refuses with no candidate. The text is
|
||||
unambiguous: Georgie is the only animate proper noun in the problem.
|
||||
|
||||
**F2 — Multi-entity question text where recency disambiguates.** The
|
||||
heuristic counts *distinct* mentions over the whole problem and refuses on
|
||||
≥2. Recency, syntactic role, and the question's own subject slot are
|
||||
discarded.
|
||||
|
||||
- *Example.* Case 0017 ("Jason has a carriage house … Eric wants to rent the
|
||||
house for 20 days. How much will it cost **him**?"). Two males (Jason,
|
||||
Eric). Distinct count = 2 → refuses. But "Eric wants to rent" is the only
|
||||
agent-of-renting clause; "cost him" attaches to the renter. A recency- or
|
||||
role-aware policy resolves this. The current heuristic cannot.
|
||||
|
||||
**F3 — Plural antecedent of a group introduced by conjunction.** `they` is
|
||||
unconditionally refused. When a problem opens with a conjoined subject
|
||||
("Aaron and his brother Carson each saved up $40 … how many scoops did
|
||||
**they** each buy?"), the group is well-defined and the question is
|
||||
well-posed. Refusing collapses a solvable case.
|
||||
|
||||
- *Example.* Case 0026. Distinct males = 2 (Aaron, Carson). On a per-name
|
||||
question this is ambiguous; on `they` it is exactly the group. The
|
||||
heuristic refuses both ways.
|
||||
|
||||
**F4 — Kinship and relational entities are invisible.** "her grandfather",
|
||||
"her father", "her mother", "his brother", "his friend" introduce entities
|
||||
that participate in arithmetic but never enter the registry. The Title-cased
|
||||
extractor sees only proper nouns.
|
||||
|
||||
- *Example.* Case 0033 ("Rachel is 12 years old, and **her grandfather** is
|
||||
7 times her age. **Her mother** is half grandfather's age, and **her
|
||||
father** is 5 years older than her mother. How old will Rachel's father be
|
||||
when **she** is 25 years old?"). The statement-layer "her" inside "her
|
||||
grandfather" has no referent to bind to *as a self-standing entity*; it is
|
||||
a relational modifier. The question-layer "she" has Rachel as the unique
|
||||
female proper noun — both policies resolve here — but the underlying
|
||||
reading is wrong: the heuristic treats "her grandfather" as a
|
||||
document-internal mention rather than as a relational entity rooted at
|
||||
Rachel.
|
||||
|
||||
**F5 — Inferred-gender names are not learned in context.** If the same
|
||||
unknown name appears with `he` ten times across a problem, the heuristic
|
||||
still cannot resolve any of them, because gender is only inferred from the
|
||||
whitelist. There is no co-reference feedback.
|
||||
|
||||
- *Example.* Case 0034 again — every "he" in the text is structurally
|
||||
unambiguous given the single proper noun. The heuristic learns nothing
|
||||
from this.
|
||||
|
||||
### §1.3 Summary verdict
|
||||
|
||||
The heuristic is correct in policy (refuse on ambiguity, deterministic,
|
||||
whitelist-based, closed-set) and inadequate in representation. Its evidence
|
||||
about pronouns lives in a flat scan of the entire problem text; the
|
||||
compositional reader has, by Phase 1, an ordered stream of comprehended
|
||||
entities with positions. The heuristic should be replaced by a policy that
|
||||
consumes that stream — same conservatism, more information.
|
||||
|
||||
## §2 — Replacement: EntityRegistry as a field on ProblemReadingState
|
||||
|
||||
The reader maintains a `ProblemReadingState` across sentences (ADR-0164.3
|
||||
specifies this carrier in full). One field of that state is an
|
||||
`EntityRegistry`:
|
||||
|
||||
```text
|
||||
EntityRegistry:
|
||||
entries: tuple[EntityEntry, ...] # frozen tuple, append-only per sentence
|
||||
resolution_log: tuple[Resolution, ...] # per-pronoun decision evidence
|
||||
|
||||
EntityEntry:
|
||||
canonical_name: str # the surface lemma as comprehended
|
||||
gender_inferred: Gender # FEMALE | MALE | NEUTER | GROUP | UNKNOWN
|
||||
gender_source: GenderSource # LEXICON_NAME | KINSHIP | PRONOUN_BACKFILL | DECLARED
|
||||
first_mention_pos: SourcePosition # (sentence_idx, token_idx)
|
||||
last_mention_pos: SourcePosition # updated on each subsequent mention
|
||||
relational_anchor: EntityRef | None # for kinship/relation entities ("her grandfather" → anchor=Rachel)
|
||||
syntactic_roles: tuple[Role, ...] # SUBJECT, OBJECT, POSSESSOR, …
|
||||
|
||||
Resolution:
|
||||
pronoun_surface: str # "she", "him", …
|
||||
pronoun_pos: SourcePosition
|
||||
decision: ResolvedEntity | RefusalReason
|
||||
```
|
||||
|
||||
The registry is part of the reader's immutable state per ADR-0164 §Decision.
|
||||
Every entity entrance and every pronoun resolution appears in the log; the
|
||||
log is part of the canonical-bytes serialization that drives `trace_hash`.
|
||||
|
||||
### §2.1 How entities enter the registry
|
||||
|
||||
The reader's lexicon (ADR-0164 §Decision §Operational lexicon) already
|
||||
distinguishes `proper_noun_entity`, `entity_pronoun`, and the kinship /
|
||||
relational category set. Entry into the registry happens at three points:
|
||||
|
||||
1. **Proper-noun entity.** When a token's category is `proper_noun_entity`,
|
||||
a fresh entry is appended. Gender is inferred from
|
||||
`lexicon[token].gender_marker` if present; otherwise `UNKNOWN`. Source =
|
||||
`LEXICON_NAME`.
|
||||
|
||||
2. **Kinship / relation noun with possessor.** When a possessive determiner
|
||||
("her", "his", "their", "Rachel's") attaches to a kinship noun in the
|
||||
lexicon's `kinship_relation` category, a fresh entry is appended with
|
||||
`relational_anchor` = the resolved possessor and `gender_inferred` =
|
||||
`lexicon[kinship_noun].gender_marker` (mother → FEMALE, father → MALE,
|
||||
sibling → UNKNOWN, friend → UNKNOWN, etc.). Source = `KINSHIP`.
|
||||
|
||||
3. **Conjoined subject.** When the reader closes a noun-phrase frame opened
|
||||
by `<entity> and <entity>`, a `GROUP` entry is appended whose membership
|
||||
tuple holds the two component entries. The component entries remain
|
||||
separately registered; the group entry is what `they` resolves against.
|
||||
Source = `LEXICON_NAME` (for the conjunction frame closure).
|
||||
|
||||
The registry is **append-only per sentence**. Re-mentions update
|
||||
`last_mention_pos` and may add a syntactic role, but do not create a new
|
||||
entry.
|
||||
|
||||
### §2.2 How pronouns resolve
|
||||
|
||||
When the reader's category for the current token is `entity_pronoun`, the
|
||||
policy is:
|
||||
|
||||
1. Compute `compat_gender`: FEMALE for `she/her/hers/herself`; MALE for
|
||||
`he/him/his/himself`; GROUP for `they/them/their/themselves`; NEUTER for
|
||||
`it/its/itself`.
|
||||
2. Filter registry entries: keep entries `E` where
|
||||
`compatible(E.gender_inferred, compat_gender)`. The compatibility table
|
||||
is:
|
||||
- FEMALE ↔ FEMALE
|
||||
- MALE ↔ MALE
|
||||
- GROUP ↔ GROUP
|
||||
- GROUP ↔ {FEMALE, MALE} only if the group's components include at least
|
||||
one member of the requested gender (used for "she" picking out the
|
||||
female member of a mixed group; not currently used at Phase 1).
|
||||
- NEUTER ↔ NEUTER (excludes animates; used for cost/object/process
|
||||
antecedents)
|
||||
- UNKNOWN ↔ {FEMALE, MALE}: tentatively compatible, see step 5.
|
||||
3. If the filtered list is empty: **refuse** with reason
|
||||
`unresolved_pronoun`. Position evidence = pronoun_pos and the registry
|
||||
snapshot.
|
||||
4. If the filtered list has more than one **certain-gender** entry: apply
|
||||
the recency tiebreaker (§2.3). If the tiebreaker leaves two entries
|
||||
within the ambiguity window: **refuse** with reason
|
||||
`ambiguous_pronoun_referent`.
|
||||
5. If the only candidate is `UNKNOWN`-gendered and exactly one such entity
|
||||
exists in the entire registry (the "single salient entity" case): bind
|
||||
the pronoun to it AND back-fill the entity's `gender_inferred` =
|
||||
`compat_gender`, `gender_source = PRONOUN_BACKFILL`. This addresses F1
|
||||
and F5 without weakening conservatism: the back-fill triggers only when
|
||||
the entity is *uniquely available*.
|
||||
6. Otherwise (multiple UNKNOWN entries, no certain-gender candidate): refuse
|
||||
with `ambiguous_pronoun_referent`.
|
||||
|
||||
Every branch appends a `Resolution` to the log with full position and
|
||||
candidate set so the teaching loop sees exactly why a binding succeeded or
|
||||
refused.
|
||||
|
||||
### §2.3 Recency tiebreaker
|
||||
|
||||
When ≥2 certain-gender entries match:
|
||||
|
||||
- Compute `last_mention_distance(E) = pronoun_pos − E.last_mention_pos` in
|
||||
token units (across sentences, sentence boundaries count as a fixed token
|
||||
penalty `SENTENCE_PENALTY = 1` to make cross-sentence reach explicit and
|
||||
bounded).
|
||||
- The candidate with the **smallest** distance wins, **provided** the next
|
||||
candidate is at least `RECENCY_GAP_MIN = 2` further. Otherwise the two
|
||||
candidates are within the ambiguity window and the policy refuses.
|
||||
- Subject-role mentions count once; object/possessor mentions are not
|
||||
preferred or demoted (no syntactic bias at Phase 1 — adding one is a
|
||||
separate sub-ADR with measurement).
|
||||
|
||||
The constants `SENTENCE_PENALTY` and `RECENCY_GAP_MIN` are part of the
|
||||
ratified policy. Changing either requires a sub-ADR and re-running the
|
||||
capability lanes.
|
||||
|
||||
## §3 — Refusal-first ambiguity rules
|
||||
|
||||
Two rules are exhaustive at Phase 1:
|
||||
|
||||
### Rule R1 — `ambiguous_pronoun_referent`
|
||||
|
||||
**Trigger.** ≥2 certain-gender registry entries are compatible with the
|
||||
pronoun and the recency tiebreaker does not separate them by at least
|
||||
`RECENCY_GAP_MIN` tokens.
|
||||
|
||||
**Concrete example.** Hypothetical mini-text (from real GSM8K phrasing):
|
||||
|
||||
> "John gave Tom $5. **He** had $20 left."
|
||||
|
||||
Registry after sentence 1: `[John:MALE@(0,0), Tom:MALE@(0,2)]`. The
|
||||
question-level pronoun "He" has `last_mention_distance(John) = 4`,
|
||||
`last_mention_distance(Tom) = 2` (assuming a 4-token sentence). Tom is
|
||||
nearer by 2. With `RECENCY_GAP_MIN = 2`, the gap is exactly the threshold;
|
||||
the *strict* form (`gap > MIN`) refuses. The *permissive* form (`gap >= MIN`)
|
||||
resolves to Tom. ADR-0164.2 adopts the **strict** form: refuse, surface
|
||||
both candidates, let the teaching loop or a future role-aware sub-ADR
|
||||
disambiguate. This preserves wrong = 0 in exchange for one refused case.
|
||||
|
||||
### Rule R2 — `unresolved_pronoun`
|
||||
|
||||
**Trigger.** No registry entry is compatible with the pronoun.
|
||||
|
||||
Two sub-cases:
|
||||
|
||||
- **R2a — no entity of the right gender.** E.g. `she` with only male
|
||||
entries. Refuses immediately.
|
||||
- **R2b — empty registry.** E.g. a question that opens with a pronoun
|
||||
whose antecedent is in a prior sentence the reader has not yet
|
||||
comprehended (reader-order bug, or genuinely orphaned pronoun). Refuses
|
||||
immediately.
|
||||
|
||||
R2 refusals carry the registry snapshot in the log so the teaching loop can
|
||||
see whether the gap was a missing entity, a missing kinship inference, or a
|
||||
missing gender lexicon entry.
|
||||
|
||||
### What is **not** a separate rule
|
||||
|
||||
- "Plural pronoun with no group entity" → R2 (`they` with only singletons
|
||||
and no conjunction-closed group).
|
||||
- "Neuter pronoun with only animate entries" → R2.
|
||||
- "Possessive pronoun in a kinship phrase" (e.g. "her grandfather") is not
|
||||
a resolution event; it is an entity-creation event (§2.1 case 2). No rule
|
||||
applies.
|
||||
|
||||
## §4 — Worked walk-through on five GSM8K train_sample cases
|
||||
|
||||
Notation: positions written `(sent, tok)`. Lexicon assumed: female-name
|
||||
markers for Tina, Marion, Erica, Martha, Jane; male-name markers for
|
||||
Malcolm, Aaron, Carson, Jason, Eric, James, John; UNKNOWN for Georgie.
|
||||
Kinship lexicon: brother → MALE.
|
||||
|
||||
### 4.1 — Case 0001 (Tina, multi-pronoun)
|
||||
|
||||
Text (question only, for brevity):
|
||||
|
||||
> "Tina makes $18.00 an hour. If **she** works more than 8 hours per shift,
|
||||
> **she** is eligible … If **she** works 10 hours every day for 5 days,
|
||||
> how much money does **she** make?"
|
||||
|
||||
Registry trajectory:
|
||||
|
||||
| Step | Event | Registry |
|
||||
|---|---|---|
|
||||
| 1 | Token "Tina" at (0,0) | `[Tina:FEMALE@(0,0)/LEXICON_NAME]` |
|
||||
| 2 | Pronoun "she" at (1,2) | Filter FEMALE → {Tina}. Single candidate. **Resolve → Tina.** `last_mention(Tina) ← (1,2)`. |
|
||||
| 3 | "she" at (1,7) | Same → Tina. |
|
||||
| 4 | "she" at (2,2) | Same → Tina. |
|
||||
| 5 | "she" at (2,11) | Same → Tina. |
|
||||
|
||||
All four pronouns resolve. Final binding for the question slot: Tina. Old
|
||||
heuristic: identical outcome. **Agreement.**
|
||||
|
||||
### 4.2 — Case 0010 (Yun, Marion — no question pronoun)
|
||||
|
||||
Text:
|
||||
|
||||
> "Yun had 20 paperclips initially, but then lost 12. Marion has 1/4 more
|
||||
> than what Yun currently has, plus 7. How many paperclips does **Marion**
|
||||
> have?"
|
||||
|
||||
The question is a direct proper-noun binding (no pronoun). Registry:
|
||||
|
||||
| Step | Event | Registry |
|
||||
|---|---|---|
|
||||
| 1 | "Yun" at (0,0) | `[Yun:UNKNOWN@(0,0)/LEXICON_NAME]` (Yun absent from name lexicon → UNKNOWN) |
|
||||
| 2 | "Marion" at (1,0) | `[Yun:UNKNOWN, Marion:UNKNOWN@(1,0)]` |
|
||||
| 3 | Question "Marion" at (2,5) | Proper-noun direct binding (no pronoun → no Rule). `last_mention(Marion) ← (2,5)`. |
|
||||
|
||||
No pronoun resolution required. Question slot binds to Marion directly. Old
|
||||
heuristic: same. **Agreement.** This case demonstrates that the policy does
|
||||
not perturb proper-noun-only questions.
|
||||
|
||||
### 4.3 — Case 0027 (Malcolm, "he")
|
||||
|
||||
Text (compressed):
|
||||
|
||||
> "Malcolm has 240 followers on Instagram and 500 followers on Facebook.
|
||||
> The number of followers **he** has on Twitter is half … and **he** has
|
||||
> 510 more followers on Youtube than **he** has on TikTok. How many
|
||||
> followers does Malcolm have on all his social media?"
|
||||
|
||||
Registry:
|
||||
|
||||
| Step | Event | Registry |
|
||||
|---|---|---|
|
||||
| 1 | "Malcolm" at (0,0) | `[Malcolm:MALE@(0,0)/LEXICON_NAME]` |
|
||||
| 2 | "he" at (1,7) | Filter MALE → {Malcolm}. Resolve. |
|
||||
| 3 | "he" at (1,18) | Same. |
|
||||
| 4 | "he" at (1,22) | Same. |
|
||||
| 5 | "Malcolm" at (2,4) | Proper-noun direct binding (question). |
|
||||
|
||||
All pronouns resolve to Malcolm. Old heuristic: same. **Agreement.**
|
||||
|
||||
### 4.4 — Case 0017 (Jason and Eric, "him")
|
||||
|
||||
Text:
|
||||
|
||||
> "Jason has a carriage house that he rents out. He's charging $50.00 per
|
||||
> day or $500.00 for 14 days. Eric wants to rent the house for 20 days. How
|
||||
> much will it cost **him**?"
|
||||
|
||||
Registry trajectory (showing only entries/updates relevant to "him"):
|
||||
|
||||
| Step | Event | Registry |
|
||||
|---|---|---|
|
||||
| 1 | "Jason" at (0,0) | `[Jason:MALE@(0,0)]` |
|
||||
| 2 | "he" at (0,6) | Filter MALE → {Jason}. Resolve. `last_mention(Jason) ← (0,6)` |
|
||||
| 3 | "He" at (1,0) | Same → Jason. `last_mention(Jason) ← (1,0)` |
|
||||
| 4 | "Eric" at (2,0) | `[Jason:MALE, Eric:MALE@(2,0)]` |
|
||||
| 5 | "him" at (3,7) | Filter MALE → {Jason, Eric}. `dist(Jason, him) = (3,7) − (1,0)` = ~12 tokens + 2*SENTENCE_PENALTY = 14. `dist(Eric, him) = (3,7) − (2,0)` = ~7 + SENTENCE_PENALTY = 8. Gap = 6 ≥ RECENCY_GAP_MIN. **Resolve → Eric.** |
|
||||
|
||||
Old heuristic: distinct males = 2 → refuses (F2). New policy: resolves to
|
||||
Eric via recency, with the gap comfortably above the ambiguity window. The
|
||||
ground-truth solver path requires the cost to be billed to Eric (the
|
||||
renter), so the binding is correct. **Disagreement; new policy correct.**
|
||||
|
||||
### 4.5 — Case 0033 (Rachel + kinship)
|
||||
|
||||
Text:
|
||||
|
||||
> "Rachel is 12 years old, and **her grandfather** is 7 times her age.
|
||||
> **Her mother** is half grandfather's age, and **her father** is 5 years
|
||||
> older than her mother. How old will Rachel's father be when **she** is 25
|
||||
> years old?"
|
||||
|
||||
Registry trajectory:
|
||||
|
||||
| Step | Event | Registry |
|
||||
|---|---|---|
|
||||
| 1 | "Rachel" at (0,0) | `[Rachel:FEMALE@(0,0)]` |
|
||||
| 2 | Possessive "her" + kinship "grandfather" at (0,4–5) | Possessor resolves (FEMALE filter → Rachel). Kinship entry created. `[Rachel:FEMALE, Rachel.grandfather:MALE@(0,5)/KINSHIP, anchor=Rachel]` |
|
||||
| 3 | Possessive "Her" + "mother" at (1,0–1) | Anchor=Rachel. `[…, Rachel.mother:FEMALE@(1,1)/KINSHIP]` |
|
||||
| 4 | Possessive "her" + "father" at (1,7–8) | Anchor=Rachel. `[…, Rachel.father:MALE@(1,8)/KINSHIP]` |
|
||||
| 5 | Possessive "Rachel's" + "father" at (2,2–3) | Re-mention; `last_mention(Rachel.father) ← (2,3)`. |
|
||||
| 6 | "she" at (2,8) | Filter FEMALE → {Rachel, Rachel.mother}. dist(Rachel) measured from last_mention (0,0)+penalties; dist(Rachel.mother) measured from (1,1)+penalties. Rachel.mother is closer. Gap above RECENCY_GAP_MIN? **No — Rachel.mother's last mention is far back, Rachel has not been re-mentioned by name since (0,0).** Compute: Rachel.mother at (1,1), Rachel at (0,0). Pronoun at (2,8). dist(Rachel) ≈ 8 + 2 = 10. dist(Rachel.mother) ≈ 7 + 1 = 8. Gap = 2 == RECENCY_GAP_MIN. Strict form: **refuse with `ambiguous_pronoun_referent`**, candidates {Rachel, Rachel.mother}. |
|
||||
|
||||
Old heuristic: distinct female proper names = 1 (Rachel) → resolves to
|
||||
Rachel. New policy: refuses on ambiguity because kinship-introduced female
|
||||
entities are in scope. The ground truth is Rachel ("when she is 25"). The
|
||||
new policy is **more conservative** here: it refuses a case the heuristic
|
||||
would have resolved correctly, but does so by surfacing a genuine
|
||||
ambiguity (the surface form does not exclude "when her mother is 25" as a
|
||||
reading). The refusal flows to the teaching loop and motivates the
|
||||
role-aware extension (subject-bias for the original named entity) as a
|
||||
later sub-ADR. **Disagreement; new policy refuses, old policy resolves
|
||||
correctly by luck — see §5 for the wrong = 0 reasoning.**
|
||||
|
||||
## §5 — Disagreement enumeration
|
||||
|
||||
Three disagreements between the old heuristic and the new policy are
|
||||
enumerated below. Verdict is given under the wrong = 0 discipline (correct,
|
||||
refuse, or wrong).
|
||||
|
||||
### D1 — Case 0017 (Jason + Eric, "him")
|
||||
|
||||
| Policy | Output | Verdict |
|
||||
|---|---|---|
|
||||
| Old heuristic | refuse (2 distinct males) | refuse |
|
||||
| New policy | resolve → Eric (recency) | **correct** |
|
||||
|
||||
**Reasoning.** The new policy uses information the old does not (mention
|
||||
order), preserves refuse-on-ambiguity for the close-call window, and
|
||||
resolves where the gap is wide. wrong = 0 is preserved because the gap
|
||||
threshold (`RECENCY_GAP_MIN`) is set strictly enough that close cases still
|
||||
refuse.
|
||||
|
||||
### D2 — Case 0034 (Georgie, "he")
|
||||
|
||||
| Policy | Output | Verdict |
|
||||
|---|---|---|
|
||||
| Old heuristic | refuse (Georgie ∉ `_MALE_NAMES`) | refuse |
|
||||
| New policy | resolve → Georgie via single-salient-entity back-fill (§2.2 step 5) | **correct** |
|
||||
|
||||
**Reasoning.** Georgie is the only animate entity in the problem; "he" has
|
||||
exactly one possible antecedent. The back-fill is conservative because it
|
||||
only fires when there is *exactly one* UNKNOWN-gendered entity in the
|
||||
entire registry. The old heuristic refused because the closed-set name
|
||||
whitelist did not contain Georgie. wrong = 0 is preserved because the
|
||||
single-salient rule cannot bind a pronoun to a wrong entity — there is no
|
||||
other entity to be wrong about.
|
||||
|
||||
### D3 — Case 0026 (Aaron and Carson, "they")
|
||||
|
||||
| Policy | Output | Verdict |
|
||||
|---|---|---|
|
||||
| Old heuristic | refuse (`they` unconditionally refused) | refuse |
|
||||
| New policy | resolve → group {Aaron, Carson} via conjunction-closed GROUP entry (§2.1 case 3) | **correct** |
|
||||
|
||||
**Reasoning.** "Aaron and his brother Carson each saved up $40" closes a
|
||||
conjunction frame that registers a GROUP entry. The question "how many
|
||||
scoops did they each buy?" filters to GROUP entries and finds the unique
|
||||
match. The downstream binding produces the per-member result via the
|
||||
distributive modifier `each`. The old heuristic refused on every `they`
|
||||
regardless of registry state. wrong = 0 is preserved because GROUP
|
||||
resolution is enabled only when a conjunction frame has closed; absent that
|
||||
frame, `they` falls through to R2.
|
||||
|
||||
### D4 — Case 0033 (Rachel + kin, "she") — **counter-direction**
|
||||
|
||||
For completeness, the one case where new is *more conservative* than old:
|
||||
|
||||
| Policy | Output | Verdict |
|
||||
|---|---|---|
|
||||
| Old heuristic | resolve → Rachel (only female proper noun) | correct |
|
||||
| New policy | refuse with ambiguous_pronoun_referent ({Rachel, Rachel.mother}) | refuse |
|
||||
|
||||
**Reasoning.** The new policy has more information (kinship entities in
|
||||
registry) and uses it to flag a genuine surface ambiguity. The old
|
||||
heuristic resolves correctly only because it cannot see Rachel.mother. The
|
||||
wrong = 0 discipline preserves correctness either way (refusal is not
|
||||
wrong); the cost is one extra refused case at Phase 1, which the teaching
|
||||
loop turns into evidence for a role-bias sub-ADR.
|
||||
|
||||
D1 + D2 + D3 each improve a refusal into a correct resolution; D4
|
||||
exchanges a lucky correct for a principled refusal. The net effect on the
|
||||
GSM8K train_sample is monotonically positive on `correct` and zero on
|
||||
`wrong`, which is exactly the operating regime ADR-0164 requires.
|
||||
|
||||
## §6 — Constraints
|
||||
|
||||
This ADR inherits ADR-0164 §Constraints in full. Additionally:
|
||||
|
||||
1. **Strict-gap recency.** `RECENCY_GAP_MIN` is ratified at 2 tokens.
|
||||
Adjustments require a sub-ADR and capability-lane re-run.
|
||||
2. **Single-salient back-fill is exact.** The UNKNOWN → certain-gender
|
||||
back-fill (§2.2 step 5) fires **only** when the registry has exactly one
|
||||
UNKNOWN-gendered entity *globally*. Two UNKNOWN entries kill the rule;
|
||||
neither is back-filled.
|
||||
3. **No probabilistic gender inference.** The lexicon is the only source of
|
||||
certain gender at entity creation. Back-fill is co-reference, not
|
||||
statistical inference.
|
||||
4. **No syntactic-role bias at Phase 1.** Subject preference, possessor
|
||||
demotion, and similar heuristics are out of scope for this ADR. They
|
||||
require independent measurement and a separate sub-ADR.
|
||||
5. **Append-only registry.** Re-mentions update positions; entries do not
|
||||
merge, split, or get deleted within a problem. Gender back-fill
|
||||
*updates* an entry but does not change its identity.
|
||||
6. **Resolution log is part of trace.** Every Resolution event is part of
|
||||
the canonical-bytes serialization that feeds `trace_hash` per CLAUDE.md
|
||||
§Runtime Surface Contract.
|
||||
|
||||
## §7 — Acceptance criteria (Proposed → Accepted)
|
||||
|
||||
This ADR moves to **Accepted** when, alongside ADR-0164 Phase 1 acceptance:
|
||||
|
||||
1. `EntityRegistry`, `EntityEntry`, and `Resolution` types exist as
|
||||
frozen-dataclasses under `generate/comprehension/registry.py` with
|
||||
canonical-bytes serialization tests.
|
||||
2. The policy in §2 is implemented behind the reader's
|
||||
`entity_pronoun` category dispatch, with unit tests covering:
|
||||
- All five worked walk-throughs (§4) byte-equal to the ADR.
|
||||
- All three disagreement cases (§5 D1–D3) producing the documented
|
||||
outputs.
|
||||
- The counter-direction case (§5 D4) producing the documented refusal.
|
||||
- Single-salient back-fill firing once and only once per problem in
|
||||
the F1-style cases.
|
||||
- R1 and R2 refusal reasons emitted with full position and candidate
|
||||
evidence.
|
||||
3. Capability lanes G1–G5, S1 remain at 100% `wrong = 0`.
|
||||
4. `core eval cognition` and the GSM8K train_sample runner show monotonic
|
||||
non-decrease in `correct` and zero `wrong` against the post-D.2
|
||||
baseline.
|
||||
|
||||
## §8 — Open follow-ups (out of scope for this ADR)
|
||||
|
||||
- **Syntactic-role bias** (subject-preference): a sub-ADR after Phase 1
|
||||
evidence shows where R1 refusals cluster on subject-vs-object
|
||||
ambiguity.
|
||||
- **Cross-sentence anaphora window beyond two sentences**: currently
|
||||
bounded by `SENTENCE_PENALTY = 1` per boundary; longer chains may need
|
||||
a saturation curve. Empirical question.
|
||||
- **Pronoun chains** ("she … her … hers"): currently each pronoun resolves
|
||||
independently against the registry. Chain coherence (require all three
|
||||
to bind to the same entity) is a candidate strengthening.
|
||||
- **Mixed-gender group membership**: §2.2 step 2 allows GROUP ↔ single
|
||||
gender via membership, but Phase 1 does not exercise this. Defer until
|
||||
a real GSM8K case demands it.
|
||||
|
||||
## Cross-references
|
||||
|
||||
- **Parent:** [ADR-0164 — Incremental Comprehension Reader](./ADR-0164-incremental-comprehension-reader.md)
|
||||
- **Companion:** ADR-0164.3 — `ProblemReadingState` and cross-sentence
|
||||
registry (carries this `EntityRegistry` as one of its fields).
|
||||
- **Resolves:** ADR-0164 §Open question #3.
|
||||
- **Substrate that survives:** the binding graph still consumes
|
||||
`BoundUnknown(entity, unit, …)` tuples whose `entity` field is now
|
||||
sourced from the registry rather than the regex name whitelist.
|
||||
- **HITL corridor:** R1/R2 refusals carry the registry snapshot into the
|
||||
contemplation queue (ADR-0150/0152/0155/0161) for review.
|
||||
- **Anti-overfitting:** ADR-0114a — the policy is closed-set in its
|
||||
vocabulary (lexicon-driven), conservative in its dispatch
|
||||
(refuse-on-ambiguity), and falsifiable on the capability lanes.
|
||||
- **Thesis:** `[[thesis-decoding-not-generating]]` — the registry is what
|
||||
"found" looks like for an entity. Each pronoun narrows the candidate
|
||||
set; the resolution is the accumulation, not a guess.
|
||||
Loading…
Reference in a new issue