Commit graph

1 commit

Author SHA1 Message Date
Shay
800cf6591e
feat(ADR-0164.P1): reader/regex hybrid coexistence + Phase 1 measurement gate (#331)
Phase A — RuntimeConfig flag:
  core/config.py: adds `comprehension_reader_questions: bool = False`
  Default OFF preserves byte-identical behaviour with today.

Phase B — Hybrid wiring in candidate-graph path:
  generate/math_candidate_graph.py:
    - _try_reader_for_question() dispatches to the comprehension reader
      BEFORE the regex question parser; refusal falls through to regex
    - reader_trace: tuple[str, ...] field on CandidateGraphResult captures
      JSON-encoded admit/fallthrough events for audit
  generate/comprehension/lifecycle_runtime_adapter.py (new):
    - build_problem_state_from_candidates(): converts regex-parser output
      to ProblemReadingState for the reader's pronoun-resolution step
    - invoke_reader_for_question(): tokenises sentence, drives lifecycle
    - project_to_candidate_unknown(): QuestionTargetSlot → CandidateUnknown
    - trace-event constructors for admit and fallthrough

Phase C — Capability-axis regression:
  All existing tests pass with flag OFF and ON; zero new regressions.
  Two pre-existing failures on main are unrelated to this PR.

Phase D — GSM8K train_sample measurement:
  evals/gsm8k_math/train_sample/v1/runner.py: --use-reader flag triggers
    baseline-off + reader-on runs and writes reader_phase1_delta.json
  evals/gsm8k_math/train_sample/v1/reader_phase1_delta.json (new):
    baseline-off: correct=3 refused=47 wrong=0
    reader-on:    correct=3 refused=47 wrong=0
    delta: all zeros — Mixed result expected (Phase 2 scope)
    wrong=0 invariant preserved in both modes.

Phase E — Coexistence tests:
  tests/test_reader_coexistence.py (new): 13 tests covering
    flag-OFF byte-identity, flag-ON determinism, wrong=0 invariant,
    trace shape validation, Brief-8 target admission, and fallthrough
    preservation for unknown-unit words.

Admission gate result: Mixed (correct=3, below the ≥10 bar).
All statement-side barriers remain in place; Phase 2 (reader for
statement sentences) is required to drive correct≥10. Documented in
reader_phase1_delta.json and train_sample/v1/runner.py docstring.
2026-05-26 21:14:11 -07:00