core/evals
Shay 7d6760ae5b feat(evals): the fluency lanes report the writer that actually ships (Phase 4)
Phase 4 of the grammar-unification arc, resolved by option (b) of the plan.

The problem: 149 green fluency cases scored `realize_target`, and
`core/cognition/pipeline.py` never calls it -- it calls `realize_semantic`.
`english_fluency_ood` reported 117/117 + 39/39 for a function that does not
speak, for the whole life of the lane.

The plan framed this as "which realizer is better". Reading the source says it
is not a quality question. `render_semantic`'s signature is
`(intent, subject, predicate, obj, secondary, language, root)` -- no `negated`,
no `quantifier`, no `tense`, no `aspect` -- and `realize_semantic` never reads
them off the step. So the serving writer cannot express content the
ArticulationStep is carrying:

    negated=False -> 'Knowledge is defined as opinion.'
    negated=True  -> 'Knowledge is defined as opinion.'

It serves the AFFIRMATIVE of a negated proposition. That is the ADR-0261 §5.1
family, not a fluency defect. Pinned here as a defect; NOT fixed, because
fixing it changes served output and belongs to Shay.

Measured -- identical contract, all seven scored corpora
--------------------------------------------------------
    bucket                                n     realize_target   realize_semantic
    feature-bearing, single node        214          207               49
    no features, multi-node             100          100                3
    no features, single node (CONTROL)   33           33               33
    total                               347          340               85

The control is what makes the rest mean anything. Every corpus hardcodes
IntentTag.UNKNOWN, so "the serving writer scores badly" could have been an
artifact of never giving it a real intent. On the 33 cases carrying nothing it
cannot express, the two are IDENTICAL -- so the gap on the other 314 is the
dropped features and the clause joining, not the intent.

Delivered
---------
- `grammatical_coverage/runner.py` reports `serving_accuracy` beside
  `accuracy`; `english_fluency_ood` delegates to that run_lane and gains it
  for free. The realizer is a parameter now instead of a hardcoded import.
- `tests/test_phase4_realizer_resolution.py` -- the control, the decomposition,
  the negation defect pin, and the §6 evidence.
- The claim is restated at both places it was made: the lane docstring and
  `grammar_roundtrip/contract.md`.

Zero served bytes change. No serving authorization needed.

§6 -- the plan's pre-commitment was wrong, and is corrected rather than
quietly edited
------------------------------------------------------------------------
§6 forks on read_rate: risen => Phase 5 diversity; near-zero => §1.8 graph-model
mismatch => ADR. It pre-committed to the second. Neither is what the
measurement says.

g_read_rate went to 1/293. The unblocked case was blocked by a one-line WRITER
defect (predicate-nominal object agreement, #135), not by §1.8. The other 292:

    no_template_match     289   reader has no SUBJ-VERB-OBJ template at all
    unknown_morphology      2   prepositional objects (reserved_word_in_np)
    unsupported_negation    1   reader has no negated-categorical template

Every one is the reader declining a CONSTRUCTION, not a projection disagreeing
about a graph it parsed. Where a construction is in both inventories the round
trip closes exactly. So the barrier is the OVERLAP of the two construction
inventories, currently one construction wide -- Phase 5's item 1, tractable,
not an ADR-scale model decision.

§1.6 read "uniform no_template_match" as evidence FOR the type mismatch. It is
not: no_template_match is a coverage fact, and the corpus was 289/293 bare
transitives, a construction the reader never claimed to read. The measurement
was mostly reporting the corpus's composition.

And a finding in my own stack
------------------------------
`tests/test_realizer_quantifier_agreement.py` lived ONLY in the `cognition`
suite, which is not on the AGENTS.md pre-push gate. So every pin Phases 3 and 4
added to it -- including the invariant covering all twelve inflection branches
-- ran in NO gate. That is why smoke stayed at 621 across two PRs that added 13
tests between them, and it is the same silent-red shape the smoke list already
calls out for test_adr_index.py.

Registered into `deductive`, which now runs 504 instead of 406.

Mutation
--------
    baseline                                             12 pass
    serving metric computed with realize_target           4 FAIL
    render_semantic GAINS a `negated` parameter           2 FAIL
    realize_semantic delegated to realize_target          6 FAIL

Row 2 matters most: if someone FIXES the negation defect, the pin forces a
deliberate revision instead of passing silently.

Still open, still Shay's: option (a), promoting realize_target to the serving
path. It now has a price tag -- 340/347 over 85/347 and the ability to say
"not", against a move in every surface hash and whatever the Shadow Coherence
Gate ruling in core/cognition/surface_resolution.py was protecting.

[Verification]: in-worktree on CPython 3.12.13 with `uv sync --locked` --
smoke 621, deductive 504 (was 406; +12 new Phase 4 pins, +86 previously
ungated), lane pins 11/11 unchanged, no pin edited.
2026-07-27 11:57:48 -07:00
..
adr_0242_v2_energy_compare
adr_0243_cognitive_lifecycle feat(adr-0243): Phase 4 falsifiability benchmark — metrics eval + CLI dispatcher 2026-07-17 12:59:24 -07:00
adr_0244_gamma_calibration fix(tests): clear the last reds — a machine-specific path and a stale drift pin 2026-07-25 16:31:06 +00:00
adr_0244_identity_gate fix(cognition): pre-stage land repairs after L2/PASSTHROUGH excision 2026-07-20 13:34:59 -07:00
adr_0244_qtop_vacuity feat(adr-0244): prove Q_top vacuity — D4 hollow-gate evidence + annotation upgrade 2026-07-17 14:39:52 -07:00
adr_0246_discrimination feat(adr-0246): Opus audit + §3.7 admit surface + serve wiring + §6.3 discrimination 2026-07-17 22:51:46 -07:00
adr_0246_geometric_suite draft(adr-0246): slice-1 scaffold — §6.1/§6.2 eval suite + malformed-F guard 2026-07-17 22:09:32 -07:00
adr_0246_grounding_feasibility feat(adr-0246): completion — §4.1 records, path serve integration, §11 feasibility (honest NULL), ADR body Proposed 2026-07-17 23:36:41 -07:00
adr_0246_mismatch_diagnostic feat(adr-0246): §3 induced-action primitives (pure, off-serving) 2026-07-17 21:36:57 -07:00
adversarial_identity
analogical_transfer feat(adr-0240/0243): seam S2 — chiral-composed, harness-driven biography write-path 2026-07-18 10:15:38 -07:00
anchor_lens_tour
anti_regression
articulation
articulation_of_status
audio_sensorium
audit_tour
calibration
capability_index chore(generalization): Tier S housekeeping — smoke promotion, capability-index entries, promotion sweep (S5) 2026-07-24 17:13:19 -07:00
classical_literature_ood
close_derived_climb
cognition fix(cognition): expose hash_surface; point the register-matrix pin at it 2026-07-25 14:06:36 +00:00
cold_start_grounding
combined_rate_oracle
compositionality
compound_intent_decomposition
comprehension
constraint_oracle
contemplation_quality
contradiction_detection
conversation
conversational_thread_coherence
cross_domain_transfer
curriculum_loop_closure feat(deduction-serve): Band v6-EX — existential witnesses, decided (ADR-0261) 2026-07-24 14:14:37 -07:00
curriculum_serve feat(curriculum): taught negatives — polarity is read (ADR-0264 R1-R4, R8) 2026-07-26 12:19:19 -07:00
deduction_serve fix(evals): the deduction lane hashes the prose it serves 2026-07-26 19:45:48 -07:00
deductive_logic
demo_composition
determination_closure
determination_estimation
deterministic_fluency
dimensional
discourse_paragraph
domain_contract_validation
edge_budget
elementary_mathematics_ood
english_fluency_ood
environment_falsification
event_vision_sensorium
fabrication_control
field_incidence
flywheel_demo
forward_semantic_control
foundational_biology_ood
foundational_physics_ood
frame_verdict_text_cwa
frontier_compare
generalization
grammar_roundtrip feat(evals): the fluency lanes report the writer that actually ships (Phase 4) 2026-07-27 11:57:48 -07:00
grammatical_coverage feat(evals): the fluency lanes report the writer that actually ships (Phase 4) 2026-07-27 11:57:48 -07:00
gsm8k_math test(reader-arc): pin deterministic tune/measure split BEFORE grammar work 2026-07-18 16:39:04 -07:00
gsm8k_parser_dev
hebrew_fluency
identity_divergence
industry_demos
inference_closure
introspection
koine_greek_fluency
l10_always_on
l10_continuity
lab
learning_arc
learning_loop
long_context_cost
math_bounded_grammar/v1
math_capability_axes
math_expert_claims/v1
math_symbolic_equivalence
math_teaching_corpus/v1
miner_loop_closure
monotonic_learning
multi_agent_composition
multi_sentence_response
multi_step_reasoning
obligation_2_ood_ratio
obligation_5_perturbation
obligation_6_depth_curve
obligation_8_adversarial
obligation_10_pack_provenance
orthogonality_tour
prompt_diversity
proof_carrying_promotion_demo
proofwriter_owa
propositional_logic
provenance
public_demo fix(ci): re-pin public_demo lane — real content drift, not the env flake (S2) 2026-07-24 16:58:05 -07:00
rate_oracle
realizer_guard
refusal_calibration
refusal_taxonomy
register_diagnostics
register_tour
relational
relational_inference/v1
relational_metric
relational_operator_ablation feat(eval): sealed relational operator ablation lane v1 2026-07-19 21:00:15 -07:00
relational_transitive
reports
results
reviewer_registry
sample_efficiency
self_consistency_over_time
sensorimotor_sensorium
sensorium
set_membership
setup_oracle
structure_mapping feat(trackb): S1–S4 symbolic SME, selector, pure-S1 coverage gain 2026-07-19 19:49:07 -07:00
syllogism
symbolic_logic
teaching_injection_resistance
total_ordering
vision_sensorium
walkthrough_chain
warmed_session_consistency
zero_code_domain_acquisition
__init__.py
_parallel.py
baseline_runner.py
CLAIMS.md
cognition_cases.jsonl
framework.py
generalized_lift_instrument.py chore(adr-0250): ratify Accepted + fold in PR #74 review findings 2026-07-18 15:07:29 -07:00
holdout_runner.py
lift_evidence_handoff.py feat(evals): seams S4+S5 — generalized-lift instrument + ports/handoff evidence 2026-07-18 10:34:26 -07:00
logic_cnf_compiler.py feat(adr-0249): P3 structural formula→CNF converter (deduction leg) 2026-07-18 12:40:31 -07:00
metrics.py fix(evals): carry hash_surface on the CaseResult the register matrix actually uses 2026-07-25 14:25:31 +00:00
multi_register_program.py feat(reader-arc): compare compiler-side guards for inverse + chained shapes 2026-07-18 17:14:16 -07:00
parallel.py
run_cognition_eval.py fix(evals): carry hash_surface on the CaseResult the register matrix actually uses 2026-07-25 14:25:31 +00:00
turn_program.py feat(adr-0249): P4 turn-program compiler + chained-relaxation executor 2026-07-18 12:55:31 -07:00
vocab_trigger_instrument.py feat(generalization): vocab-trigger instrument — mechanism-vs-coverage refusal histogram (S3) 2026-07-24 16:54:13 -07:00