core/evals
Shay c69f99485d feat(evals): measure the reader/writer construction inventories against each other
Phase 5 item 1 asked for the overlap to be sized "by measuring the reader's
construction set against the writer's rather than by growing corpora blindly".
New lane `evals/construction_inventory` does that: it sweeps the writer's whole
parameter space (every RhetoricalMove x IntentTag x predicate x quantifier x
tense x aspect, both public entry points, 32292 cells), quotients it by the
reader's OWN function-word skeleton, and comprehends each construction under
three vocabularies.

  writer constructions            1739
  reader constructions              19   (mint-site AST-guarded)
  overlap, faithful                  6
  accepts but MIS-READS             22
  refuses                         1711
  vocabulary-dependent               0

Faithfulness, not acceptance, is the criterion — and that reverses two claims
in the plan's §6 RESULT, using a metric that was already on the page:

* g_read_rate is 1/293 but **g_args_rate is 0.0**. The one surface that "reads",
  `all molecules are defined as compounds`, is comprehended as
  `subset(molecule, defined_as_compound)` — the reader chunked the writer's verb
  phrase into a class name. It accepted; it did not comprehend.
* "one construction wide" was wrong both ways: zero on that corpus, six over the
  writer's actual output space. A corpus cannot report an inventory's size.

Three findings re-order Phase 5 item 1:

1. The reader FABRICATES on 22 constructions — neither reads nor refuses.
   `every dog is a mammal` -> member(every_dog, mammal);
   `furthermore, all dogs are mammals` -> asserted(furthermore).
   Both are ordinary user English, not writer artefacts. Root cause: _RESERVED
   lacks the function words the writer emits, and _parse_propositional accepts
   any single token as a fact.
2. It reaches SERVED output. deduction_surface recites
   `Given: furthermore; p implies q; p.` — a premise the user never stated — and
   chat/runtime.py realizes declarative turns into the held self, so a
   fabricated atom is vault-writable. Widening the inventory first would widen
   the fabrication surface with it.
3. ADR-0265's defect class survives inside ADR-0265's designated owner of clause
   grammar: the four aspect arms of _inflect_predicate bind `negated` to a
   wildcard and never read it, so `dog has been defined as mammal` serves both
   the assertion and the denial (10530/16146 points). Unreachable today — no
   producer sets aspect — so a loaded gun, not a casualty. It survived because
   ADR-0265's invariant is structural (is `negated` threaded?) and cannot see an
   arm that receives the flag and ignores it. A behavioural sweep can.

The two fixes are written already, as the mutations that turn the defect pins
red. They are NOT applied here: they change what CORE comprehends from user
input, which is a serving change on the truth path — authorization-gated, ADR
first, on the ADR-0261 §5.1 refuse-don't-drop precedent.

Guards, so the tables cannot rot: reader constructions are pinned to an AST
count of the reader's 10 mint sites; the declared tense/aspect axes are pinned
to _inflect_predicate's match arms; the committed corpus is pinned to the
fillers in use. 14 pins, 13 mutations observed RED.

[Verification]: in-worktree, CPython 3.12.13, uv sync --locked.
  deductive 517 (was 503, +14 — count moved, registration confirmed)
  smoke     641 (unchanged; the file is registered in `deductive`)
  lane SHAs 11/11 match, no pin edited
  pyright   0 errors on all new files
  mutations 13/13 red, including both fabrication fixes
2026-07-27 14:11:07 -07:00
..
adr_0242_v2_energy_compare
adr_0243_cognitive_lifecycle feat(adr-0243): Phase 4 falsifiability benchmark — metrics eval + CLI dispatcher 2026-07-17 12:59:24 -07:00
adr_0244_gamma_calibration fix(tests): clear the last reds — a machine-specific path and a stale drift pin 2026-07-25 16:31:06 +00:00
adr_0244_identity_gate fix(cognition): pre-stage land repairs after L2/PASSTHROUGH excision 2026-07-20 13:34:59 -07:00
adr_0244_qtop_vacuity feat(adr-0244): prove Q_top vacuity — D4 hollow-gate evidence + annotation upgrade 2026-07-17 14:39:52 -07:00
adr_0246_discrimination feat(adr-0246): Opus audit + §3.7 admit surface + serve wiring + §6.3 discrimination 2026-07-17 22:51:46 -07:00
adr_0246_geometric_suite draft(adr-0246): slice-1 scaffold — §6.1/§6.2 eval suite + malformed-F guard 2026-07-17 22:09:32 -07:00
adr_0246_grounding_feasibility feat(adr-0246): completion — §4.1 records, path serve integration, §11 feasibility (honest NULL), ADR body Proposed 2026-07-17 23:36:41 -07:00
adr_0246_mismatch_diagnostic feat(adr-0246): §3 induced-action primitives (pure, off-serving) 2026-07-17 21:36:57 -07:00
adversarial_identity
analogical_transfer feat(adr-0240/0243): seam S2 — chiral-composed, harness-driven biography write-path 2026-07-18 10:15:38 -07:00
anchor_lens_tour
anti_regression
articulation
articulation_of_status
audio_sensorium
audit_tour
calibration
capability_index chore(generalization): Tier S housekeeping — smoke promotion, capability-index entries, promotion sweep (S5) 2026-07-24 17:13:19 -07:00
classical_literature_ood
close_derived_climb
cognition fix(cognition): expose hash_surface; point the register-matrix pin at it 2026-07-25 14:06:36 +00:00
cold_start_grounding
combined_rate_oracle
compositionality
compound_intent_decomposition
comprehension
constraint_oracle
construction_inventory feat(evals): measure the reader/writer construction inventories against each other 2026-07-27 14:11:07 -07:00
contemplation_quality
contradiction_detection
conversation
conversational_thread_coherence
cross_domain_transfer
curriculum_loop_closure feat(deduction-serve): Band v6-EX — existential witnesses, decided (ADR-0261) 2026-07-24 14:14:37 -07:00
curriculum_serve feat(curriculum): taught negatives — polarity is read (ADR-0264 R1-R4, R8) 2026-07-26 12:19:19 -07:00
deduction_serve fix(evals): the deduction lane hashes the prose it serves 2026-07-26 19:45:48 -07:00
deductive_logic
demo_composition
determination_closure
determination_estimation
deterministic_fluency
dimensional
discourse_paragraph
domain_contract_validation
edge_budget
elementary_mathematics_ood
english_fluency_ood
environment_falsification
event_vision_sensorium
fabrication_control
field_incidence
flywheel_demo
forward_semantic_control
foundational_biology_ood
foundational_physics_ood
frame_verdict_text_cwa
frontier_compare
generalization
grammar_roundtrip feat(evals): measure the reader/writer construction inventories against each other 2026-07-27 14:11:07 -07:00
grammatical_coverage feat(evals): the fluency lanes report the writer that actually ships (Phase 4) 2026-07-27 11:57:48 -07:00
gsm8k_math test(reader-arc): pin deterministic tune/measure split BEFORE grammar work 2026-07-18 16:39:04 -07:00
gsm8k_parser_dev
hebrew_fluency
identity_divergence
industry_demos
inference_closure
introspection
koine_greek_fluency
l10_always_on
l10_continuity
lab
learning_arc
learning_loop
long_context_cost
math_bounded_grammar/v1
math_capability_axes
math_expert_claims/v1
math_symbolic_equivalence
math_teaching_corpus/v1
miner_loop_closure
monotonic_learning
multi_agent_composition
multi_sentence_response
multi_step_reasoning
obligation_2_ood_ratio
obligation_5_perturbation
obligation_6_depth_curve
obligation_8_adversarial
obligation_10_pack_provenance
orthogonality_tour
prompt_diversity
proof_carrying_promotion_demo
proofwriter_owa
propositional_logic
provenance
public_demo fix(ci): re-pin public_demo lane — real content drift, not the env flake (S2) 2026-07-24 16:58:05 -07:00
rate_oracle
realizer_guard
refusal_calibration
refusal_taxonomy
register_diagnostics
register_tour
relational
relational_inference/v1
relational_metric
relational_operator_ablation feat(eval): sealed relational operator ablation lane v1 2026-07-19 21:00:15 -07:00
relational_transitive
reports
results
reviewer_registry
sample_efficiency
self_consistency_over_time
sensorimotor_sensorium
sensorium
set_membership
setup_oracle
structure_mapping feat(trackb): S1–S4 symbolic SME, selector, pure-S1 coverage gain 2026-07-19 19:49:07 -07:00
syllogism
symbolic_logic
teaching_injection_resistance
total_ordering
vision_sensorium
walkthrough_chain
warmed_session_consistency
zero_code_domain_acquisition
__init__.py
_parallel.py
baseline_runner.py
CLAIMS.md
cognition_cases.jsonl
framework.py
generalized_lift_instrument.py chore(adr-0250): ratify Accepted + fold in PR #74 review findings 2026-07-18 15:07:29 -07:00
holdout_runner.py
lift_evidence_handoff.py feat(evals): seams S4+S5 — generalized-lift instrument + ports/handoff evidence 2026-07-18 10:34:26 -07:00
logic_cnf_compiler.py feat(adr-0249): P3 structural formula→CNF converter (deduction leg) 2026-07-18 12:40:31 -07:00
metrics.py fix(evals): carry hash_surface on the CaseResult the register matrix actually uses 2026-07-25 14:25:31 +00:00
multi_register_program.py feat(reader-arc): compare compiler-side guards for inverse + chained shapes 2026-07-18 17:14:16 -07:00
parallel.py
run_cognition_eval.py fix(evals): carry hash_surface on the CaseResult the register matrix actually uses 2026-07-25 14:25:31 +00:00
turn_program.py feat(adr-0249): P4 turn-program compiler + chained-relaxation executor 2026-07-18 12:55:31 -07:00
vocab_trigger_instrument.py feat(generalization): vocab-trigger instrument — mechanism-vs-coverage refusal histogram (S3) 2026-07-24 16:54:13 -07:00