a447dce5d1
INV-25 makes the GSM8K lesson structural: no capability claim is valid unless its gold is computed by a procedure sharing no code with the system under test. Three meaningfully-failing checks (proven able to fail): 25a oracle imports no SUT module (AST); 25b every committed deductive gold is reproduced by the independent oracle AND matched by the engine; 25c an unsound engine disagrees on committed cases. SHA-pin the deductive lane (deductive_logic_v1, dev+holdout+external 716/716, wrong=0, refused=0) via a deterministic --report writer + run_as_module (the lane dir's local generate.py shadows the package in script mode) + CLAIMS regen. Fix drift the review surfaced: contract.md + pivot doc claimed an 8,000/7,340 fuzz and an 'external mirror' -> corrected to the real gated 3,000-case fuzz (2,796 definite) and 'hand-authored, NOT a published-benchmark mirror'; pivot doc GSM8K holdout 0->5 (all 5 are R1 reconstruction; composer scored 0). Validated: smoke 78, deductive 23, proof 29, invariants 44, lane-sha+claims 9. |
||
|---|---|---|
| .. | ||
| adversarial_identity | ||
| anchor_lens_tour | ||
| anti_regression | ||
| articulation | ||
| articulation_of_status | ||
| audio_sensorium | ||
| audit_tour | ||
| calibration | ||
| classical_literature_ood | ||
| cognition | ||
| cold_start_grounding | ||
| compositionality | ||
| compound_intent_decomposition | ||
| contemplation_quality | ||
| contradiction_detection | ||
| conversation | ||
| conversational_thread_coherence | ||
| cross_domain_transfer | ||
| curriculum_loop_closure | ||
| deductive_logic | ||
| demo_composition | ||
| deterministic_fluency | ||
| discourse_paragraph | ||
| domain_contract_validation | ||
| elementary_mathematics_ood | ||
| english_fluency_ood | ||
| fabrication_control | ||
| flywheel_demo | ||
| forward_semantic_control | ||
| foundational_biology_ood | ||
| foundational_physics_ood | ||
| frontier_compare | ||
| grammatical_coverage | ||
| gsm8k_math | ||
| gsm8k_parser_dev | ||
| hebrew_fluency | ||
| identity_divergence | ||
| industry_demos | ||
| inference_closure | ||
| introspection | ||
| koine_greek_fluency | ||
| lab | ||
| learning_arc | ||
| learning_loop | ||
| long_context_cost | ||
| math_bounded_grammar/v1 | ||
| math_capability_axes | ||
| math_expert_claims/v1 | ||
| math_symbolic_equivalence | ||
| math_teaching_corpus/v1 | ||
| miner_loop_closure | ||
| monotonic_learning | ||
| multi_agent_composition | ||
| multi_sentence_response | ||
| multi_step_reasoning | ||
| obligation_2_ood_ratio | ||
| obligation_5_perturbation | ||
| obligation_6_depth_curve | ||
| obligation_8_adversarial | ||
| obligation_10_pack_provenance | ||
| orthogonality_tour | ||
| prompt_diversity | ||
| provenance | ||
| public_demo | ||
| realizer_guard | ||
| refusal_calibration | ||
| refusal_taxonomy | ||
| register_diagnostics | ||
| register_tour | ||
| reports | ||
| results | ||
| reviewer_registry | ||
| sample_efficiency | ||
| self_consistency_over_time | ||
| sensorimotor_sensorium | ||
| sensorium | ||
| symbolic_logic | ||
| teaching_injection_resistance | ||
| vision_sensorium | ||
| walkthrough_chain | ||
| warmed_session_consistency | ||
| zero_code_domain_acquisition | ||
| __init__.py | ||
| _parallel.py | ||
| baseline_runner.py | ||
| CLAIMS.md | ||
| cognition_cases.jsonl | ||
| framework.py | ||
| holdout_runner.py | ||
| metrics.py | ||
| parallel.py | ||
| run_cognition_eval.py | ||