core/evals
Shay 4426f387d1 feat(demo): ADR-0074 — orthogonality tour (anchor-lens × register)
A single demo that walks the full 3 × 3 × 2 matrix (register × lens
× prompts, 18 cells) and pins five claims simultaneously, packaging
both single-axis invariants into one composition gate.

The single-axis tours assert opposite invariants:

  register-tour    : per (lens, prompt), trace_hash CONSTANT across
                     registers (R5 / ADR-0072).
  anchor-lens-tour : per (register, prompt), engaged lens diverges
                     in trace_hash from the unanchored baseline
                     (L1.4 / ADR-0073d).

Orthogonality-tour packages both claims simultaneously across the
full matrix, plus three surface-level claims that pin the markers
operators actually see.

Composed claims (all five must hold)

  A) inner_register_invariant_within_lens
     For each (lens, prompt) cell, the three register runs share an
     identical trace_hash.  (R5 register-tour, applied 6 times:
     3 lenses × 2 prompts.)

  B) outer_lens_distinctness_within_register
     For each (register, prompt) cell where any non-unanchored lens
     engages, that engaged lens's trace_hash differs from the
     unanchored baseline at the same (register, prompt).
     (L1.4 anchor-lens-tour, applied 6 times: 3 registers × 2 prompts.)

  C) surface_carries_register_marker_under_convivial
     Every convivial cell with a non-empty surface has a non-empty
     register_variant_id.

  D) surface_carries_lens_annotation_when_engaged
     Every engaged cell carries [lens(<id>):<mode>] in surface AND
     a non-empty anchor_lens_mode_label.

  E) no_substrate_glyph_leak_across_grid
     No cell's surface contains Greek/Hebrew/Syriac/Arabic glyphs.
     (ADR-0073c gate re-asserted across the full matrix.)

CLI wiring

  core demo orthogonality-tour            human-readable grid + claims
  core demo orthogonality-tour --json     structured report

Exit code 0 iff all five claims hold.

Files

  evals/orthogonality_tour/__init__.py             NEW
  evals/orthogonality_tour/run_tour.py             NEW
  core/cli.py                                       EDIT
    - cmd_demo handler wires orthogonality-tour
    - demo choices + EPILOG examples updated
  tests/test_orthogonality_tour_demo.py             NEW (9 tests)
  docs/decisions/ADR-0074-orthogonality-tour.md     NEW

Sanity check baked into tests
  test_engaged_cells_appear_for_both_non_trivial_lenses pins that
  grc_logos_v1 engages on knowledge in all 3 registers (3 cells)
  and he_logos_v1 engages on truth in all 3 registers (3 cells).
  Prevents the lift claims being vacuously satisfied by a future
  engagement regression.

Lane evidence

  - 9 new orthogonality-tour tests pass.
  - core demo register-tour      → all_claims_supported: True
  - core demo anchor-lens-tour   → all_claims_supported: True
  - core demo orthogonality-tour → all_claims_supported: True
  - python -m core.cli eval cognition → byte-identical 100/100/91.7/100.
  - Full lane: 2745 passed / 4 skipped / 1 pre-existing failure
    (+9 over L1.4's 2736; the one failure remains
    test_all_preamble_explains_combined_run, unrelated).

No runtime / composer / loader / pack / schema changes.  Pure demo
consumer of existing telemetry contracts.
2026-05-19 20:33:33 -07:00
..
adversarial_identity docs(identity): empirical finding — fix #3 needs upstream ingest-gate work 2026-05-16 14:23:20 -07:00
anchor_lens_tour feat(anchor_lens): ADR-0073d — L1.4 telemetry, CLI flag, tour demo 2026-05-19 20:21:41 -07:00
anti_regression feat(adr-0057): anti-regression demo — three-gate defense against learning harm 2026-05-18 10:52:23 -07:00
articulation feat(demo): core demo articulation — discourse-planner spine, end-to-end 2026-05-19 13:41:24 -07:00
articulation_of_status feat(epistemic): realizer-side closure — refusal_calibration + articulation_of_status graduate 2026-05-17 10:12:59 -07:00
audit_tour feat(adr-0042): audit-tour demo — pack-layer story in four scenes 2026-05-17 22:06:45 -07:00
calibration fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
classical_literature_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
cognition feat(adr-0060): correction acknowledgement carries corrected-topic lemma 2026-05-18 14:14:27 -07:00
cold_start_grounding feat(evals+bench): isolation lanes, holdouts, planner-on bench sub-bench 2026-05-19 12:42:55 -07:00
compositionality feat(compositionality): compose_relations operator lifts lane 68.8% → 100% 2026-05-16 22:44:06 -07:00
compound_intent_decomposition feat(evals+bench): isolation lanes, holdouts, planner-on bench sub-bench 2026-05-19 12:42:55 -07:00
contradiction_detection feat(epistemic): contradiction coherence checker — CONTESTED transitions wired, last Tier 4.5 row closes 2026-05-17 10:36:48 -07:00
conversation feat(demo): humanise teaching-grounded surface for layperson display 2026-05-19 14:14:02 -07:00
conversational_thread_coherence feat(evals+bench): isolation lanes, holdouts, planner-on bench sub-bench 2026-05-19 12:42:55 -07:00
cross_domain_transfer feat(algebra): null-preserving versor_apply path + un-skip 2 invariant tests 2026-05-16 21:40:37 -07:00
deterministic_fluency feat(evals): deterministic_fluency lane — six structural predicates 2026-05-19 07:16:44 -07:00
discourse_paragraph feat(compositionality): compose_relations operator lifts lane 68.8% → 100% 2026-05-16 22:44:06 -07:00
elementary_mathematics_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
english_fluency_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
forward_semantic_control docs(cli): self-explanatory demos — preambles + per-directory READMEs 2026-05-17 16:39:50 -07:00
foundational_biology_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
foundational_physics_ood fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
grammatical_coverage fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
hebrew_fluency feat(phase5): land 5.2–5.7 — six new fluency lanes, parallel sweep 2026-05-16 20:59:31 -07:00
identity_divergence feat(adr-0043): Phase-2 pack measurements — claims → numbers 2026-05-17 22:19:24 -07:00
industry_demos fix(adr-0046): make forward-graph-constraint branch mergeable 2026-05-18 05:57:46 -07:00
inference_closure feat(algebra): null-preserving versor_apply path + un-skip 2 invariant tests 2026-05-16 21:40:37 -07:00
introspection feat(phase3): core/cognition/explain.py — close Gap 3 introspection 2026-05-16 15:09:48 -07:00
koine_greek_fluency feat(phase5): land 5.2–5.7 — six new fluency lanes, parallel sweep 2026-05-16 20:59:31 -07:00
learning_loop feat(adr-0064): cross-pack teaching chains + relations_chains_v1 seed (Phase 1.3+1.4) 2026-05-18 16:04:20 -07:00
long_context_cost feat(adr-0044, adr-0045): domain ethics pack + long-context comparison 2026-05-17 22:31:47 -07:00
monotonic_learning feat(evals): v3 lanes — monotonic-learning passes, adversarial-identity reveals gap 2026-05-16 13:42:47 -07:00
multi_agent_composition fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour 2026-05-16 21:21:06 -07:00
multi_sentence_response feat(evals+bench): isolation lanes, holdouts, planner-on bench sub-bench 2026-05-19 12:42:55 -07:00
multi_step_reasoning feat(algebra): null-preserving versor_apply path + un-skip 2 invariant tests 2026-05-16 21:40:37 -07:00
orthogonality_tour feat(demo): ADR-0074 — orthogonality tour (anchor-lens × register) 2026-05-19 20:33:33 -07:00
provenance feat(evals): parallel runner + adversarial-identity v2 2026-05-16 13:10:26 -07:00
refusal_calibration feat(adr-0043): Phase-2 pack measurements — claims → numbers 2026-05-17 22:19:24 -07:00
register_tour feat(register): R5 — operator-visible register telemetry + tour demo 2026-05-19 19:03:07 -07:00
reports feat(adr-0023): Forward Semantic Control proof evidence — Accepted 2026-05-17 12:55:19 -07:00
results chore(evals): refresh pack-measurements artifact to current runtime 2026-05-19 19:16:33 -07:00
sample_efficiency feat(phase4): sample-efficiency v1 — first quantitative-curve lane 2026-05-16 15:39:28 -07:00
self_consistency_over_time feat(runtime+evals): warm-path pack grounding + three long-span lanes 2026-05-19 08:26:38 -07:00
symbolic_logic feat(evals): v2 lanes for calibration and symbolic-logic 2026-05-16 13:17:41 -07:00
teaching_injection_resistance feat(epistemic): truth-seeking schema audit — 3 leaks closed, 4 new lanes, 3 new invariants 2026-05-17 07:27:41 -07:00
walkthrough_chain feat(evals+bench): isolation lanes, holdouts, planner-on bench sub-bench 2026-05-19 12:42:55 -07:00
warmed_session_consistency feat(evals+bench): isolation lanes, holdouts, planner-on bench sub-bench 2026-05-19 12:42:55 -07:00
zero_code_domain_acquisition feat began creation of zero code domain acquisition. did not complete yet. 2026-05-16 06:31:01 -07:00
__init__.py Add cognitive eval harness and calibration replay (#30) 2026-05-15 07:41:36 -07:00
baseline_runner.py feat(evals): frontier structural-zero baselines for Phase 2 v1 lanes 2026-05-16 12:45:28 -07:00
CLAIMS.md feat(bench): bench cost — $/1000 turns + latency, with disclosed assumptions 2026-05-17 10:53:08 -07:00
cognition_cases.jsonl feat: vault recall index, Rust versor parity, cognitive pack expansion 2026-05-15 15:34:39 -07:00
framework.py feat(adr-0054): vault recall indexing/batching + holdout split wired 2026-05-18 07:58:57 -07:00
holdout_runner.py feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
metrics.py Add cognitive eval harness and calibration replay (#30) 2026-05-15 07:41:36 -07:00
parallel.py feat(evals): parallel runner + adversarial-identity v2 2026-05-16 13:10:26 -07:00
run_cognition_eval.py Add cognitive eval harness and calibration replay (#30) 2026-05-15 07:41:36 -07:00