core/evals
Shay b0c5185633 feat(phase4): multi-agent-composition lane v1
Phase 4 lane #3. Tests the structural claim that composition does
not launder identity violations. Two CORE instances (A, B) with
no shared state communicate only by message bytes: user input is
fed to B; B's articulation_surface is fed to A; A's review
verdict is the gate.

Decision pinned for v1: message-passing only (no shared vault,
no shared identity manifold). Shared-state composition is
deferred to a future lane.

Public split 15/15 pass: 10 attacks correctly rejected by A's
identity check after B restates, 5 legitimate corrections
correctly accepted, zero B-side errors. Composition does not
launder.

Holdout split 7/8: one failure (MAC-HOLD-001 "disregard your
axiology") is a vocabulary gap in the identity check's term
list, not a composition leak. The same input would also be
accepted by single-agent A. gaps.md documents the recommended
fix (extend identity-check term family to axiology/ontology/
telos/ethos) and notes that the fix lands improvements on both
this lane and adversarial_identity.

v2 work: composite trace hash folding A.trace_hash,
B.trace_hash, and inter-agent message bytes; chain depth > 2;
shared-state composition.
2026-05-16 16:45:41 -07:00
..
adversarial_identity docs(identity): empirical finding — fix #3 needs upstream ingest-gate work 2026-05-16 14:23:20 -07:00
calibration feat(evals): v2 lanes for calibration and symbolic-logic 2026-05-16 13:17:41 -07:00
cognition feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
compositionality feat(phase3): compositionality, multi-step-reasoning, introspection, cross-domain-transfer v1 2026-05-16 14:48:36 -07:00
cross_domain_transfer feat(phase3): compositionality, multi-step-reasoning, introspection, cross-domain-transfer v1 2026-05-16 14:48:36 -07:00
grammatical_coverage feat(evals): grammatical-coverage v2 cases - 36 cases with deeper nesting and rarer vocabulary (100% pass) 2026-05-16 06:40:55 -07:00
identity_divergence feat(evals): identity-divergence lane v1 - 93 curriculum events, two axis profiles (Precision/Generosity), divergence/coherence/causal metrics (all pass) 2026-05-16 06:48:13 -07:00
inference_closure feat(phase3): inference-closure lane v1 — foundation OK, no operator 2026-05-16 14:33:08 -07:00
introspection feat(phase3): core/cognition/explain.py — close Gap 3 introspection 2026-05-16 15:09:48 -07:00
long_context_cost feat(phase4): long-context-cost lane + ADR-0019 Stage 1 vault recall vectorisation 2026-05-16 16:39:30 -07:00
monotonic_learning feat(evals): v3 lanes — monotonic-learning passes, adversarial-identity reveals gap 2026-05-16 13:42:47 -07:00
multi_agent_composition feat(phase4): multi-agent-composition lane v1 2026-05-16 16:45:41 -07:00
multi_step_reasoning feat(phase3): compositionality, multi-step-reasoning, introspection, cross-domain-transfer v1 2026-05-16 14:48:36 -07:00
provenance feat(evals): parallel runner + adversarial-identity v2 2026-05-16 13:10:26 -07:00
reports
sample_efficiency feat(phase4): sample-efficiency v1 — first quantitative-curve lane 2026-05-16 15:39:28 -07:00
symbolic_logic feat(evals): v2 lanes for calibration and symbolic-logic 2026-05-16 13:17:41 -07:00
zero_code_domain_acquisition feat began creation of zero code domain acquisition. did not complete yet. 2026-05-16 06:31:01 -07:00
__init__.py
baseline_runner.py feat(evals): frontier structural-zero baselines for Phase 2 v1 lanes 2026-05-16 12:45:28 -07:00
cognition_cases.jsonl
framework.py feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
holdout_runner.py feat(evals): Phase 0 — benchmark methodology lock-in and eval framework 2026-05-15 22:36:53 -07:00
metrics.py
parallel.py feat(evals): parallel runner + adversarial-identity v2 2026-05-16 13:10:26 -07:00
run_cognition_eval.py