Add vision evidence and sensorimotor contracts

docs/decisions/ADR-0208-environmental-sensorium-loop.md

@@ -0,0 +1,65 @@
+# ADR-0208: Environmental Sensorium Loop
+
+**Status:** Proposed
+**Date:** 2026-06-04
+**Domains:** `sensorium/environment/`, `sensorium/compiler/`, `sensorium/*`, future sensorimotor compilers
+**Depends on:** ADR-0013, ADR-0180, ADR-0181, ADR-0197, ADR-0198
+
+## Decision
+
+CORE will represent a moment of environmental evidence as an `ObservationFrame`:
+a deterministic bundle of already-compiled afferent `CompilationUnitLike`
+deltas. The frame is not a fusion layer, not a shared embedding space, and not a
+mutable world model.
+
+```text
+environment
+  -> modality compilers
+  -> compiled afferent units
+  -> ObservationFrame
+  -> Delta-CRDT merge
+  -> field / recall / cognition
+  -> governed efferent decode
+  -> action result / proprioception re-enters as afferent evidence
+```
+
+## Contract
+
+`ObservationFrame` contains:
+
+```text
+frame_id
+monotonic_tick
+source_clock
+units: tuple[CompilationUnitLike, ...]
+causal_parent_ids
+environment_sha256
+trace_hash
+```
+
+Rules:
+
+- Units are canonicalized by `merge_key` and exact duplicates deduplicate.
+- Trace records contain hashes and provenance only, never raw pixels, PCM, or
+  actuator payloads.
+- Audio chunks, vision tiles, text turns, and future proprioceptive feedback
+  remain native compilation units.
+- Motor commands and action traces are efferent; they are not observation units.
+- Action outcomes re-enter through afferent sensorimotor/proprioceptive
+  compilers.
+
+## Consequences
+
+This closes the architectural gap between independent modality compilers and an
+embodied environment loop without inventing late fusion. Cross-modal coherence
+is recovered after merge through exact manifold recall and field resonance. The
+hot path stays local and deterministic; fleet/offline aggregation remains a
+proposal/review path, not runtime truth.
+
+## Proof Obligations
+
+- Same afferent units in any arrival order produce the same frame trace hash.
+- Unsafe raw payloads fail before entering frame traces.
+- Efferent action records fail if passed as observation units.
+- Sensorimotor feedback can enter as afferent evidence without enabling motor
+  emission.

docs/decisions/ADR-0209-sensorimotor-feedback-contract.md

@@ -0,0 +1,70 @@
+# ADR-0209: Sensorimotor Feedback Is Afferent
+
+**Status:** Proposed
+**Date:** 2026-06-04
+**Domains:** `sensorium/sensorimotor/`, `sensorium/protocol.py`, future robotics integrations
+**Depends on:** ADR-0013, ADR-0198, ADR-0208
+
+## Decision
+
+CORE will treat proprioception, tactile/contact state, actuator state feedback,
+and action result evidence as **afferent sensorimotor input**. Motor commands
+remain **efferent** and are governed separately by `EfferentGate`.
+
+```text
+proprioception / contact / actuator feedback
+  -> sensorimotor compiler
+  -> SensorimotorCompilationUnit
+  -> ObservationFrame
+
+field action intent
+  -> EfferentGate + AuthorityToken
+  -> governed decode / refusal
+  -> environment effect
+  -> result feedback re-enters as sensorimotor input
+```
+
+## Contract
+
+The v1 afferent signal is quantized and replayable:
+
+```text
+ProprioceptiveSignal
+  pose_q
+  velocity_q
+  force_torque_q
+  contact_q
+  actuator_state_q
+  source_sha256
+  canonical_sha256
+```
+
+The compiler emits:
+
+```text
+SensorimotorIR
+SensorimotorCompilationUnit
+ContentAddressedDelta
+```
+
+No decoder, trajectory executor, actuator driver, robot interface, tool call, or
+skill invocation is introduced by this contract.
+
+## Consequences
+
+This reserves the correct robotics shape without making unsafe action emission
+look like ordinary perception. A robot can later close the loop through
+environment orchestration, but the two halves remain type-separated:
+
+- sensorimotor feedback is evidence;
+- motor command is authorized action;
+- action results become new evidence only after they re-enter through an
+  afferent compiler.
+
+## Proof Obligations
+
+- Same canonical proprioceptive signal produces identical unit and merge key.
+- IR replay reproduces the projection.
+- Sensorimotor deltas merge idempotently.
+- Sensorimotor compiler exposes no decode path.
+- Trace records contain no command or trajectory payload.

evals/vision_sensorium/__init__.py

@@ -0,0 +1,2 @@
+"""Deterministic vision compiler eval lane."""
+

evals/vision_sensorium/expected_ir.jsonl

@@ -0,0 +1,9 @@
+{"canonical_sha256": "2539e3b2ac17082d709bf08d8b93b80ea3d6d8104bb18f93a3c3579a27272770", "event_type_counts": {"orient.edge_energy": 5, "region.chroma": 5, "region.flat": 5, "salient.figure_ground": 5, "texture.regularity": 5}, "id": "flat_gray", "unit_count": 5, "unit_ir_sha256": ["1805976d658a70c473365ba1825ce51230bb527a149ae07f0aa1fd007db6975b", "76c82a15ea0f67cd612679c04025b5a91adf8349e8e2e0e909d3b15bfd4b1250", "f593b65fee41df238fe74ede402f6695b3c2975b2b3d3156d6f8b47844ef5ac2", "2e6e3d938cdb22a8bcab977d54353ee944ce4b589f8024ba3beafdeb182fefac", "5df381dd3a8ae41050f344393b79756e395e3da63e3b1cf3a2fce05ee1e93a44"]}
+{"canonical_sha256": "de941bccf3aab2c76c4c513a6b7745eba39b299ee708d7ee1c1f02c81af558b1", "event_type_counts": {"orient.edge_energy": 5, "region.chroma": 5, "region.contrast": 1, "region.flat": 4, "salient.figure_ground": 5, "texture.regularity": 5}, "id": "vertical_edge", "unit_count": 5, "unit_ir_sha256": ["c0bb3e2c3606df5d9f430d8c4b87a279bb2068df8534e1ae2e2cf2417b75523a", "3264a01a8291126b02c29dbaffa9c8f1994e7c035c1ca8f2d4ff82913be57b3c", "dfca4c49efa1041140fb48959136cf93485ad91b53c47a4813705563068221d4", "406af11b33f6aec73b398e3daf283aaf3271c4374895484582997c1fcf9cc888", "a41ec8ce38f9b96c1487da31551c078120d87ae5c65a1c6ab7498413aa98f2d5"]}
+{"canonical_sha256": "81ebbbf4e07009bba424159810e06274bb453a1cfa4221187c14287db77d6f0b", "event_type_counts": {"contour.closure": 2, "orient.edge_energy": 5, "region.blob": 1, "region.chroma": 5, "region.contrast": 2, "region.corner": 2, "region.flat": 3, "salient.figure_ground": 5, "texture.regularity": 5}, "id": "corner_block", "unit_count": 5, "unit_ir_sha256": ["8559deef8b525dc9b9658f2608d71034fbeb4f72198c3655618df92995f12426", "3264a01a8291126b02c29dbaffa9c8f1994e7c035c1ca8f2d4ff82913be57b3c", "3c75d4ac5655de36c25ac42f52b6abe8ee795bad6494c24f706758031ddbd554", "406af11b33f6aec73b398e3daf283aaf3271c4374895484582997c1fcf9cc888", "3594727932503362bb78893ca8b22b2841e60f7d05590f3b5f885a171dcf0539"]}
+{"canonical_sha256": "3d6bd7f67919d37eb9a3f2222c9c91dac6b47df5a904edf50c67072016c45ca0", "event_type_counts": {"contour.closure": 5, "orient.edge_energy": 5, "region.blob": 5, "region.chroma": 5, "region.contrast": 5, "region.corner": 5, "salient.figure_ground": 5, "texture.regularity": 5}, "id": "center_blob", "unit_count": 5, "unit_ir_sha256": ["f85e1a041b6d1b295000e52b57fdfdfbaaf5480567e9944f11a198516276d0e8", "a783fcd737812546c82e8f840b5987bbf93f427dba0e5a1c98b273520349a97f", "e38b05b3418a97cd19ece411d90cb5af0a836c7a35214ce96e62813c0d81cf3e", "c65af4c1c9ec7d85ea8673b08a71d8f7005ea538f6aab96ef44a42dd84928af7", "2c49e77da998e90693aa25f41d495c1cff369e5ba7e49805bf16acf9290dc44a"]}
+{"canonical_sha256": "f5ac89e012cc68167c370552ef3e3058d8b1ef0758e4e4b8943cb88e2b787bef", "event_type_counts": {"contour.closure": 5, "orient.edge_energy": 5, "region.chroma": 5, "region.contrast": 5, "region.corner": 5, "salient.figure_ground": 5, "texture.regularity": 5}, "id": "checker_texture", "unit_count": 5, "unit_ir_sha256": ["250503203675980bfb964ea8ba5b8dca48bd4b394b9dcfb92a83578858f2e9e3", "255a2331c646ff561851de1f852e0850439b9d2dbdf3828a07ccb30f23fd9946", "2f76219af0dded3845f28400ec731db0b7ca94b47a75cea109b7aa9c468130b0", "e13b7ea49cd608d717c1f8f19df5805f8e3b3d079420a088808e46eb48379d6d", "2c5084229cece80f8fdea81f056847e67dfdfea5b287eb7234380a46a2da66c9"]}
+{"canonical_sha256": "a91aa64832aa6d75e5631b54de97b5d620def288b2cce898f09535922dcb6b1e", "event_type_counts": {"orient.edge_energy": 5, "region.chroma": 5, "region.contrast": 5, "salient.figure_ground": 5, "texture.regularity": 5}, "id": "contrast_ramp", "unit_count": 5, "unit_ir_sha256": ["bf5b935e984489baa72563f26b18a8106deda4e8be1ed84bf211397a228624ab", "14f0dc93ee144bc57264873248a3f6bd137c06cc0fd14d99bc7d9aa494a705e8", "ad165740c173f9599ea40b2cdfb23bb3d961aaa04c06744d7cabf035398f561a", "356fd042b284604030dbe81965df66a88dc4ad5b7a8d721013f2c039c9080fc5", "a9b48462e0c9133cfe994eec25a5629656013436bcbeb307b82a9200467b3355"]}
+{"canonical_sha256": "211bf8d52098a28d425fa604a60f2c4a2d1e09ad9451f23b08b9cd34de6c70b9", "event_type_counts": {"orient.edge_energy": 5, "region.chroma": 5, "region.contrast": 1, "region.flat": 4, "salient.figure_ground": 5, "texture.regularity": 5}, "id": "chroma_split", "unit_count": 5, "unit_ir_sha256": ["093e51212fcf40dd10d3085906772d20a9eac52a93895ad0b8698bf08b65ed7a", "61991cf7ad68437449727e2dcd11d47828962a177683b4fcee78fc71449342ea", "294368f2a968e4b07718a32bae1b0c0b3691d136f96cbea8c20b08a00063e571", "99ee396623df3187039256625aa23ff56fb95056f74a639380d4434dd3919e25", "d63532072fcafb030a0483a4df69d9100ff7e90c24b9f7cc73d9dc0bd6d84354"]}
+{"canonical_sha256": "8022d2e39e825baae1e516e13ee44cdb8aee7960a4cf29e3267ae8c3f177fa7c", "event_type_counts": {"orient.edge_energy": 5, "region.blob": 5, "region.chroma": 5, "region.contrast": 5, "salient.figure_ground": 5, "texture.regularity": 5}, "id": "salient_spot", "unit_count": 5, "unit_ir_sha256": ["6adb1086d482d152c6f13e1e04cd90e3d06c03bd2ebca6a9460a9dc27b8ef649", "fbf6301c26f35fbb1c2775b2aa7e679872af555c33769155c70051a312c687c1", "90965d7e5597559fd92315a54a463b4f0980269c40f18d4de4279059c89c993e", "451a0b869e8c7ac483fed4b9909187ab590eb8ad771d3fdb0888a6034809cf10", "be45a2c65e350585b627903bdca98be2fc2a069d8461b1d2ceee6081a3f90eff"]}
+{"canonical_sha256": "280d4b12edb93ff93573e059af9ffcc8a3dcb5aad8d7de62e4403754205b8cb8", "event_type_counts": {"contour.closure": 5, "orient.edge_energy": 5, "region.blob": 5, "region.chroma": 5, "region.contrast": 5, "region.corner": 5, "salient.figure_ground": 5, "texture.regularity": 5}, "id": "contour_box", "unit_count": 5, "unit_ir_sha256": ["a586f1d7aa07f157fd3b2b1c7c85b4585fc77aa9bbb64f8515db3e5f1f99c38c", "f7c65d09469fcb2fe6bf90f399eb97607dfb0e634accd85117c6f8e90a0480b5", "118716ded59c86aa2eef07ec31e174c8c5d630454ba0a1e5becd0abfe8aa6af0", "b04d9b186b13fc06c73f5928daf8224b2a0fdd3b75558ec9b7fdb8c7d0903bff", "05981a687df449108ce2382a2e1c230008d900876ca85474c420d4c9c798e6bb"]}

evals/vision_sensorium/fixtures.json

@@ -0,0 +1,24 @@
+{
+  "size": 32,
+  "comment": "Deterministic visual synthesis specs. Fixtures are designed with predictable measured facts so the gate grades lexer/parser semantics as well as determinism.",
+  "fixtures": [
+    {"id": "flat_gray", "kind": "flat", "rgb": [0.5, 0.5, 0.5],
+     "expect": "flat low-contrast field"},
+    {"id": "vertical_edge", "kind": "edge", "orientation": "vertical",
+     "expect": "hard oriented edge and contrast"},
+    {"id": "corner_block", "kind": "corner",
+     "expect": "corner/junction response"},
+    {"id": "center_blob", "kind": "blob",
+     "expect": "center blob / region onset"},
+    {"id": "checker_texture", "kind": "checker", "period": 4,
+     "expect": "high-frequency texture"},
+    {"id": "contrast_ramp", "kind": "ramp",
+     "expect": "luminance contrast gradient"},
+    {"id": "chroma_split", "kind": "chroma_split",
+     "expect": "strong chroma regime"},
+    {"id": "salient_spot", "kind": "salient_spot",
+     "expect": "salient figure on ground"},
+    {"id": "contour_box", "kind": "contour_box",
+     "expect": "closed contour-like border"}
+  ]
+}

evals/vision_sensorium/generate_expected.py

@@ -0,0 +1,68 @@
+"""Regenerate frozen expected artifacts for the vision eval lane."""
+
+from __future__ import annotations
+
+import json
+from collections import Counter
+from pathlib import Path
+
+from evals.vision_sensorium.synth import synthesize
+from sensorium.vision import VisionCompiler, canonicalize_image
+from sensorium.vision.types import VisionIR
+
+_HERE = Path(__file__).resolve().parent
+
+
+def event_type_counts(ir: VisionIR) -> dict[str, int]:
+    events = (
+        *ir.regions,
+        *ir.contour_arcs,
+        *ir.orient_events,
+        *ir.texture_atoms,
+        *ir.salient_events,
+        *ir.content_anchors,
+    )
+    return dict(sorted(Counter(e.event_type for e in events).items()))
+
+
+def main() -> None:
+    spec = json.loads((_HERE / "fixtures.json").read_text())
+    compiler = VisionCompiler()
+    ir_lines: list[str] = []
+    projection: dict[str, list[dict]] = {}
+
+    for fx in spec["fixtures"]:
+        image = canonicalize_image(synthesize(fx), size=int(spec["size"]))
+        units = compiler.compile_image(image)
+        counts = Counter()
+        for unit in units:
+            counts.update(event_type_counts(unit.vision_ir))
+        ir_lines.append(json.dumps({
+            "id": fx["id"],
+            "canonical_sha256": image.canonical_sha256,
+            "unit_count": len(units),
+            "unit_ir_sha256": [unit.ir_sha256 for unit in units],
+            "event_type_counts": dict(sorted(counts.items())),
+        }, sort_keys=True))
+        projection[fx["id"]] = [
+            {
+                "coord": {
+                    "scale_level": unit.coord.scale_level,
+                    "tile_row": unit.coord.tile_row,
+                    "tile_col": unit.coord.tile_col,
+                },
+                "projection_sha256": unit.projection_sha256,
+                "reference_versor": [float(x) for x in unit.versor.tolist()],
+            }
+            for unit in units
+        ]
+
+    (_HERE / "expected_ir.jsonl").write_text("\n".join(ir_lines) + "\n")
+    (_HERE / "expected_projection.json").write_text(
+        json.dumps(projection, indent=2, sort_keys=True) + "\n"
+    )
+    print(f"wrote expected artifacts for {len(spec['fixtures'])} fixtures")
+
+
+if __name__ == "__main__":
+    main()

evals/vision_sensorium/synth.py

@@ -0,0 +1,66 @@
+"""Deterministic synthetic image fixtures for vision_core_v1."""
+
+from __future__ import annotations
+
+import numpy as np
+
+SIZE = 32
+
+
+def _flat(rgb: list[float], size: int) -> np.ndarray:
+    out = np.zeros((size, size, 3), dtype=np.float32)
+    out[:, :, :] = np.asarray(rgb, dtype=np.float32)
+    return out
+
+
+def synthesize(spec: dict) -> np.ndarray:
+    """Return a float32 RGB image for a fixture spec."""
+    size = int(spec.get("size", SIZE))
+    kind = spec["kind"]
+    if kind == "flat":
+        return _flat(list(spec.get("rgb", [0.5, 0.5, 0.5])), size)
+    if kind == "edge":
+        out = _flat([0.15, 0.15, 0.15], size)
+        if spec.get("orientation") == "horizontal":
+            out[size // 2:, :, :] = 0.9
+        else:
+            out[:, size // 2:, :] = 0.9
+        return out
+    if kind == "corner":
+        out = _flat([0.1, 0.1, 0.1], size)
+        out[4:16, 4:7, :] = 0.95
+        out[4:7, 4:16, :] = 0.95
+        out[11:16, 11:16, :] = 0.75
+        return out
+    if kind == "blob":
+        out = _flat([0.2, 0.2, 0.2], size)
+        yy, xx = np.mgrid[:size, :size]
+        mask = (xx - size / 2) ** 2 + (yy - size / 2) ** 2 <= (size / 5) ** 2
+        out[mask, :] = 0.95
+        return out.astype(np.float32)
+    if kind == "checker":
+        period = int(spec.get("period", 4))
+        yy, xx = np.mgrid[:size, :size]
+        mask = ((xx // period) + (yy // period)) % 2
+        out = np.repeat(mask[:, :, None].astype(np.float32), 3, axis=2)
+        return out
+    if kind == "ramp":
+        x = np.linspace(0.0, 1.0, size, dtype=np.float32)
+        ramp = np.repeat(x[None, :, None], size, axis=0)
+        return np.repeat(ramp, 3, axis=2)
+    if kind == "chroma_split":
+        out = _flat([0.1, 0.1, 0.8], size)
+        out[:, size // 2:, :] = np.asarray([0.9, 0.15, 0.1], dtype=np.float32)
+        return out
+    if kind == "salient_spot":
+        out = _flat([0.45, 0.45, 0.45], size)
+        out[size // 2 - 3:size // 2 + 3, size // 2 - 3:size // 2 + 3, :] = 1.0
+        return out
+    if kind == "contour_box":
+        out = _flat([0.2, 0.2, 0.2], size)
+        out[7:25, 7:11, :] = 1.0
+        out[7:25, 21:25, :] = 1.0
+        out[7:11, 7:25, :] = 1.0
+        out[21:25, 7:25, :] = 1.0
+        return out
+    raise ValueError(f"unknown vision fixture kind: {kind!r}")

sensorium/environment/__init__.py

@@ -0,0 +1,5 @@
+"""Environmental observation contracts for sensorium units."""
+
+from sensorium.environment.frame import ObservationFrame, build_observation_frame
+
+__all__ = ["ObservationFrame", "build_observation_frame"]

sensorium/environment/frame.py

@@ -0,0 +1,107 @@
+"""Deterministic environmental observation frames.
+
+An ObservationFrame is a traceable bundle of already-compiled afferent units.
+It is not a fusion layer and it never accepts efferent action commands as
+observation evidence.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Iterable
+
+import numpy as np
+
+from sensorium.audio.checksum import sha256_json
+from sensorium.compiler.protocol import CompilationUnitLike, MergeKey
+
+_UNSAFE_ATTRS = ("pixels", "samples", "pcm", "waveform", "raw_bytes", "action_trace")
+
+
+def _reject_unsafe_unit(unit: CompilationUnitLike) -> None:
+    if bool(getattr(unit, "efferent", False)):
+        raise ValueError("efferent action traces are not afferent observation units")
+    if str(getattr(unit, "pack_id", "")).startswith("motor"):
+        raise ValueError("motor/efferent packs are not observation units")
+    for attr in _UNSAFE_ATTRS:
+        if hasattr(unit, attr):
+            value = getattr(unit, attr)
+            if isinstance(value, (np.ndarray, bytes, bytearray)):
+                raise TypeError(f"unsafe observation payload on unit: {attr}")
+
+
+def _unit_record(unit: CompilationUnitLike) -> dict[str, object]:
+    _reject_unsafe_unit(unit)
+    return {
+        "merge_key": list(unit.merge_key),
+        "canonical_sha256": unit.canonical_sha256,
+        "ir_sha256": unit.ir_sha256,
+        "pack_id": unit.pack_id,
+        "pack_manifest_sha256": unit.pack_manifest_sha256,
+        "projection_sha256": unit.projection_sha256,
+        "versor_condition": float(unit.versor_condition),
+    }
+
+
+def _canonical_units(units: Iterable[CompilationUnitLike]) -> tuple[CompilationUnitLike, ...]:
+    ordered = sorted(tuple(units), key=lambda u: u.merge_key)
+    deduped: list[CompilationUnitLike] = []
+    last_key: MergeKey | None = None
+    for unit in ordered:
+        if unit.merge_key != last_key:
+            deduped.append(unit)
+            last_key = unit.merge_key
+    return tuple(deduped)
+
+
+@dataclass(frozen=True, slots=True)
+class ObservationFrame:
+    """A deterministic environmental slice over afferent compiled units."""
+
+    frame_id: str
+    monotonic_tick: int
+    source_clock: str
+    units: tuple[CompilationUnitLike, ...]
+    causal_parent_ids: tuple[str, ...]
+    environment_sha256: str
+    trace_hash: str
+
+
+def build_observation_frame(
+    *,
+    monotonic_tick: int,
+    source_clock: str,
+    units: Iterable[CompilationUnitLike],
+    causal_parent_ids: tuple[str, ...] = (),
+) -> ObservationFrame:
+    if monotonic_tick < 0:
+        raise ValueError("monotonic_tick must be non-negative")
+    canonical_units = _canonical_units(units)
+    unit_records = [_unit_record(unit) for unit in canonical_units]
+    env_payload = {
+        "monotonic_tick": int(monotonic_tick),
+        "source_clock": str(source_clock),
+        "causal_parent_ids": list(causal_parent_ids),
+        "unit_records": unit_records,
+    }
+    environment_sha256 = sha256_json(env_payload)
+    trace_hash = sha256_json({
+        "kind": "ObservationFrame",
+        "environment_sha256": environment_sha256,
+        "unit_merge_keys": [record["merge_key"] for record in unit_records],
+    })
+    frame_id = sha256_json({
+        "kind": "ObservationFrame.id",
+        "monotonic_tick": int(monotonic_tick),
+        "source_clock": str(source_clock),
+        "trace_hash": trace_hash,
+    })
+    return ObservationFrame(
+        frame_id=frame_id,
+        monotonic_tick=int(monotonic_tick),
+        source_clock=str(source_clock),
+        units=canonical_units,
+        causal_parent_ids=tuple(causal_parent_ids),
+        environment_sha256=environment_sha256,
+        trace_hash=trace_hash,
+    )

sensorium/protocol.py

@@ -42,6 +42,7 @@ class Modality(str, Enum):
     TEXT   = "text"
     VISION = "vision"
     AUDIO  = "audio"
+    SENSORIMOTOR = "sensorimotor"
     MOTOR  = "motor"
 
 

sensorium/sensorimotor/__init__.py

@@ -0,0 +1,30 @@
+"""Afferent sensorimotor / proprioceptive compiler contract."""
+
+from sensorium.sensorimotor.arena import (
+    SensorimotorArena,
+    SensorimotorDelta,
+    merge_sensorimotor_deltas,
+    sensorimotor_merge_trace_hash,
+)
+from sensorium.sensorimotor.compiler import SensorimotorCompiler, canonicalize_proprioception
+from sensorium.sensorimotor.trace import sensorimotor_evidence_trace
+from sensorium.sensorimotor.types import (
+    ProprioceptiveSignal,
+    SensorimotorCompilationUnit,
+    SensorimotorEvent,
+    SensorimotorIR,
+)
+
+__all__ = [
+    "ProprioceptiveSignal",
+    "SensorimotorArena",
+    "SensorimotorCompilationUnit",
+    "SensorimotorCompiler",
+    "SensorimotorDelta",
+    "SensorimotorEvent",
+    "SensorimotorIR",
+    "canonicalize_proprioception",
+    "merge_sensorimotor_deltas",
+    "sensorimotor_evidence_trace",
+    "sensorimotor_merge_trace_hash",
+]

sensorium/sensorimotor/arena.py

@@ -0,0 +1,60 @@
+"""Sensorimotor CRDT wrappers."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from sensorium.compiler.arena import LocalArena
+from sensorium.compiler.delta import ContentAddressedDelta, merge_deltas
+from sensorium.compiler.trace import merge_trace_hash
+from sensorium.sensorimotor.trace import sensorimotor_evidence_trace
+from sensorium.sensorimotor.types import SensorimotorCompilationUnit
+
+
+@dataclass(frozen=True, slots=True)
+class SensorimotorDelta:
+    _inner: ContentAddressedDelta[SensorimotorCompilationUnit]
+
+    @classmethod
+    def from_units(
+        cls,
+        units: tuple[SensorimotorCompilationUnit, ...] | list[SensorimotorCompilationUnit],
+    ) -> "SensorimotorDelta":
+        return cls(ContentAddressedDelta.from_units(units))
+
+    @property
+    def units(self) -> tuple[SensorimotorCompilationUnit, ...]:
+        return self._inner.units
+
+    def join(self, other: "SensorimotorDelta") -> "SensorimotorDelta":
+        return SensorimotorDelta(self._inner.join(other._inner))
+
+    @property
+    def merge_keys(self) -> tuple[tuple[str, str, str], ...]:
+        return self._inner.merge_keys
+
+    def __len__(self) -> int:
+        return len(self._inner)
+
+
+class SensorimotorArena:
+    __slots__ = ("_arena",)
+
+    def __init__(self) -> None:
+        self._arena: LocalArena[SensorimotorCompilationUnit] = LocalArena()
+
+    def push(self, unit: SensorimotorCompilationUnit) -> None:
+        self._arena.push(unit)
+
+    def snapshot(self) -> SensorimotorDelta:
+        return SensorimotorDelta(self._arena.snapshot())
+
+
+def merge_sensorimotor_deltas(
+    deltas: list[SensorimotorDelta] | tuple[SensorimotorDelta, ...],
+) -> SensorimotorDelta:
+    return SensorimotorDelta(merge_deltas(delta._inner for delta in deltas))
+
+
+def sensorimotor_merge_trace_hash(delta: SensorimotorDelta) -> str:
+    return merge_trace_hash(delta._inner, sensorimotor_evidence_trace)

sensorium/sensorimotor/compiler.py

@@ -0,0 +1,166 @@
+"""Deterministic afferent sensorimotor compiler."""
+
+from __future__ import annotations
+
+import math
+
+import numpy as np
+
+from algebra.cl41 import geometric_product
+from algebra.versor import unitize_versor, versor_condition
+from sensorium.audio.checksum import sha256_array, sha256_json
+from sensorium.sensorimotor.types import (
+    ProprioceptiveSignal,
+    SensorimotorCompilationUnit,
+    SensorimotorEvent,
+    SensorimotorIR,
+)
+
+CL41_DIM = 32
+VERSOR_CONDITION_MAX = 1e-6
+THETA_STEP = math.pi / 512.0
+_EVENT_ORDER = (
+    "proprio.pose",
+    "proprio.velocity",
+    "haptic.force_torque",
+    "haptic.contact",
+    "actuator.state",
+)
+_BLADE_BY_EVENT = {
+    "proprio.pose": 6,
+    "proprio.velocity": 7,
+    "haptic.force_torque": 8,
+    "haptic.contact": 10,
+    "actuator.state": 11,
+}
+_BASE_BY_EVENT = {
+    "proprio.pose": 48,
+    "proprio.velocity": 64,
+    "haptic.force_torque": 80,
+    "haptic.contact": 96,
+    "actuator.state": 112,
+}
+
+
+def canonicalize_proprioception(
+    *,
+    pose_q: tuple[int, ...] = (),
+    velocity_q: tuple[int, ...] = (),
+    force_torque_q: tuple[int, ...] = (),
+    contact_q: tuple[int, ...] = (),
+    actuator_state_q: tuple[int, ...] = (),
+    source_id: str = "",
+) -> ProprioceptiveSignal:
+    payload = {
+        "pose_q": list(pose_q),
+        "velocity_q": list(velocity_q),
+        "force_torque_q": list(force_torque_q),
+        "contact_q": list(contact_q),
+        "actuator_state_q": list(actuator_state_q),
+        "source_id": source_id,
+    }
+    source_sha256 = sha256_json(payload)
+    canonical_payload = {k: payload[k] for k in payload if k != "source_id"}
+    return ProprioceptiveSignal(
+        pose_q=tuple(int(v) for v in pose_q),
+        velocity_q=tuple(int(v) for v in velocity_q),
+        force_torque_q=tuple(int(v) for v in force_torque_q),
+        contact_q=tuple(int(v) for v in contact_q),
+        actuator_state_q=tuple(int(v) for v in actuator_state_q),
+        source_sha256=source_sha256,
+        canonical_sha256=sha256_json(canonical_payload),
+    )
+
+
+def _event(event_type: str, values: tuple[int, ...]) -> SensorimotorEvent:
+    attrs = tuple((f"q{idx}", int(value)) for idx, value in enumerate(values))
+    return SensorimotorEvent(event_type, attrs, ())
+
+
+def _parse(signal: ProprioceptiveSignal) -> SensorimotorIR:
+    events = (
+        _event("proprio.pose", signal.pose_q),
+        _event("proprio.velocity", signal.velocity_q),
+        _event("haptic.force_torque", signal.force_torque_q),
+        _event("haptic.contact", signal.contact_q),
+        _event("actuator.state", signal.actuator_state_q),
+    )
+    payload = [
+        {
+            "event_type": ev.event_type,
+            "attrs": [list(pair) for pair in ev.attrs],
+            "evidence_ids": list(ev.evidence_ids),
+        }
+        for ev in events
+    ]
+    return SensorimotorIR(events, sha256_json({"events": payload}))
+
+
+def _build_rotor(blade_index: int, theta_q: int) -> np.ndarray:
+    out = np.zeros(CL41_DIM, dtype=np.float64)
+    half = (theta_q * THETA_STEP) / 2.0
+    out[0] = math.cos(half)
+    out[blade_index] = math.sin(half)
+    return out
+
+
+def _theta_q(event: SensorimotorEvent) -> int:
+    total = sum(abs(int(value)) for _, value in event.attrs if isinstance(value, int))
+    return max(0, min(768, _BASE_BY_EVENT[event.event_type] + total))
+
+
+def compile_events(events: tuple[SensorimotorEvent, ...]) -> tuple[np.ndarray, float]:
+    rank = {name: idx for idx, name in enumerate(_EVENT_ORDER)}
+    v = np.zeros(CL41_DIM, dtype=np.float64)
+    v[0] = 1.0
+    for event in sorted(events, key=lambda ev: rank[ev.event_type]):
+        r = _build_rotor(_BLADE_BY_EVENT[event.event_type], _theta_q(event))
+        v = geometric_product(v, r)
+        v = unitize_versor(v)
+    vc = float(versor_condition(v))
+    if vc >= VERSOR_CONDITION_MAX:
+        raise ValueError(
+            f"sensorimotor compilation failed versor check: {vc:.3e} >= {VERSOR_CONDITION_MAX:.0e}"
+        )
+    return v.astype(np.float32), vc
+
+
+class SensorimotorCompiler:
+    """Compiler for afferent proprioceptive feedback only."""
+
+    modality = "sensorimotor"
+
+    def __init__(self, pack_id: str = "sensorimotor_core_v1") -> None:
+        self._pack_id = pack_id
+        self._manifest_sha256 = sha256_json({
+            "pack_id": pack_id,
+            "basis_version": "sensorimotor-basis-v1",
+            "events": list(_EVENT_ORDER),
+        })
+
+    def compile_signal(self, signal: ProprioceptiveSignal) -> SensorimotorCompilationUnit:
+        ir = _parse(signal)
+        versor, vc = compile_events(ir.events)
+        return SensorimotorCompilationUnit(
+            canonical_sha256=signal.canonical_sha256,
+            ir_sha256=ir.ir_sha256,
+            pack_id=self._pack_id,
+            pack_manifest_sha256=self._manifest_sha256,
+            projection_sha256=sha256_array(versor),
+            versor=versor,
+            versor_condition=vc,
+            sensorimotor_ir=ir,
+        )
+
+    def compile_ir(self, ir: SensorimotorIR) -> SensorimotorCompilationUnit:
+        versor, vc = compile_events(ir.events)
+        return SensorimotorCompilationUnit(
+            canonical_sha256="",
+            ir_sha256=ir.ir_sha256,
+            pack_id=self._pack_id,
+            pack_manifest_sha256=self._manifest_sha256,
+            projection_sha256=sha256_array(versor),
+            versor=versor,
+            versor_condition=vc,
+            sensorimotor_ir=ir,
+        )

sensorium/sensorimotor/trace.py

@@ -0,0 +1,18 @@
+"""Trace-safe sensorimotor evidence."""
+
+from __future__ import annotations
+
+from sensorium.sensorimotor.types import SensorimotorCompilationUnit
+
+
+def sensorimotor_evidence_trace(unit: SensorimotorCompilationUnit) -> dict[str, object]:
+    return {
+        "modality": "sensorimotor",
+        "pack_id": unit.pack_id,
+        "canonical_sha256": unit.canonical_sha256,
+        "ir_sha256": unit.ir_sha256,
+        "pack_manifest_sha256": unit.pack_manifest_sha256,
+        "projection_sha256": unit.projection_sha256,
+        "merge_key": list(unit.merge_key),
+        "versor_condition": unit.versor_condition,
+    }

sensorium/sensorimotor/types.py

@@ -0,0 +1,47 @@
+"""Typed sensorimotor IR for afferent proprioceptive feedback."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+import numpy as np
+
+
+@dataclass(frozen=True, slots=True)
+class ProprioceptiveSignal:
+    pose_q: tuple[int, ...]
+    velocity_q: tuple[int, ...]
+    force_torque_q: tuple[int, ...]
+    contact_q: tuple[int, ...]
+    actuator_state_q: tuple[int, ...]
+    source_sha256: str
+    canonical_sha256: str
+
+
+@dataclass(frozen=True, slots=True)
+class SensorimotorEvent:
+    event_type: str
+    attrs: tuple[tuple[str, int | str], ...]
+    evidence_ids: tuple[str, ...]
+
+
+@dataclass(frozen=True, slots=True)
+class SensorimotorIR:
+    events: tuple[SensorimotorEvent, ...]
+    ir_sha256: str
+
+
+@dataclass(frozen=True, slots=True)
+class SensorimotorCompilationUnit:
+    canonical_sha256: str
+    ir_sha256: str
+    pack_id: str
+    pack_manifest_sha256: str
+    projection_sha256: str
+    versor: np.ndarray
+    versor_condition: float
+    sensorimotor_ir: SensorimotorIR
+
+    @property
+    def merge_key(self) -> tuple[str, str, str]:
+        return (self.canonical_sha256, self.ir_sha256, self.projection_sha256)

sensorium/vision/lexer.py

@@ -36,7 +36,7 @@ def lex_tile(signal: VisionTileSignal) -> tuple[VisualEvent, ...]:
         angle = float(np.arctan2(np.mean(gy), np.mean(gx)) + np.pi)
         orient_q = int(np.floor((angle / (2.0 * np.pi)) * 16.0)) % 16
     edge_q = _bin(energy_mean, max_value=0.5)
-    corner_q = _bin(float(np.mean(np.abs(gx * gy))), max_value=0.25)
+    corner_q = _bin(float(np.mean(np.abs(gx * gy))), max_value=0.02)
     center = luma[luma.shape[0] // 4: 3 * luma.shape[0] // 4, luma.shape[1] // 4: 3 * luma.shape[1] // 4]
     blob_q = _bin(abs(float(np.mean(center)) - mean), max_value=0.5)
     texture_q = _bin(float(np.mean(np.abs(energy - energy_mean))), max_value=0.5)

tests/test_observation_frame_contract.py

@@ -0,0 +1,75 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+import numpy as np
+import pytest
+
+from sensorium.environment import build_observation_frame
+
+
+@dataclass(frozen=True, slots=True)
+class _Unit:
+    canonical_sha256: str
+    ir_sha256: str
+    pack_id: str
+    pack_manifest_sha256: str
+    projection_sha256: str
+    versor: np.ndarray
+    versor_condition: float = 0.0
+
+    @property
+    def merge_key(self) -> tuple[str, str, str]:
+        return (self.canonical_sha256, self.ir_sha256, self.projection_sha256)
+
+
+def _unit(name: str, pack_id: str) -> _Unit:
+    v = np.zeros(32, dtype=np.float32)
+    v[0] = 1.0
+    return _Unit(name, f"ir-{name}", pack_id, "manifest", f"proj-{name}", v)
+
+
+def test_observation_frame_is_order_invariant_and_deduped():
+    audio = _unit("a", "audio_core_v1")
+    vision = _unit("v", "vision_core_v1")
+    text = _unit("t", "en")
+    f1 = build_observation_frame(monotonic_tick=7, source_clock="local", units=[audio, vision, text, audio])
+    f2 = build_observation_frame(monotonic_tick=7, source_clock="local", units=[text, audio, vision])
+    assert f1.trace_hash == f2.trace_hash
+    assert f1.environment_sha256 == f2.environment_sha256
+    assert len(f1.units) == 3
+    assert tuple(unit.merge_key for unit in f1.units) == tuple(sorted(unit.merge_key for unit in f1.units))
+
+
+def test_mixed_units_remain_content_addressed():
+    frame = build_observation_frame(
+        monotonic_tick=1,
+        source_clock="edge",
+        causal_parent_ids=("parent",),
+        units=[_unit("audio", "audio_core_v1"), _unit("vision", "vision_core_v1")],
+    )
+    assert frame.frame_id
+    assert frame.causal_parent_ids == ("parent",)
+    assert frame.units[0].merge_key < frame.units[1].merge_key
+
+
+def test_unsafe_payloads_are_rejected_from_frame_trace():
+    @dataclass(frozen=True, slots=True)
+    class BadUnit(_Unit):
+        samples: bytes = b"pcm"
+
+    with pytest.raises(TypeError, match="unsafe observation payload"):
+        build_observation_frame(monotonic_tick=0, source_clock="local", units=[BadUnit(
+            "a", "ir-a", "audio_core_v1", "manifest", "proj-a", np.zeros(32, dtype=np.float32)
+        )])
+
+
+def test_efferent_action_trace_is_not_an_afferent_unit():
+    @dataclass(frozen=True, slots=True)
+    class ActionUnit(_Unit):
+        efferent: bool = True
+
+    with pytest.raises(ValueError, match="efferent"):
+        build_observation_frame(monotonic_tick=0, source_clock="local", units=[ActionUnit(
+            "m", "ir-m", "motor_test", "manifest", "proj-m", np.zeros(32, dtype=np.float32)
+        )])

tests/test_sensorimotor_contract.py

@@ -0,0 +1,73 @@
+from __future__ import annotations
+
+import numpy as np
+
+from sensorium.protocol import Modality
+from sensorium.sensorimotor import (
+    SensorimotorCompiler,
+    SensorimotorDelta,
+    canonicalize_proprioception,
+    merge_sensorimotor_deltas,
+    sensorimotor_evidence_trace,
+    sensorimotor_merge_trace_hash,
+)
+
+
+def _signal():
+    return canonicalize_proprioception(
+        pose_q=(10, -4, 3),
+        velocity_q=(1, 0, -1),
+        force_torque_q=(2, 3, 5),
+        contact_q=(1, 0, 1, 0),
+        actuator_state_q=(7, 8),
+        source_id="test-sensor",
+    )
+
+
+def test_sensorimotor_is_afferent_modality_label():
+    assert Modality.SENSORIMOTOR.value == "sensorimotor"
+
+
+def test_same_proprioceptive_signal_produces_identical_unit():
+    compiler = SensorimotorCompiler()
+    u1 = compiler.compile_signal(_signal())
+    u2 = compiler.compile_signal(_signal())
+    assert np.array_equal(u1.versor, u2.versor)
+    assert u1.merge_key == u2.merge_key
+    assert u1.versor.shape == (32,)
+    assert u1.versor.dtype == np.float32
+    assert u1.versor_condition < 1e-6
+
+
+def test_sensorimotor_ir_replay_is_deterministic():
+    compiler = SensorimotorCompiler()
+    unit = compiler.compile_signal(_signal())
+    replay = compiler.compile_ir(unit.sensorimotor_ir)
+    assert np.array_equal(unit.versor, replay.versor)
+    assert unit.ir_sha256 == replay.ir_sha256
+    assert unit.projection_sha256 == replay.projection_sha256
+
+
+def test_sensorimotor_delta_merge_is_idempotent():
+    compiler = SensorimotorCompiler()
+    unit = compiler.compile_signal(_signal())
+    delta = SensorimotorDelta.from_units([unit])
+    merged = merge_sensorimotor_deltas([delta, delta])
+    assert merged.merge_keys == delta.merge_keys
+    assert sensorimotor_merge_trace_hash(merged) == sensorimotor_merge_trace_hash(delta)
+
+
+def test_sensorimotor_trace_has_no_actuator_command_payload():
+    unit = SensorimotorCompiler().compile_signal(_signal())
+    trace = sensorimotor_evidence_trace(unit)
+    assert trace["modality"] == "sensorimotor"
+    assert "command" not in trace
+    assert "trajectory" not in trace
+    for value in trace.values():
+        assert not isinstance(value, (np.ndarray, bytes, bytearray))
+
+
+def test_sensorimotor_compiler_exposes_no_decode_path():
+    compiler = SensorimotorCompiler()
+    assert not hasattr(compiler, "decode")
+    assert not hasattr(compiler, "decode_batch")

tests/test_vision_eval_gates.py

@@ -0,0 +1,127 @@
+"""Vision compiler eval gate table."""
+
+from __future__ import annotations
+
+import json
+from collections import Counter
+from pathlib import Path
+
+import numpy as np
+import pytest
+
+from evals.vision_sensorium.synth import synthesize
+from sensorium.adapters.vision import make_vision_pack
+from sensorium.registry import ModalityRegistry
+from sensorium.vision import VisionCompiler, canonicalize_image, vision_evidence_trace
+from sensorium.vision.grid import iter_tile_signals
+from sensorium.vision.types import VisionIR
+
+_EVAL_DIR = Path("evals/vision_sensorium")
+TOL = 1e-6
+
+
+def _load_fixtures() -> list[dict]:
+    return json.loads((_EVAL_DIR / "fixtures.json").read_text())["fixtures"]
+
+
+def _load_expected_ir() -> dict[str, dict]:
+    out: dict[str, dict] = {}
+    for line in (_EVAL_DIR / "expected_ir.jsonl").read_text().splitlines():
+        if line.strip():
+            row = json.loads(line)
+            out[row["id"]] = row
+    return out
+
+
+def _load_expected_projection() -> dict[str, list[dict]]:
+    return json.loads((_EVAL_DIR / "expected_projection.json").read_text())
+
+
+def _event_type_counts(ir: VisionIR) -> dict[str, int]:
+    events = (
+        *ir.regions,
+        *ir.contour_arcs,
+        *ir.orient_events,
+        *ir.texture_atoms,
+        *ir.salient_events,
+        *ir.content_anchors,
+    )
+    return dict(sorted(Counter(e.event_type for e in events).items()))
+
+
+FIXTURES = _load_fixtures()
+EXPECTED_IR = _load_expected_ir()
+EXPECTED_PROJ = _load_expected_projection()
+IDS = [fx["id"] for fx in FIXTURES]
+
+
+@pytest.fixture(scope="module")
+def compiler() -> VisionCompiler:
+    return VisionCompiler()
+
+
+@pytest.mark.parametrize("fx", FIXTURES, ids=IDS)
+def test_vision_gate_table(fx, compiler):
+    image = canonicalize_image(synthesize(fx))
+    units = compiler.compile_image(image)
+    signals_by_coord = {signal.coord: signal for signal in iter_tile_signals(image)}
+    fid = fx["id"]
+
+    assert image.canonical_sha256 == EXPECTED_IR[fid]["canonical_sha256"]
+    assert len(units) == EXPECTED_IR[fid]["unit_count"]
+
+    counts = Counter()
+    for idx, unit in enumerate(units):
+        assert unit.versor.shape == (32,)
+        assert unit.versor.dtype == np.float32
+        assert unit.versor_condition < TOL
+
+        again = compiler.compile_tile(signals_by_coord[unit.coord])
+        assert np.array_equal(unit.versor, again.versor)
+        assert unit.merge_key == again.merge_key
+
+        replay = compiler.compile_ir(unit.vision_ir)
+        assert np.array_equal(unit.versor, replay.versor)
+        assert unit.ir_sha256 == replay.ir_sha256 == EXPECTED_IR[fid]["unit_ir_sha256"][idx]
+
+        counts.update(_event_type_counts(unit.vision_ir))
+        reference = np.asarray(EXPECTED_PROJ[fid][idx]["reference_versor"], dtype=np.float32)
+        assert unit.projection_sha256 == EXPECTED_PROJ[fid][idx]["projection_sha256"]
+        assert np.allclose(unit.versor, reference, atol=TOL)
+
+    assert dict(sorted(counts.items())) == EXPECTED_IR[fid]["event_type_counts"]
+
+
+@pytest.mark.parametrize("fx", FIXTURES, ids=IDS)
+def test_vision_trace_hygiene_no_pixels(fx, compiler):
+    image = canonicalize_image(synthesize(fx))
+    for unit in compiler.compile_image(image):
+        trace = vision_evidence_trace(unit)
+        assert "pixels" not in trace
+        for value in trace.values():
+            assert not isinstance(value, (np.ndarray, bytes, bytearray))
+
+
+def test_vision_gate_closure_refuses_projection():
+    image = canonicalize_image(synthesize(FIXTURES[1]))
+    tile = iter_tile_signals(image)[0]
+    reg = ModalityRegistry()
+    reg.mount(make_vision_pack("vision_core_v1"), sample=tile)
+    with pytest.raises(RuntimeError, match="gate is not engaged"):
+        reg.project("vision_core_v1", tile)
+
+
+def test_semantic_expectations_match_designed_fixtures():
+    required = {
+        "flat_gray": {"region.flat"},
+        "vertical_edge": {"region.contrast", "orient.edge_energy"},
+        "corner_block": {"region.corner"},
+        "center_blob": {"region.blob"},
+        "checker_texture": {"texture.regularity"},
+        "contrast_ramp": {"region.contrast"},
+        "chroma_split": {"region.chroma"},
+        "salient_spot": {"salient.figure_ground"},
+        "contour_box": {"contour.closure"},
+    }
+    for fid, event_types in required.items():
+        assert event_types <= set(EXPECTED_IR[fid]["event_type_counts"]), fid