78ddab79b4
Closes the consumption-half of the math teaching loop for two of three sub-types per docs/handoff/CONSUMPTION-WIRING-DISPATCH-PACK.md (PR #397). Companion to the doctrinal brief in PR #396. Modules ------- - language_packs/compile_frames.py — byte-deterministic compile of frames/*.jsonl → frames.jsonl (sorted by (frame_category, surface_form)) - language_packs/compile_compositions.py — same shape for compositions/*.jsonl → compositions.jsonl - generate/comprehension/frame_registry.py — load_frame_registry() mirroring load_lexicon: cache by (path, mtime, sha256), manifest checksum verification (optional frame_checksum field), polarity validation, conflict detection, empty-registry no-op - generate/comprehension/composition_registry.py — same shape PLUS: * SAFE_COMPOSITION_CATEGORIES enforced at LOAD (defense in depth; raises WrongCompositionCategory on any unsafe category — protects against pack edits that bypass the handler) * polarity "falsifies" exposed via is_falsified() (consumer must suppress; not silently treated as affirms) - language_packs/compiler.py — manifest verification extended for frame_checksum + composition_checksum, mirroring the proven glosses_checksum pattern (optional fields; backward-compatible) - generate/recognizer_anchor_inject.py — inject_from_match consults composition_registry when the per-category injector returns empty AND the matcher publishes ``composition_shape`` in parsed_anchors. Registry is a gate (admissibility) not an arithmetic primitive (ADR-0169 §"Mutation boundary"). Tests (38 new, all green) ------------------------- tests/test_frame_registry_load.py (11 tests) tests/test_composition_registry_load.py (11 tests) tests/test_composition_consult_in_injector.py ( 6 tests) tests/test_consumption_case_0050_hazard_pin.py( 3 tests, parametrized over allowlist) tests/test_consumption_empty_registry_no_op.py( 4 tests) tests/test_consumption_partition.py ( 3 tests) Registered in core/cli.py "packs" suite. Suite results ------------- core test --suite teaching -q → 93 passed core test --suite runtime -q → 20 passed core test --suite packs -q → 51 passed core eval gsm8k_math --split public → 150/150, wrong=0 Truth-test rows (6-row binding table in dispatch pack): #1 Case 0019 admits ............. PARTIAL — see Scope Boundary below #2 Case 0050 stays refused ....... PASS #3 train_sample 3/47 → ≥4/46 ..... PARTIAL — same as #1 #4 wrong == 0 preserved .......... PASS #5 public split 150/150 .......... PASS #6 Empty-registry no-op .......... PASS Scope Boundary (honest finding) ------------------------------- Rows #1 and #3 (case 0019 admission) require a matcher extension that publishes ``composition_shape`` + a pre-composed CandidateInitial in parsed_anchors. The existing currency_amount / multiplicative_aggregation matchers in generate/recognizer_match.py are detection-only (return empty parsed_anchors). This PR ships the consumption infrastructure correctly but the runtime path remains dormant until a follow-up PR extends the matcher. The dispatch pack's truth test #1/#3 cannot fire without that extension. The wiring is positioned correctly: inject_from_match → consult composition_registry → admit on affirms-with-payload, suppress on falsifies, refuse on absence. A synthetic recognizer match with populated composition_shape + composed_initial DOES admit through the new path (covered by 6 tests in test_composition_consult_in_injector.py). A follow-up brief naming the matcher-extension work is the recommended next step. Anti-regression invariants verified ----------------------------------- - wrong == 0 on core eval gsm8k_math (public 150/150) - case 0050 stays refused (parametrized over allowlist categories) - ADR-0166 — no new eval lanes - ADR-0167 partition — no cognition imports in any new module - Empty-registry runtime byte-identical to today (no-op test) - SAFE_COMPOSITION_CATEGORIES enforced at write AND load - polarity semantics (affirms vs falsifies) honored - engine_state/* never committed
86 lines
3 KiB
Python
86 lines
3 KiB
Python
"""CW-1 — compile frames/*.jsonl into a deterministic frames.jsonl artifact.
|
|
|
|
Mirrors the lexicon compile pattern (see ``language_packs/compiler.py``).
|
|
Reads per-category source files under ``{pack}/frames/*.jsonl``, normalizes
|
|
ordering, and writes the compiled artifact ``{pack}/frames.jsonl`` with
|
|
entries sorted by ``(frame_category, surface_form)``.
|
|
|
|
Returns the sha256 of the compiled bytes so callers (typically the
|
|
manifest writer) can pin ``frame_checksum`` per CLAUDE.md
|
|
"Semantic Pack Discipline".
|
|
|
|
Trust boundary: read-only over the reviewed pack; no engine_state writes,
|
|
no corpus mutation. Empty source directory produces an empty (zero-byte)
|
|
compiled artifact — the loader treats this as a no-op.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import hashlib
|
|
import json
|
|
from pathlib import Path
|
|
from typing import Any
|
|
|
|
|
|
def _canonical_entry(rec: dict[str, Any]) -> dict[str, Any]:
|
|
"""Project the entry to its canonical-bytes shape.
|
|
|
|
Drops unknown keys to keep the compiled artifact byte-stable across
|
|
upstream schema additions. Required keys: surface_form, frame_category,
|
|
polarity, provenance, evidence_hashes.
|
|
"""
|
|
return {
|
|
"surface_form": str(rec["surface_form"]),
|
|
"frame_category": str(rec["frame_category"]),
|
|
"polarity": str(rec["polarity"]),
|
|
"provenance": str(rec.get("provenance", "")),
|
|
"evidence_hashes": [str(h) for h in rec.get("evidence_hashes", [])],
|
|
}
|
|
|
|
|
|
def compile_frames(pack_path: Path) -> tuple[bytes, str]:
|
|
"""Compile ``{pack_path}/frames/*.jsonl`` → bytes + sha256.
|
|
|
|
The compiled bytes are written to ``{pack_path}/frames.jsonl`` if
|
|
they differ from the existing file (or it does not exist). The
|
|
sha256 is returned so the caller may update the pack manifest.
|
|
|
|
Deterministic ordering: source files iterated alphabetically; entries
|
|
within each file are re-sorted globally by
|
|
``(frame_category, surface_form)`` before write.
|
|
|
|
Empty source directory (or missing) → zero-byte artifact.
|
|
"""
|
|
source_dir = pack_path / "frames"
|
|
entries: list[dict[str, Any]] = []
|
|
if source_dir.is_dir():
|
|
for src in sorted(source_dir.glob("*.jsonl")):
|
|
for line in src.read_text(encoding="utf-8").splitlines():
|
|
line = line.strip()
|
|
if not line:
|
|
continue
|
|
entries.append(_canonical_entry(json.loads(line)))
|
|
|
|
entries.sort(key=lambda e: (e["frame_category"], e["surface_form"]))
|
|
|
|
if not entries:
|
|
compiled_bytes = b""
|
|
else:
|
|
compiled_bytes = (
|
|
"\n".join(
|
|
json.dumps(e, ensure_ascii=False, sort_keys=True, separators=(",", ":"))
|
|
for e in entries
|
|
)
|
|
+ "\n"
|
|
).encode("utf-8")
|
|
|
|
out_path = pack_path / "frames.jsonl"
|
|
existing = out_path.read_bytes() if out_path.exists() else None
|
|
if existing != compiled_bytes:
|
|
out_path.write_bytes(compiled_bytes)
|
|
|
|
sha256 = hashlib.sha256(compiled_bytes).hexdigest()
|
|
return compiled_bytes, sha256
|
|
|
|
|
|
__all__ = ["compile_frames"]
|