Commit graph

1 commit

Author SHA1 Message Date
Shay
4670e391ec feat(phase5+bench): cross-pack supersede + articulation benchmark suite
Phase 5 (ADR-0067 follow-up):
  teaching/cross_pack_supersede.py — supersede_cross_pack_chain()
  CLI: core teaching supersede ... --cross-pack
    --subject-pack-id ... --object-pack-id ...
  Strict per-chain residency, anti-leakage, byte-identical rollback
  on any post-append re-load failure.  9 new tests.

Articulation benchmark suite (Phase 4 capability proof):
  benchmarks/articulation.py — 5 sub-benches
    [1] breadth        — every intent shape (9 + OOV + cross-pack)
    [2] determinism    — N reruns / unique-surface count
    [3] footprint      — psutil RSS profile across T turns
    [4] cross-topic    — thread context across mixed subjects
    [5] ollama-compare — opt-in side-by-side with local Ollama
  CLI: core bench --suite articulation
    --runs N (det rerun count)
    --turns N (footprint sample window)
    --ollama-model MODEL --ollama-reruns N
  Full operator preamble + JSON report path.
  10 new tests cover the bench shape (psutil import-skipped).

Documentation:
  benchmarks/README.md — full operator manual: catalogue of every
    bench suite, how to read good/neutral/bad results for each sub-
    bench, why CORE vs Ollama comparisons are valid on the
    determinism axis and not on linguistic quality, workflow guide.
  README.md — articulation bench listed in the live-demo grid and
    quick-start examples.

Reference run (llama3:8b, 100 turns, 5 reruns):
  determinism_all_identical=True
  per-turn ΔRSS ≈ 23 KiB
  CORE byte_identical_on_every_prompt=True
  Ollama unique_surfaces≥2 on every prompt

Verification:
  18 new tests pass
  Full lane: 2116 passed, 2 skipped, 0 failed in 2:38
2026-05-18 17:44:59 -07:00