Commit graph

2 commits

Author SHA1 Message Date
Shay
f03979f025 feat: integrate hardened CLOSE yardstick into determinism & teaching regression surfaces
- docs/testing-lanes.md + Makefile: recommended determinism regression invocation (uv run python -m evals.close_derived_climb + contract pytest) as part of standard verification story / rerun flows.
- evals/anti_regression/run_demo.py + tests/test_anti_regression_demo.py: hermetic embedding of the yardstick (lived IdleTickResult flag, semantic determine rule=direct, content_replay_checksum) into the core anti-regression / teaching demo flow (core demo anti-regression now carries + reports the Claim-B signals; active corpus remains untouched).
- Supporting docs: contract.md (uv + refs), anti_regression_demo.md (complementary note), runtime_contracts.md (determination surface cross-ref).
- New ratification artifact (pre-impl) + all success criteria met.

See ratification for chosen approach + why only correct path.
All existing tests/invariants green; hermeticity preserved; no core/engine changes.
2026-06-16 17:25:05 -07:00
Shay
6f4b2b7b2c feat(adr-0057): anti-regression demo — three-gate defense against learning harm
`core demo anti-regression` (+ `--json`) is a self-contained walkthrough of
the three independent gates that every reviewed-corpus extension must pass.
Designed for showcasing CORE's epistemic discipline to reviewers / industry
observers — no LLM provider has a published equivalent.

Scenes:
- S1. Eligibility predicate refuses an undetermined-polarity candidate
  before any replay is invoked.  ProposalError raised; no log row.
- S2. Replay-equivalence gate auto-rejects a regressing candidate with
  the named regressed metrics in the operator note.  Uses the documented
  `run_replay=` kwarg of `propose_from_candidate` to inject a controlled
  regression of the same `ReplayEvidence` shape the real gate produces.
- S3. Real `teaching.replay.run_replay_equivalence` runs the cognition
  public lane.  A replay-equivalent candidate reaches 'pending' — operator
  `--accept` is still required to write.

Each scene asserts the active corpus is byte-identical pre/post.

- evals/anti_regression/run_demo.py — `run_demo(emit_json=False)` returns
  a structured `DemoReport`; verbose human output by default, JSON on flag.
- core/cli.py — `core demo anti-regression` target wired alongside
  audit-tour / pack-measurements / long-context-comparison.
- tests/test_anti_regression_demo.py — 5 tests pin each scene's
  load-bearing claim + the corpus-byte-identical invariant.

Lane state: anti-regression-demo 5 new — green.  Demo runs in ~10s end-to-end.
2026-05-18 10:52:23 -07:00