Shay
|
36aad75202
|
feat(cli): ADR-0024 chain test-suite aliases + core demo subcommand
Two layers of CLI surface so reviewers / investors / industry
observers can run the ADR-0024 chain evidence end-to-end without
typing test file paths or hunting for runner scripts.
Layer 1 — test-suite aliases:
core test --suite refusal (Phase 2 typed refusals)
core test --suite margin (Phase 3 / ADR-0026 ranked-with-margin)
core test --suite rotor (Phase 4 / ADR-0025 rotor admissibility)
core test --suite inner-loop (ADR-0024 inner-loop, all 4 sub-tests)
core test --suite phase5 (stratified mechanism-isolation)
core test --suite phase6 (3-condition comparative demo)
core test --suite adr-0024 (full chain, 98 tests, ~2 min)
Layer 2 — `core demo` subcommand:
core demo phase5 stratified pass/refuse table + per-family
breakdown (5 families, both modes)
core demo phase6 three head-to-head verdicts vs baseline
(replay determinism / traced rejection /
coherent refusal)
core demo all both phases + combined summary
core demo list-results index every JSON report in the central
results directory with headline metrics
All demo runs:
- Write fresh JSON to evals/forward_semantic_control/results/
- Refresh the results/index.json manifest so reviewers see every
available report in one place
- Accept --json for machine-readable output
Central results directory: evals/forward_semantic_control/results/
phase2_inner_loop_report.json
phase3_v2_report.json
phase4_characterization_*.json
phase5_report.json
phase5_benign_inner_loop_report.json
phase6_demo_report.json
index.json (auto-generated manifest)
Files:
core/cli.py — +9 suite aliases, +cmd_demo (3 targets +
list-results), +index manifest writer
tests/test_cli_demo.py — 14 contract tests pinning Layer 1 + 2
evals/forward_semantic_control/results/index.json — auto-generated
Tests: 1099 passed, 2 skipped (+14 from Phase 6 baseline).
|
2026-05-17 16:21:37 -07:00 |
|