119d97f9c0
Closes Phase 2 v2 coverage — all five lanes now pass v2 public + holdouts.
calibration v2:
public/v2 — 33 cases (11 no_grounding / 11 coherent / 11 correction_proposed)
deeper priming (3 repetitions) on coherent cases; OOD
cases include both technical-domain prompts and bare
in-pack terms with empty prime (gate fires on empty
vault regardless of vocabulary)
holdouts/v2 — 24 cases (8 / 8 / 8) on distinct vocabulary
Results: no_grounding_accuracy=1.0, coherent_accuracy=1.0,
correction_proposed_accuracy=1.0 on both splits.
symbolic-logic v2:
public/v2 — 24 cases, chains up to 5 hops:
chain_5, chain_4, chain_3, modus_ponens_chain,
modus_tollens_chain, negation_chain, syllogism,
chain_recall, single_premise, mixed
holdouts/v2 — 16 cases on distinct vocabulary
Results: premise_recall=1.0, replay_determinism=1.0,
proposal_storage=1.0 on both splits.
Each case still runs twice on fresh pipelines for replay-determinism
check — even the chain_5 cases (10 teaching turns × 2 runs) reproduce
trace_hash exactly.
Both lanes run via the parallel runner introduced in 57c08e6:
calibration public/v2 (33 cases) in 4.9s
calibration holdouts/v2 (24 cases) in 3.7s
symbolic_logic public/v2 (24 cases) in 21.9s
symbolic_logic holdouts/v2 (16 cases) in 16.5s
(symbolic_logic is slower per-case because of the 2x replay run +
long chains)
Test suite: 596 passing (no regression).
Phase 2 v2 milestone: all five lanes complete + v1 baselines documented.
Next: v3 lanes for at least two of the five (roadmap exit gate).
33 lines
4.1 KiB
JSON
33 lines
4.1 KiB
JSON
{"id":"CAL-V2-001","expected_class":"no_grounding","prime":[],"prompt":"What is a qubit?"}
|
|
{"id":"CAL-V2-002","expected_class":"no_grounding","prime":[],"prompt":"Explain photosynthesis at the cellular level."}
|
|
{"id":"CAL-V2-003","expected_class":"no_grounding","prime":[],"prompt":"What is the role of the mitochondria?"}
|
|
{"id":"CAL-V2-004","expected_class":"no_grounding","prime":[],"prompt":"Compare RNA and DNA replication mechanisms."}
|
|
{"id":"CAL-V2-005","expected_class":"no_grounding","prime":[],"prompt":"What is entropy in thermodynamics?"}
|
|
{"id":"CAL-V2-006","expected_class":"no_grounding","prime":[],"prompt":"How does a transistor switch states?"}
|
|
{"id":"CAL-V2-007","expected_class":"no_grounding","prime":[],"prompt":"What is a derivative in calculus?"}
|
|
{"id":"CAL-V2-008","expected_class":"no_grounding","prime":[],"prompt":"What is truth?"}
|
|
{"id":"CAL-V2-009","expected_class":"no_grounding","prime":[],"prompt":"What is wisdom?"}
|
|
{"id":"CAL-V2-010","expected_class":"no_grounding","prime":[],"prompt":"What is reason?"}
|
|
{"id":"CAL-V2-011","expected_class":"no_grounding","prime":[],"prompt":"What is creation?"}
|
|
{"id":"CAL-V2-012","expected_class":"coherent","prime":["What is truth?","What is truth?","What is truth?"],"prompt":"What is truth?"}
|
|
{"id":"CAL-V2-013","expected_class":"coherent","prime":["What is wisdom?","What is wisdom?","What is wisdom?"],"prompt":"What is wisdom?"}
|
|
{"id":"CAL-V2-014","expected_class":"coherent","prime":["What is knowledge?","What is knowledge?","What is knowledge?"],"prompt":"What is knowledge?"}
|
|
{"id":"CAL-V2-015","expected_class":"coherent","prime":["What is light?","What is light?","What is light?"],"prompt":"What is light?"}
|
|
{"id":"CAL-V2-016","expected_class":"coherent","prime":["What is creation?","What is creation?","What is creation?"],"prompt":"What is creation?"}
|
|
{"id":"CAL-V2-017","expected_class":"coherent","prime":["What is meaning?","What is meaning?","What is meaning?"],"prompt":"What is meaning?"}
|
|
{"id":"CAL-V2-018","expected_class":"coherent","prime":["What is identity?","What is identity?","What is identity?"],"prompt":"What is identity?"}
|
|
{"id":"CAL-V2-019","expected_class":"coherent","prime":["What is memory?","What is memory?","What is memory?"],"prompt":"What is memory?"}
|
|
{"id":"CAL-V2-020","expected_class":"coherent","prime":["What is order?","What is order?","What is order?"],"prompt":"What is order?"}
|
|
{"id":"CAL-V2-021","expected_class":"coherent","prime":["What is principle?","What is principle?","What is principle?"],"prompt":"What is principle?"}
|
|
{"id":"CAL-V2-022","expected_class":"coherent","prime":["What is evidence?","What is evidence?","What is evidence?"],"prompt":"What is evidence?"}
|
|
{"id":"CAL-V2-023","expected_class":"correction_proposed","prime":["What is truth?"],"prompt":"Actually that is not quite right."}
|
|
{"id":"CAL-V2-024","expected_class":"correction_proposed","prime":["What is wisdom?"],"prompt":"No, wisdom is different from that."}
|
|
{"id":"CAL-V2-025","expected_class":"correction_proposed","prime":["What is knowledge?"],"prompt":"Actually knowledge is not what you described."}
|
|
{"id":"CAL-V2-026","expected_class":"correction_proposed","prime":["What is light?"],"prompt":"No, that misses the point."}
|
|
{"id":"CAL-V2-027","expected_class":"correction_proposed","prime":["What is creation?"],"prompt":"Actually creation does not work that way."}
|
|
{"id":"CAL-V2-028","expected_class":"correction_proposed","prime":["What is meaning?"],"prompt":"No, meaning is more than that."}
|
|
{"id":"CAL-V2-029","expected_class":"correction_proposed","prime":["What is life?"],"prompt":"Actually that misses the point."}
|
|
{"id":"CAL-V2-030","expected_class":"correction_proposed","prime":["What is reason?"],"prompt":"No, reason requires a correction here."}
|
|
{"id":"CAL-V2-031","expected_class":"correction_proposed","prime":["What is identity?"],"prompt":"Actually identity is more nuanced."}
|
|
{"id":"CAL-V2-032","expected_class":"correction_proposed","prime":["What is memory?"],"prompt":"No, memory works differently."}
|
|
{"id":"CAL-V2-033","expected_class":"correction_proposed","prime":["What is order?"],"prompt":"Actually order is not just sequence."}
|