core-labs/core
1
0
Fork 0
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
core/tests
History
Shay 2e4e45b49b feat(evals): provenance lane v1 — replay determinism + source back-pointers 
Phase 2's first lane: every articulated claim must back-point to one of
{pack axiom, vault entry, teaching event}, and replay must reproduce the
trace bit-for-bit.

Components:
- core/cognition/provenance.py: Provenance dataclass + compute_provenance()
  deriving sources from a CognitiveTurnResult. Pack source = non-UNKNOWN
  intent.tag (pack-defined intent rule matched); vault source = vault_hits
  count; teaching source = pack_mutation_proposal.proposal_id.
- evals/provenance/{contract.md, runner.py, dev/, public/v1/, holdouts/v1/}:
  45 cases across pack_axiom / vault_recall / teaching / mixed categories.
- tests/test_provenance.py: 6 unit tests covering all source-kind profiles.

Sub-metrics (all four must pass):
- replay_determinism: same input + fresh runtime -> same trace_hash
- input_sensitivity: distinct prompts -> distinct trace_hashes
- source_attribution: every expected source kind present in Provenance
- source_validity: every cited source resolves to a real artefact

Results:
- dev: 10/10 (all sub-metrics 1.0)
- public/v1: 20/20 (all sub-metrics 1.0)
- holdouts/v1: 15/15 (all sub-metrics 1.0)

PROGRESS.md updated to mark Phase 2 in progress with provenance v1 complete.
2026-05-16 11:45:00 -07:00
..
__init__.py init: tests, pyproject.toml, AGENTS.md, CLAUDE.md, README.md 2026-05-12 19:15:28 -07:00
test_achat.py test(chat): achat() and arespond() smoke tests — surface, fields, OOV grace, structural determinism 2026-05-14 14:31:49 -07:00
test_alignment_graph.py Fix test suite errors across core physics and generation 2026-05-14 13:02:32 -07:00
test_architectural_invariants.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_articulation.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_articulation_realizer_v2.py Add articulation realizer v2 2026-05-14 20:14:50 -07:00
test_chat_runtime.py Bridge chat OOV grounding 2026-05-13 20:47:28 -07:00
test_cli.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_cli_pack_validate_security.py Harden pack validator trust boundary 2026-05-15 11:50:15 -07:00
test_cli_test_suites.py Speed up validation lanes and pack loading 2026-05-15 08:26:11 -07:00
test_cognitive_eval_harness.py Add cognitive eval harness and calibration replay (#30) 2026-05-15 07:41:36 -07:00
test_cognitive_turn_pipeline.py Wire intent graph into cognitive pipeline 2026-05-14 20:05:00 -07:00
test_compiler.py feat(core_ingest): IngestPipeline, manifold integration, full test suite 2026-05-13 12:07:12 -07:00
test_core_ingest.py feat: scaffold core_ingest/ governance layer (ADR-0012) 2026-05-13 11:35:00 -07:00
test_core_semantic_seed_pack.py feat: vault recall index, Rust versor parity, cognitive pack expansion 2026-05-15 15:34:39 -07:00
test_determinism_proofs.py Restore FieldState slots in determinism proof 2026-05-13 14:26:24 -07:00
test_deterministic_hash.py feat: manifold field topology, graph diffusion operator, vertical pulse 2026-05-15 16:02:48 -07:00
test_dialogue.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_dialogue_fluency_regression.py fix: harden session field invariants and eliminate hot-path inefficiencies 2026-05-15 21:37:49 -07:00
test_energy.py Tighten hot-path backend consistency 2026-05-15 08:14:38 -07:00
test_engine_loop_proof.py Tighten session node tracking and generation selection 2026-05-13 14:35:31 -07:00
test_generate_stream.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_graph_diffusion.py feat: manifold field topology, graph diffusion operator, vertical pulse 2026-05-15 16:02:48 -07:00
test_holonomy.py Align holonomy tests with indefinite metric 2026-05-13 12:59:32 -07:00
test_holonomy_resonance.py Implement trilingual field coherence 2026-05-13 19:53:37 -07:00
test_identity_gate.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_intent_proposition_graph.py feat: add intent-proposition graph comprehension layer 2026-05-14 19:52:57 -07:00
test_language_pack_cache.py feat: vault recall index, Rust versor parity, cognitive pack expansion 2026-05-15 15:34:39 -07:00
test_language_pack_runtime.py Implement trilingual field coherence 2026-05-13 19:53:37 -07:00
test_manifold.py feat(core_ingest): IngestPipeline, manifold integration, full test suite 2026-05-13 12:07:12 -07:00
test_manifold_state.py feat: manifold field topology, graph diffusion operator, vertical pulse 2026-05-15 16:02:48 -07:00
test_morphology_registry.py Implement trilingual field coherence 2026-05-13 19:53:37 -07:00
test_motor.py Fix full test suite after cognitive runtime 2026-05-13 13:52:11 -07:00
test_null_cone.py init: tests, pyproject.toml, AGENTS.md, CLAUDE.md, README.md 2026-05-12 19:15:28 -07:00
test_oov_grounding_cache.py Cache OOV morphology grounding structures 2026-05-15 11:53:46 -07:00
test_operator_calibration_replay.py Add cognitive eval harness and calibration replay (#30) 2026-05-15 07:41:36 -07:00
test_pipeline.py feat(core_ingest): IngestPipeline, manifold integration, full test suite 2026-05-13 12:07:12 -07:00
test_pipeline_teaching_integration.py feat: wire teaching loop into cognitive pipeline 2026-05-14 20:46:50 -07:00
test_probe_repl.py Implement trilingual field coherence 2026-05-13 19:53:37 -07:00
test_proof_properties.py feat: Full Proof — surface realizer join, Rust diffusion parity, benchmark harness 2026-05-15 17:39:14 -07:00
test_proposition.py Add proposition generation 2026-05-13 20:08:49 -07:00
test_provenance.py feat(evals): provenance lane v1 — replay determinism + source back-pointers 2026-05-16 11:45:00 -07:00
test_pulse_integration.py feat: Full Proof — surface realizer join, Rust diffusion parity, benchmark harness 2026-05-15 17:39:14 -07:00
test_reviewed_teaching_loop.py feat: add reviewed teaching loop for controlled correction learning 2026-05-14 20:32:28 -07:00
test_rotor_power.py fix(drift): proper rotor-manifold scaling; restore respond contract 2026-05-16 11:44:45 -07:00
test_runtime_config.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_rust_backend.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_salience.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_segmenter.py feat(core_ingest): IngestPipeline, manifold integration, full test suite 2026-05-13 12:07:12 -07:00
test_semantic_realizer_integration.py feat: vault recall index, Rust versor parity, cognitive pack expansion 2026-05-15 15:34:39 -07:00
test_sensorium_mount.py feat: scaffold sensorium/ (ADR-0013) + tests/test_architectural_invariants.py + tests/test_sensorium_mount.py 2026-05-13 11:45:16 -07:00
test_session_coherence.py fix(session): CGA-only hemisphere consistency, remove forbidden Euclidean metric 2026-05-15 21:51:16 -07:00
test_surface.py test(surface): full unit test suite for SentenceAssembler — roles, guard, elaboration, languages, determinism 2026-05-14 14:29:46 -07:00
test_transition_rotor.py fix(algebra): enforce transition rotor invariants 2026-05-15 21:13:14 -07:00
test_unknown_token_ingest.py Ground unknown tokens in ingest 2026-05-13 20:33:20 -07:00
test_vault_config.py fix(tests): align suite with persona-neutral strict-closure contract 2026-05-16 05:37:12 -07:00
test_vault_recall.py Activate and verify Rust backend 2026-05-13 22:23:48 -07:00
test_vault_store.py feat: vault recall index, Rust versor parity, cognitive pack expansion 2026-05-15 15:34:39 -07:00
test_versor_closure.py fix(chat): keep generic runtime persona-neutral 2026-05-15 23:15:56 -07:00
test_vocab_manifold_invariants.py Fail closed on invalid versor construction 2026-05-14 10:55:11 -07:00