Commit graph

3 commits

Author SHA1 Message Date
Shay
34295e55ce perf(test-infra): pytest-xdist + module-scoped demo fixtures
Full lane wall-time: 6:35 → 2:25 (2.7× speedup).  No behavioral
changes; same 1933 passed, 2 skipped.

Three wins, biggest first:

1. pytest-xdist as a project dependency.

   ``pyproject.toml`` gains ``pytest-xdist>=3.6``.  ``cmd_test``
   injects ``-n auto`` for ``--suite full`` when xdist is importable;
   curated suites stay single-process because worker-spawn overhead
   is net-negative on the smaller suites.  Operator can override
   via passing ``-n <N>`` or ``--dist`` explicitly.

   Verified: ``core test --suite full -q`` prints ``bringing up
   nodes...`` and parallelises across the runner's CPUs.

2. Module-scoped fixture for run_demo() in test_learning_loop_demo.py.

   The 7 demo tests each previously called ``run_demo(emit_json=True)``
   from scratch — and ``run_demo`` itself runs the cognition lane
   twice via the replay-equivalence gate.  ~15s/file → ~3s/file.

   Module scope (not session) is intentional: pytest-xdist
   distributes by test, so a session-scoped fixture would still be
   re-evaluated per worker that picks up a test from this file.
   Module scope keeps the cost paid once per worker per file, which
   is the actual lower bound.

3. Module-scoped fixture for the teaching-loop bench.

   ``test_teaching_loop_bench.py``'s 5 tests previously each ran
   ``run_teaching_loop_determinism(runs=2 or 3)`` — 12 pipeline
   invocations across the file.  One ``runs=3`` invocation shared
   across all 5 tests covers every assertion: ~25s → ~7s.

For local iteration, ``core test --suite cognition -q`` etc. remain
fast (no xdist overhead).  The full-lane speedup is most visible
under CI / pre-merge runs.
2026-05-18 16:12:27 -07:00
Shay
7c80b791ec fix(tests): retire 13 stale failures from full lane — corpus saturation drift
The full lane carried 13 long-standing red tests whose premises were
invalidated by reviewed-corpus growth that landed in earlier commits.
None reflected runtime bugs — all four classes are corpus-state drift
where the test fixture became stale.  Curated lanes were green, full
lane stayed quietly red.  Closes that gap.

1. test_teaching_audit (2 tests).
   * test_audit_real_corpus_runs_clean asserted dropped == () and
     lines_on_disk == lines_loaded — premise written before any
     supersession existed.  Curriculum saturation v2 (commit a0edbb4)
     ratified the wisdom_grounds_judgment → wisdom_requires_knowledge
     supersession; the audit now correctly shows 1 dropped line.
     Rewritten as the line-conservation invariant:
       lines_loaded + len(dropped) == lines_on_disk
     plus a typed-reason check on every dropped entry.
   * test_default_superseded_by_is_null_in_loaded_entries asserted
     ALL loaded entries have superseded_by == None.  Wrong even by
     ADR-0055 design: the replacement entry IS loaded and carries
     the back-pointer to the retired chain.  Rewritten as the
     active-set invariant: any non-null superseded_by on a loaded
     entry must reference a dropped (retired) chain id, never a live
     one — no double-live state.

2. test_learning_loop_demo (7 tests).
   The demo's headline prompt was "Why does thought exist?", and the
   ADR-0057 demo trilogy (commit 82dac4b) chose (thought, cause) as
   the cold cell.  Cognition saturation v2 (commit a0edbb4) ratified
   cause_thought_reveals_meaning into the active corpus — so the
   cold turn now grounds, no discovery candidate is emitted, every
   demo scene breaks.  Rotated the cold subject to ``narrative``
   (pack-resident, no chain, same thematic shape, same affirming
   evidence pointer cause_creation_reveals_meaning).  Demo headline,
   evals/learning_loop/run_demo.py, core/cli.py preamble, and the
   test assertions all updated together so the demo reads cleanly:
       before: [none]     I don't know — insufficient grounding...
       after : [teaching] narrative — teaching-grounded ... narrative
                          reveals meaning ...

3. test_discovery_candidates (4 tests).
   Test fixture used (judgment, CAUSE) as the still-cold pair.
   Epistemology v1 (commit 2acf71f) ratified
   cause_judgment_requires_wisdom — (judgment, cause) is no longer
   cold.  Rotated to ``principle`` (pack-resident, no chain on either
   intent today).  Added a pytest.skip self-guard so when a future
   curriculum unit ratifies a (principle, *) chain the test rotates
   cleanly instead of going red.

Full lane: 1892 passed, 2 skipped, 0 failed (was 4 failed pre-fix,
13 failed pre-ADR-0063).  Cognition eval unchanged: public 100/100/
91.7/100, holdout 100/100/83.3/100.
2026-05-18 15:23:22 -07:00
Shay
a71b321a9a feat(adr-0055-0057): learning-loop demo — cold turn to grounded surface, end-to-end
`core demo learning-loop` (+ `--json`) walks a single prompt through the
full ADR-0055..0057 inter-session-memory architecture:

  S1. Cold turn          → universal disclosure, grounding_source=none
  S2. Discovery emission → DiscoveryCandidate to attached sink
  S3. Operator proposal  → real replay-equivalence gate, no regression
  S4. Operator accept    → TRANSIENT corpus only; active untouched
  S5. Same prompt        → teaching-grounded surface with the new chain

Before / after on the deterministic prompt "Why does thought exist?":

  before: [none]     I don't know — insufficient grounding for that yet.
  after:  [teaching] thought — teaching-grounded (cognition_chains_v1):
          cognition.thought; logos.internal. thought reveals meaning
          (cognition.meaning). No session evidence yet.

The active corpus on disk is byte-identical pre/post.  The demo writes
only to a transient corpus, then swaps `_CORPUS_PATH` for the after
turn — the same pattern the replay-equivalence gate uses.

- evals/learning_loop/run_demo.py — `run_demo(emit_json=False)` returns
  a structured `DemoReport` with both surfaces and per-scene detail.
- core/cli.py — `core demo learning-loop` target wired.
- tests/test_learning_loop_demo.py — 7 tests pin: full loop closes,
  before is ungrounded, after contains new chain atoms (thought /
  reveal / meaning), discovery emits ≥1, replay gate reports no
  regression, S4 byte-identical active + 1 line on transient, same
  prompt drives both surfaces.

Lane state: learning-loop-demo 7 new — green.  Demo runs in ~15s
end-to-end (cognition lane runs twice via replay gate).

No LLM provider has a published equivalent of this loop: per-fact
provenance from operator accept to surface, replay-equivalence gate
proving non-regression, byte-identical active state regardless of
outcome, full audit trail back to the originating cold turn.
2026-05-18 10:57:41 -07:00