core/core/contemplation/runner.py
Shay efd280d45f
feat(l10): autonomous idle frontier-contemplation — the always-on life learns on its own (#758)
* feat(l10): autonomous idle frontier-contemplation — the always-on life learns on its own

Frontier survey move #2, path (c): give the always-on idle life a stream of experience via
SYMBOLIC intake (not the afferent→field seam, which is deferred pending a commensurability
ruling — docs/analysis/afferent-field-ingest-scope-2026-06-15.md). The idle heartbeat
converges (idle_tick drains a finite backlog + saturates is-a closure, then does nothing
forever); this makes it autonomously MINE its frontier with no user turn.

Reuse-heavy (the mechanism largely existed): a gated idle pass wires ADR-0080 contemplation
into the always-on loop.

- core/config.py: contemplate_frontier_during_idle (default off → no behavior change).
- chat/runtime.py: idle_tick pass 2b — when on, runs core.contemplation.run_contemplation()
  (mines the checked-in frontier-compare reports) and persists a reviewable ContemplationRun
  to <engine_state>/contemplation_runs/idle_<substrate_hash>.json. IDEMPOTENT per frontier
  (an already-mined frontier is not re-persisted → the life converges, no churn) and MEMOIZED
  per session (the static frontier is mined once, not re-read every beat). SPECULATIVE-only
  (ADR-0080: ContemplationFinding.__post_init__ RAISES if not SPECULATIVE → wrong=0 by
  construction; never ratified here — the HITL path is untouched). IdleTickResult gains
  frontier_findings.
- chat/always_on.py: run_continuous did_work + HeartbeatRecord count frontier work, so the
  heartbeat/soak convergence (H3) stays honest.
- core/cli.py: `core always-on --contemplate-frontier` (explicit learning-daemon mode) +
  honest per-beat log ("+N findings").

Adversarial 3-lens review (wrong=0/convergence, determinism/trust-boundary, test-vacuity)
found 7; fixed the real ones: the MEDIUM torn-write (write_contemplation_run is now atomic
temp+os.replace — the indefinite-uptime daemon expects SIGKILL mid-write, and the skip-guard
would never repair a torn canonical file), the gitignore breadth (contemplation_runs/ at any
engine-state root), the 64-bit filename truncation (full substrate_hash), and the
re-mine-every-beat cost (session memoization). Deferred + documented: the shared
contemplation miner embeds absolute report paths in finding CONTENT — a portability wrinkle
that does NOT affect wrong=0/convergence/soak determinism (those key on the content-only
substrate_hash) and touches shared code + its test surface; a follow-up.

Tests (non-vacuous): 7 — off-by-default (no behavior change), mines-when-enabled,
findings-SPECULATIVE (wrong=0), converges-idempotent, heartbeat-mines-then-converges,
mines-once-per-session (memoization), atomic-no-orphan. 55 idle/contemplation/always-on +
96 invariants/contemplation + 28 smoke green; `core always-on --contemplate-frontier` mines
end-to-end. engine_state contemplation_runs are per-life runtime state (gitignored).

* fix(l10): isolate idle frontier pass + persist frontier_findings (#758)

Three review blockers on the autonomous idle frontier-contemplation pass:

1. Daemon isolation. idle_tick's pass 2b called run_contemplation /
   write_contemplation_run bare inside the always-on loop (run_continuous
   wraps idle_tick in nothing but an exit-checkpoint finally), so a malformed
   frontier report or a transient FS fault would propagate out and KILL the
   continuous life. Wrap the optional pass best-effort (the exit-checkpoint
   boundary idiom): degrade to 0 findings + RuntimeWarning, leave did_work
   untouched, let a later beat retry.

2. Durable did_work explanation. HeartbeatRecord carried frontier_findings
   but serialize_report omitted it and AlwaysOnReport had no run total, so
   lived_life.json could show did_work=true with facts=0/proposals=0 and no
   persisted cause. Persist frontier_findings per-record + total_frontier_findings.

3. Honest HITL claim. The docstring/CLI/config said findings are "reviewable
   via the HITL path", but the existing HITL queue + workbench reader scan
   <repo>/contemplation/runs while the pass writes <engine_state>/contemplation_runs/.
   Narrow the claim to "persisted reviewable artifact, not yet in the HITL
   queue" (wiring that discovery is a follow-up).

Tests: 4 new non-vacuous regressions (mining-failure degrade, write-failure
degrade, always-on loop survives frontier failure, lived_life persists
frontier_findings) — all fail against the unpatched pass. 38 L10/always-on/
workbench tests green.
2026-06-14 23:59:49 -07:00

198 lines
6.5 KiB
Python

from __future__ import annotations
import hashlib
import json
import os
from pathlib import Path
from typing import Iterable
from core.contemplation.miners.contradiction_detection import (
mine_contradiction_detection_report,
)
from core.contemplation.miners.frontier_compare import mine_frontier_compare_report
from core.contemplation.schema import (
ContemplationFinding,
ContemplationRun,
format_contemplation_finding_jsonl,
)
from core.contemplation.snapshot import ContemplationSubstrate
from teaching.discovery_sink import DiscoveryCandidateSink
def _config_hash(payload: dict[str, object]) -> str:
encoded = json.dumps(
payload,
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
)
return hashlib.sha256(encoded.encode("utf-8")).hexdigest()
def _emit_findings(
findings: Iterable[ContemplationFinding],
sink: DiscoveryCandidateSink | None,
) -> None:
"""Stream each finding through the shared sink protocol when set.
No-op when *sink* is ``None`` — preserves the existing "build a
``ContemplationRun`` blob" path for callers that want a single
in-memory result.
When a sink is supplied, each finding is emitted as one canonical
JSONL line via :func:`format_contemplation_finding_jsonl`. This
is the unification point with the discovery candidate stream
(ADR-0055 Phase B sinks): both flow through the same
``DiscoveryCandidateSink`` protocol, both land in append-only
monthly JSONL files when paired with
:class:`teaching.discovery_sink.DiscoveryMonthlyFileSink`.
Sink errors are NOT swallowed — see ADR-0055 fail-fast contract.
"""
if sink is None:
return
for finding in findings:
sink.emit(format_contemplation_finding_jsonl(finding))
def contemplate_frontier_reports(
report_paths: Iterable[str | Path],
*,
pack_ids: Iterable[str] = (),
notes: Iterable[str] = (),
sink: DiscoveryCandidateSink | None = None,
) -> ContemplationRun:
"""Run ADR-0080 Phase 1 over explicit frontier-compare reports.
The runner is read-only. It does not discover files implicitly, does not
mutate packs, does not write teaching examples, and does not promote any
finding beyond SPECULATIVE.
When *sink* is supplied each finding is also emitted as one
canonical JSONL line via the shared
:class:`teaching.discovery_sink.DiscoveryCandidateSink` protocol,
so contemplation findings flow into the same append-only stream
discovery candidates use.
"""
paths = tuple(Path(p) for p in report_paths)
substrate = ContemplationSubstrate.from_report_paths(
paths,
pack_ids=tuple(pack_ids),
notes=tuple(notes),
)
findings: list[ContemplationFinding] = []
for path in paths:
findings.extend(
mine_frontier_compare_report(
path,
substrate_hash=substrate.substrate_hash,
)
)
_emit_findings(findings, sink)
config_hash = _config_hash(
{
"runner": "contemplate_frontier_reports",
"report_paths": [str(p) for p in paths],
"pack_ids": tuple(sorted(set(pack_ids))),
"notes": tuple(notes),
}
)
return ContemplationRun(
substrate_hash=substrate.substrate_hash,
config_hash=config_hash,
findings=tuple(findings),
)
def run_contemplation(
report_paths: Iterable[str | Path] | None = None,
*,
pack_ids: Iterable[str] = (),
notes: Iterable[str] = (),
) -> ContemplationRun:
"""Run ADR-0080 Phase 1 over frontier-compare reports.
This is the stable operator-facing entry point for Phase 1. If no
explicit paths are supplied it reads the checked-in
``evals/frontier_compare/results/*.json`` reports in deterministic
path order. It never writes packs, teaching examples, proposal logs,
or discovery sinks.
"""
if report_paths is None:
root = Path(__file__).resolve().parents[2]
paths = tuple(sorted(root.glob("evals/frontier_compare/results/*.json")))
else:
paths = tuple(Path(p) for p in report_paths)
return contemplate_frontier_reports(
paths,
pack_ids=pack_ids,
notes=notes,
sink=None,
)
def contemplate_contradiction_reports(
report_paths: Iterable[str | Path],
*,
pack_ids: Iterable[str] = (),
notes: Iterable[str] = (),
sink: DiscoveryCandidateSink | None = None,
) -> ContemplationRun:
"""Run ADR-0080 Phase 1 over explicit contradiction-detection reports.
Mirrors :func:`contemplate_frontier_reports` for the
``evals/contradiction_detection`` lane. Same read-only guarantees,
same SPECULATIVE-only finding contract, separate runner so the
config hash records which lane was contemplated.
"""
paths = tuple(Path(p) for p in report_paths)
substrate = ContemplationSubstrate.from_report_paths(
paths,
pack_ids=tuple(pack_ids),
notes=tuple(notes),
)
findings: list[ContemplationFinding] = []
for path in paths:
findings.extend(
mine_contradiction_detection_report(
path,
substrate_hash=substrate.substrate_hash,
)
)
_emit_findings(findings, sink)
config_hash = _config_hash(
{
"runner": "contemplate_contradiction_reports",
"report_paths": [str(p) for p in paths],
"pack_ids": tuple(sorted(set(pack_ids))),
"notes": tuple(notes),
}
)
return ContemplationRun(
substrate_hash=substrate.substrate_hash,
config_hash=config_hash,
findings=tuple(findings),
)
def write_contemplation_run(run: ContemplationRun, path: str | Path) -> None:
# Atomic write (temp + os.replace): the always-on life is an indefinite-uptime process
# where a SIGKILL/crash mid-write is expected, and the idle-pass skip-guard would NEVER
# repair a torn canonical file. A crash leaves only the orphan .tmp; the canonical path
# is either absent (re-mined next boot) or complete — never a half-written artifact.
target = Path(path)
target.parent.mkdir(parents=True, exist_ok=True)
payload = json.dumps(run.as_dict(), ensure_ascii=False, indent=2, sort_keys=True) + "\n"
tmp = target.with_name(f".{target.name}.{os.getpid()}.tmp")
tmp.write_text(payload, encoding="utf-8")
os.replace(tmp, target)
__all__ = [
"contemplate_contradiction_reports",
"contemplate_frontier_reports",
"run_contemplation",
"write_contemplation_run",
]