aed273b14a
The always-on PROCESS shipped (run_continuous + daemon + CLI), but the claim "it holds over uptime" was UNFALSIFIED: the existing evals/l10_continuity soak drives the TURN loop and is disjoint from run_continuous; the daemon path was covered only by ≤5-beat unit tests. This lane converts "built" → "proven over horizon" for the idle path. evals/l10_always_on/ mirrors the l10_continuity predicate/mutation harness but drives the IDLE heartbeat: it seeds a real continuous life (held self + a cognitive turn to excite the field), runs N beats with NO user turn, and gates four falsifiable predicates, each with a *_holds (real soak) AND a *_bites (mutation) test per CLAUDE.md schema-as-proof: - H1 closure — every observed idle beat versor_condition < 1e-6 (held over uptime, READ never repaired); bites on a breached beat AND on a vacuous no-field soak. - H2 bounded idle — a no-work beat adds NOTHING to the vault (no idle leak — invisible at 5 beats, fatal at 100k); bites on idle vault growth. - H3 convergence — a saturated idle life SETTLES and stays settled (no re-awakening), at rest at the end, closure intact on the tail; bites on never-settling + on re-awakening. - H4 reboot resume — a reboot mid-soak resumes the SAME life (strict identity guard passes, pre-reboot DERIVED learning survives, post-reboot closure holds); bites on failed resume + on lost learning. Measured (5000 idle beats, reboot@2500, ~20s): ALL gates pass — versor_condition flat at 1.389e-07 (no drift, no repair), vault bounded at 6 (no idle leak), converged at beat 1 with a 4999-beat at-rest tail, reboot resumed the same life with learning intact. This is the empirical resolution of the L10 riskiest-unknown FOR THE IDLE PATH (the closure-by- construction ruling covered the field-transition walk; this covers indefinite idle uptime). Honestly NOT covered (recorded in the report, no silent skip): the continuously-LEARNING life's resource cost under a sustained new-fact stream (O(n²) snapshot; per-run lived_life) — out of scope until an afferent/intake feed + incremental persistence exist (the next frontier). Run on demand: PYTHONPATH=. python -m evals.l10_always_on [n_beats] [reboot_beat]. Path- scoped out of CI smoke (like l10_continuity); runs in post-merge full-pytest. Additive — a new eval lane, no existing files touched. 12/12 lane tests + architectural invariants green.
3.7 KiB
L10 Always-On Heartbeat Soak — Contract
Status: soak (falsifiable long-horizon gate) · Not in default smoke (run on demand / nightly).
The continuity lane (evals/l10_continuity) soaks the turn loop. This lane soaks the
idle heartbeat (chat/always_on.run_continuous — the loop the core always-on daemon
drives). It seeds a real continuous life (a held self + a cognitive turn to excite the
field), then runs the engine over N beats with no user turn, and evaluates falsifiable
predicates over the per-beat evidence. It converts the claim "the always-on process is
built" into "the always-on process holds over uptime" — the idle-path claims that the
short daemon unit tests (≤5 beats) and the turn-loop soak (disjoint from run_continuous)
do not prove at horizon.
Run: PYTHONPATH=. .venv/bin/python -m evals.l10_always_on [n_beats] [reboot_beat]
(pass a large n_beats — e.g. 100000 — for a true long-horizon soak; default 24).
Predicates
| ID | Proves | Fails loudly when | Mutation-verified bite |
|---|---|---|---|
| H1 closure | every OBSERVED idle beat has versor_condition < 1e-6 (closure holds over idle uptime, READ never repaired) |
the idle heartbeat drifts/corrupts the field, or the field never existed (vacuous) | a beat with versor_condition ≥ 1e-6; an all-None (no-field) soak |
| H2 bounded idle | a did_work=False beat adds NOTHING to the vault (no idle resource leak) |
a converged idle life keeps growing a store — invisible at 5 beats, fatal at 100k | an idle beat whose vault_size grew over the prior beat |
| H3 convergence | a saturated idle life SETTLES and stays settled (no re-awakening), at rest at the end, closure intact on the tail | the life churns forever, thrashes (work-after-rest), or breaks closure once settled | a never-settling run; a rest→work re-awakening |
| H4 reboot resume | a reboot mid-soak resumes the SAME life (strict identity guard passes, pre-reboot DERIVED learning survives, post-reboot closure holds) | a reboot forks a new life, loses learning, or breaks closure | resumed_cleanly=False; learned_fact_survived=False |
Each predicate has a *_holds test (real soak) and a *_bites test (mutation), per the
CLAUDE.md schema-as-proof discipline: a predicate that cannot fail under the violation it
nominally catches is decoration, not proof.
The measured result
On a 5000-beat soak (reboot at 2500): all gates pass. versor_condition is flat at
1.389e-07 across all 5000 beats (no drift — the idle heartbeat never perturbs the field,
no repair), the vault stays bounded at 6 entries (no idle leak), the life converges at
beat 1 and the 4999-beat tail stays at rest with closure intact, and the reboot at 2500
resumes the same life with its derived learning intact. This is the empirical resolution of
the L10 riskiest-unknown for the idle path (the closure-by-construction ruling covered
the field-transition walk; this covers indefinite idle uptime).
Not covered (no silent skips)
- H5 — learning-life resource cost. This lane proves the idle (converged) life is
resource-bounded. The cost of a continuously-learning life under a sustained
new-fact stream — the full-snapshot checkpoint is O(n²) in facts,
lived_life.jsonis per-run — is out of scope until an afferent/intake feed and incremental persistence exist. Recorded asnot_coveredin the report; a follow-up.
The deterministic_digest in the report freezes the per-beat shape (did_work /
field_valid / learning counts / vault size — all deterministic) + the verdicts, excluding
the machine-variant raw versor_condition float. Pin it once the lane is trusted so a
regression flips it.