core-labs/core
core-labs/core
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

feat(telemetry): discovery-yield-per-served-turn baseline (ADR-0255) #104

Merged
core-labs merged 1 commit from feat/discovery-yield-baseline-telemetry into main 2026-07-23 15:00:35 +00:00
Conversation 0 Commits 1 Files changed 7 +540
core-labs commented 2026-07-23 14:39:15 +00:00
Owner
Copy link

2026-07-23 directive: instrument discovery-yield-per-served-turn telemetry, scoped exactly as ruled — "candidates proposed per served turn on clean, post-reset traffic."

Problem

The T11 reset (#102) wiped the discovery-candidate ledger 465→0 but deliberately left turn_count (14990) unchanged — resetting a scalar that tracks real historical turns would be fabrication. That leaves no way to measure repopulation speed: turn_count spans the whole pre-reset history, while the ledger is now clean. candidates / turn_count would silently blend pre- and post-reset traffic.

Fix

  • engine_state/__init__.py::save_manifest — new additive-optional turn_count_baseline: int | None = None param. Stamps the turn_count value at the moment the ledger was last reset. Does not bump schema_version (same discipline as engine_identity/parent_engine_identity).
  • teaching/discovery_yield.py (new) — pure compute_discovery_yield(store): served_turns_since_reset = turn_count - turn_count_baseline, candidates_since_reset = len(candidates), yield_rate = candidates/served_turns (or None when zero served turns — never a ZeroDivisionError). Fails closed: returns None when no baseline is stamped, rather than fabricating a denominator against an un-marked epoch.
  • core teaching discovery-yield [--json] (new CLI, core/cli_teaching.py + core/cli.py) — read-only, mirrors the core teaching coverage shape. Exits 1 with a clear stderr message when no baseline exists.
  • scripts/ops/stamp_discovery_yield_baseline_20260723.py (new) — the one-off that stamps the T11 reset epoch into the live store, mirroring reset_candidate_corpus_t11_20260722.py's atomic generation-dir pattern (begin_generation → carry forward everything faithfully → save_manifest(turn_count_baseline=turn_count) → commit_generation(keep=1)) with the same idempotent abort-if-already-stamped guard.
  • docs/adr/ADR-0255 — the decision, explicit non-goals (not a live/background rate — no always-on PROCESS exists yet; not a new telemetry sink; no serving-physics touched), and validation record.

Not yet executed

The stamping script has been dry-run verified against a throwaway copy of the live store's gen-21703 (idempotent guard confirmed, identity/candidates preserved byte-faithfully) but not yet run against the real live store — that mutation was blocked by the local sandbox as a cross-worktree write and needs to be run explicitly, either via CORE_ENGINE_STATE_DIR=<repo>/engine_state uv run python scripts/ops/stamp_discovery_yield_baseline_20260723.py after this merges, or by me with your go-ahead.

[Verification]

  • tests/test_discovery_yield.py — 13 passed (RED→GREEN, each gate has a _bites mutation-proof variant per CLAUDE.md schema-as-proof discipline)
  • tests/test_adr_0219_generation_checkpoint.py + tests/test_discovery_candidates.py + tests/test_conftest_engine_state_isolation.py + tests/test_adr_0146_engine_state.py — 73 passed (no regression in engine_state/discovery)
  • tests/test_cli.py + tests/test_cli_test_suites.py — 21 passed
  • pre-push gate fired live on push: smoke 180 passed (133.5s) + warmed_session lane 10 passed (141.0s) → gate PASSED

Base da3447e9, head de6e3929. Merge on your authorization (no-merge-automation).

2026-07-23 directive: instrument `discovery-yield-per-served-turn` telemetry, scoped exactly as ruled — **"candidates proposed per served turn on clean, post-reset traffic."** ## Problem The T11 reset (#102) wiped the discovery-candidate ledger 465→0 but deliberately left `turn_count` (14990) unchanged — resetting a scalar that tracks real historical turns would be fabrication. That leaves no way to measure repopulation speed: `turn_count` spans the whole pre-reset history, while the ledger is now clean. `candidates / turn_count` would silently blend pre- and post-reset traffic. ## Fix - **`engine_state/__init__.py::save_manifest`** — new additive-optional `turn_count_baseline: int | None = None` param. Stamps the `turn_count` value at the moment the ledger was last reset. Does not bump `schema_version` (same discipline as `engine_identity`/`parent_engine_identity`). - **`teaching/discovery_yield.py`** (new) — pure `compute_discovery_yield(store)`: `served_turns_since_reset = turn_count - turn_count_baseline`, `candidates_since_reset = len(candidates)`, `yield_rate = candidates/served_turns` (or `None` when zero served turns — never a `ZeroDivisionError`). **Fails closed**: returns `None` when no baseline is stamped, rather than fabricating a denominator against an un-marked epoch. - **`core teaching discovery-yield [--json]`** (new CLI, `core/cli_teaching.py` + `core/cli.py`) — read-only, mirrors the `core teaching coverage` shape. Exits 1 with a clear stderr message when no baseline exists. - **`scripts/ops/stamp_discovery_yield_baseline_20260723.py`** (new) — the one-off that stamps the T11 reset epoch into the live store, mirroring `reset_candidate_corpus_t11_20260722.py`'s atomic generation-dir pattern (`begin_generation` → carry forward everything faithfully → `save_manifest(turn_count_baseline=turn_count)` → `commit_generation(keep=1)`) with the same idempotent abort-if-already-stamped guard. - **`docs/adr/ADR-0255`** — the decision, explicit non-goals (not a live/background rate — no always-on PROCESS exists yet; not a new telemetry sink; no serving-physics touched), and validation record. ## Not yet executed The stamping script has been **dry-run verified** against a throwaway copy of the live store's `gen-21703` (idempotent guard confirmed, identity/candidates preserved byte-faithfully) but **not yet run against the real live store** — that mutation was blocked by the local sandbox as a cross-worktree write and needs to be run explicitly, either via `CORE_ENGINE_STATE_DIR=<repo>/engine_state uv run python scripts/ops/stamp_discovery_yield_baseline_20260723.py` after this merges, or by me with your go-ahead. ## [Verification] - `tests/test_discovery_yield.py` — 13 passed (RED→GREEN, each gate has a `_bites` mutation-proof variant per CLAUDE.md schema-as-proof discipline) - `tests/test_adr_0219_generation_checkpoint.py` + `tests/test_discovery_candidates.py` + `tests/test_conftest_engine_state_isolation.py` + `tests/test_adr_0146_engine_state.py` — 73 passed (no regression in engine_state/discovery) - `tests/test_cli.py` + `tests/test_cli_test_suites.py` — 21 passed - pre-push gate fired live on push: smoke **180 passed** (133.5s) + warmed_session lane **10 passed** (141.0s) → gate PASSED Base `da3447e9`, head `de6e3929`. Merge on your authorization (no-merge-automation).
core-labs added 1 commit 2026-07-23 14:39:16 +00:00 
2026-07-23 directive: instrument candidates-proposed-per-served-turn on
clean, post-reset traffic. Adds an additive-optional turn_count_baseline
manifest field (engine_state) stamping the turn_count at the last clean
reset of the discovery-candidate ledger, a pure compute_discovery_yield
function (teaching/discovery_yield.py), and a read-only `core teaching
discovery-yield` CLI surface. Fails closed (returns None / exits 1) when
no baseline has been stamped rather than fabricating a denominator.

scripts/ops/stamp_discovery_yield_baseline_20260723.py stamps the T11
reset epoch into the live store, mirroring reset_candidate_corpus_t11's
atomic generation-dir pattern. Dry-run verified against a throwaway copy
of the live store's gen-21703; not yet executed against the real store
(see PR body).
core-labs merged commit 6a54d27a78 into main 2026-07-23 15:00:35 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: core-labs/core#104
No description provided.
Delete branch "feat/discovery-yield-baseline-telemetry"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?