Add agent efficiency and security doctrine
- update AGENTS.md with standing efficiency/performance and security doctrine - align CLAUDE.md with current performance/security expectations - update Copilot/Codex instructions with hot-path, trust-boundary, and CLI validation defaults - refresh work sequencing now that eval and calibration are on main
This commit is contained in:
parent
366f7a08c4
commit
f973e61bc2
3 changed files with 199 additions and 20 deletions
49
.github/copilot-instructions.md
vendored
49
.github/copilot-instructions.md
vendored
|
|
@ -24,6 +24,8 @@ The cognitive path is centered on:
|
||||||
- `generate/graph_planner.py`
|
- `generate/graph_planner.py`
|
||||||
- `generate/realizer.py`
|
- `generate/realizer.py`
|
||||||
- `teaching/correction.py`, `teaching/review.py`, `teaching/store.py`
|
- `teaching/correction.py`, `teaching/review.py`, `teaching/store.py`
|
||||||
|
- `evals/*`
|
||||||
|
- `calibration/*`
|
||||||
- `language_packs/data/en_core_cognition_v1`
|
- `language_packs/data/en_core_cognition_v1`
|
||||||
|
|
||||||
The runtime response contract is documented in `docs/runtime_contracts.md`.
|
The runtime response contract is documented in `docs/runtime_contracts.md`.
|
||||||
|
|
@ -53,6 +55,36 @@ Forbidden hot-path repair sites:
|
||||||
Do not add grade monitors, drift timers, watchdog repair functions, ANN/HNSW,
|
Do not add grade monitors, drift timers, watchdog repair functions, ANN/HNSW,
|
||||||
cosine similarity, or approximate recall.
|
cosine similarity, or approximate recall.
|
||||||
|
|
||||||
|
## Efficiency and Performance
|
||||||
|
|
||||||
|
Treat performance as part of the architecture. Slow feedback causes poor
|
||||||
|
engineering decisions and hides regressions.
|
||||||
|
|
||||||
|
When touching hot paths, prefer:
|
||||||
|
|
||||||
|
- backend-dispatched algebra when semantics match
|
||||||
|
- import hoisting and removal of repeated structure-building
|
||||||
|
- deterministic immutable caches or safe copied data
|
||||||
|
- exact CGA batching/vectorization instead of approximate search
|
||||||
|
- small validation lanes and bounded eval cases for iterative work
|
||||||
|
|
||||||
|
Do not improve speed by weakening invariants, skipping construction checks,
|
||||||
|
adding hot-path repair, using approximate recall, or mutating shared cached state
|
||||||
|
unsafely.
|
||||||
|
|
||||||
|
## Security and Trust Boundaries
|
||||||
|
|
||||||
|
When touching user-controlled text, dynamic imports, filesystem paths, CLI
|
||||||
|
reports, pack validators, or logs, enforce and test the trust boundary.
|
||||||
|
|
||||||
|
Required defaults:
|
||||||
|
|
||||||
|
- arbitrary-code execution must be explicit and opt-in
|
||||||
|
- unsafe pack IDs and path traversal must be rejected
|
||||||
|
- raw user text should not be leaked in expanded logging unless local/debug is explicit
|
||||||
|
- pack mutations stay proposal-only unless a reviewed path applies them
|
||||||
|
- report/file writes must be bounded to caller-specified paths with clear behavior
|
||||||
|
|
||||||
## Surface Contract
|
## Surface Contract
|
||||||
|
|
||||||
Keep these separate:
|
Keep these separate:
|
||||||
|
|
@ -90,20 +122,31 @@ core test --suite packs -q
|
||||||
core test --suite runtime -q
|
core test --suite runtime -q
|
||||||
core test --suite algebra -q
|
core test --suite algebra -q
|
||||||
core test --suite full -q
|
core test --suite full -q
|
||||||
|
core eval cognition
|
||||||
```
|
```
|
||||||
|
|
||||||
For a feature PR, run the smallest relevant suite and then `full` when
|
For a feature PR, run the smallest relevant suite and then `full` when
|
||||||
practical.
|
practical.
|
||||||
|
|
||||||
|
## Current Work Sequence
|
||||||
|
|
||||||
|
1. Keep CLI lanes and `core eval cognition` green.
|
||||||
|
2. Tighten hot-path backend consistency and semantics-preserving performance.
|
||||||
|
3. Harden pack/OOV/logging trust boundaries.
|
||||||
|
4. Add exact vault recall indexing/batching without approximate search.
|
||||||
|
5. Add Rust backend parity only after Python semantics are locked by tests.
|
||||||
|
6. Expand curriculum teaching after replay/eval/calibration remain deterministic.
|
||||||
|
|
||||||
## PR Standard
|
## PR Standard
|
||||||
|
|
||||||
Every change should state:
|
Every change should state:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
Capability added/protected:
|
Capability/performance/security boundary added or protected:
|
||||||
Invariant protected:
|
Invariant protected:
|
||||||
CLI suite run:
|
CLI suite/eval run:
|
||||||
No hidden normalization / stochastic fallback / unreviewed mutation:
|
No hidden normalization / stochastic fallback / approximate recall / unreviewed mutation:
|
||||||
|
Trust boundary enforced when relevant:
|
||||||
```
|
```
|
||||||
|
|
||||||
Prefer small PRs. Do not combine baseline repair, feature work, and broad
|
Prefer small PRs. Do not combine baseline repair, feature work, and broad
|
||||||
|
|
|
||||||
87
AGENTS.md
87
AGENTS.md
|
|
@ -118,8 +118,71 @@ Important modules:
|
||||||
- `generate/realizer.py` / `generate/templates.py` — deterministic realization.
|
- `generate/realizer.py` / `generate/templates.py` — deterministic realization.
|
||||||
- `teaching/*` — reviewed teaching/correction lifecycle.
|
- `teaching/*` — reviewed teaching/correction lifecycle.
|
||||||
- `language_packs/data/en_core_cognition_v1` — compact cognition seed pack.
|
- `language_packs/data/en_core_cognition_v1` — compact cognition seed pack.
|
||||||
|
- `evals/*` — deterministic cognition evidence harness.
|
||||||
|
- `calibration/*` — bounded replay-based operator calibration.
|
||||||
- `docs/runtime_contracts.md` — runtime response, memory, identity, and testing contracts.
|
- `docs/runtime_contracts.md` — runtime response, memory, identity, and testing contracts.
|
||||||
|
|
||||||
|
## Efficiency and Performance Doctrine
|
||||||
|
|
||||||
|
Performance is an architectural property. Do not treat it as an afterthought
|
||||||
|
that will be cleaned up after features land.
|
||||||
|
|
||||||
|
Before modifying hot paths, identify whether the change touches:
|
||||||
|
|
||||||
|
- algebra backend dispatch (`algebra/backend.py`)
|
||||||
|
- versor application / closure (`algebra/versor.py`)
|
||||||
|
- propagation (`field/propagate.py`)
|
||||||
|
- injection / OOV grounding (`ingest/gate.py`)
|
||||||
|
- vault recall/storage (`vault/store.py`)
|
||||||
|
- session turn loop (`session/context.py`)
|
||||||
|
- runtime/eval loops (`chat/runtime.py`, `core/cognition/*`, `evals/*`)
|
||||||
|
|
||||||
|
Required approach:
|
||||||
|
|
||||||
|
1. Prefer semantics-preserving cleanup before new knobs.
|
||||||
|
2. Route hot-path algebra through `algebra.backend` when semantics are identical.
|
||||||
|
3. Hoist repeated imports and repeated structure-building out of tight loops.
|
||||||
|
4. Cache only deterministic, immutable, or safely copied structures.
|
||||||
|
5. Keep exact CGA recall exact; optimize scans with batching/vectorization, not approximation.
|
||||||
|
6. Prove speed-oriented changes through existing CLI lanes and, when practical, small benchmark/eval evidence.
|
||||||
|
|
||||||
|
Never improve speed by:
|
||||||
|
|
||||||
|
- weakening `versor_condition` thresholds
|
||||||
|
- skipping closure checks at construction boundaries
|
||||||
|
- adding hot-path repair/normalization
|
||||||
|
- replacing exact CGA with cosine/ANN/HNSW
|
||||||
|
- hiding failures behind retry loops without telemetry
|
||||||
|
- mutating shared cached state unsafely
|
||||||
|
|
||||||
|
For test speed, prefer better validation lanes, small-case eval tests, fixture reuse where safe, and pack/load caching with immutability guarantees. Do not delete meaningful tests just because the full suite is slow.
|
||||||
|
|
||||||
|
## Security and Trust-Boundary Doctrine
|
||||||
|
|
||||||
|
Every agent must identify user-controlled input and dynamic execution surfaces.
|
||||||
|
Security hardening should be built into the same PRs that touch those surfaces.
|
||||||
|
|
||||||
|
High-risk surfaces:
|
||||||
|
|
||||||
|
- `core pack validate` dynamic validator execution
|
||||||
|
- language/source pack loading
|
||||||
|
- OOV token grounding and logs
|
||||||
|
- CLI commands that echo user input
|
||||||
|
- report/eval output paths
|
||||||
|
- pack mutation proposals
|
||||||
|
- any future file/network/database integration
|
||||||
|
|
||||||
|
Required approach:
|
||||||
|
|
||||||
|
1. Make arbitrary-code execution explicit and opt-in.
|
||||||
|
2. Reject path traversal and unsafe pack IDs before filesystem access.
|
||||||
|
3. Centralize display/log handling for user-controlled strings when expanding logging.
|
||||||
|
4. Keep pack mutation proposal-only unless an explicit reviewed path applies it.
|
||||||
|
5. Avoid leaking raw sensitive tokens in errors/reports unless the command is explicitly local/debug.
|
||||||
|
6. Preserve deterministic replay evidence for security-relevant decisions.
|
||||||
|
|
||||||
|
Do not add hidden background execution, dynamic imports from untrusted paths, shell passthroughs, or broad filesystem writes without an explicit trust boundary and tests.
|
||||||
|
|
||||||
## Chat Surface Contract
|
## Chat Surface Contract
|
||||||
|
|
||||||
Do not collapse these fields:
|
Do not collapse these fields:
|
||||||
|
|
@ -171,11 +234,12 @@ Never compute a manifest checksum from a pre-serialization Python string.
|
||||||
|
|
||||||
Current capability sequence:
|
Current capability sequence:
|
||||||
|
|
||||||
1. Keep CLI test suites green.
|
1. Keep CLI test suites and `core eval cognition` green.
|
||||||
2. Integrate semantic seed surfaces into realizer/cognition quality.
|
2. Tighten hot-path backend consistency and semantics-preserving performance.
|
||||||
3. Add cognitive eval harness.
|
3. Harden pack/OOV/logging trust boundaries.
|
||||||
4. Add operator calibration from deterministic replay evidence.
|
4. Add exact vault recall indexing/batching without approximate search.
|
||||||
5. Expand curriculum teaching only after the loop remains deterministic.
|
5. Add Rust backend parity only after Python semantics are locked by tests.
|
||||||
|
6. Expand curriculum teaching only after replay/eval/calibration remain deterministic.
|
||||||
|
|
||||||
Do not add dashboards, broad infra, or large test matrices unless they directly
|
Do not add dashboards, broad infra, or large test matrices unless they directly
|
||||||
protect or unlock one of the above capabilities.
|
protect or unlock one of the above capabilities.
|
||||||
|
|
@ -192,6 +256,7 @@ core test --suite packs -q
|
||||||
core test --suite runtime -q
|
core test --suite runtime -q
|
||||||
core test --suite algebra -q
|
core test --suite algebra -q
|
||||||
core test --suite full -q
|
core test --suite full -q
|
||||||
|
core eval cognition
|
||||||
```
|
```
|
||||||
|
|
||||||
For targeted work, run the smallest relevant suite first, then `full` before
|
For targeted work, run the smallest relevant suite first, then `full` before
|
||||||
|
|
@ -206,6 +271,9 @@ Good tests protect:
|
||||||
- identity protection
|
- identity protection
|
||||||
- reviewed correction safety
|
- reviewed correction safety
|
||||||
- semantic pack loadability and deterministic ordering
|
- semantic pack loadability and deterministic ordering
|
||||||
|
- eval/calibration determinism
|
||||||
|
- hot-path performance semantics
|
||||||
|
- explicit security trust boundaries
|
||||||
|
|
||||||
Bad tests preserve private helper shapes, stale constructors, punctuation trivia
|
Bad tests preserve private helper shapes, stale constructors, punctuation trivia
|
||||||
outside documented contracts, or legacy behavior that contradicts the current
|
outside documented contracts, or legacy behavior that contradicts the current
|
||||||
|
|
@ -216,10 +284,11 @@ architecture.
|
||||||
Every PR must answer:
|
Every PR must answer:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
What cognitive capability did this add or protect?
|
What cognitive capability, performance property, or security boundary did this add or protect?
|
||||||
What invariant proves it did not corrupt the field?
|
What invariant proves it did not corrupt the field?
|
||||||
Which CLI suite proves the relevant lane?
|
Which CLI suite/eval proves the relevant lane?
|
||||||
Did it avoid hidden normalization, stochastic fallback, and unreviewed mutation?
|
Did it avoid hidden normalization, stochastic fallback, approximate recall, and unreviewed mutation?
|
||||||
|
If it touches user input, files, dynamic imports, or logs, what trust boundary was enforced?
|
||||||
```
|
```
|
||||||
|
|
||||||
Prefer small, load-bearing PRs. Do not mix baseline fixes, feature work, and
|
Prefer small, load-bearing PRs. Do not mix baseline fixes, feature work, and
|
||||||
|
|
@ -230,4 +299,4 @@ large reorganization unless the coupling is unavoidable.
|
||||||
Raw input becomes a closed versor field once; thought evolves through exact
|
Raw input becomes a closed versor field once; thought evolves through exact
|
||||||
versor transitions and CGA recall; cognition is structured as intent,
|
versor transitions and CGA recall; cognition is structured as intent,
|
||||||
proposition graph, articulation target, deterministic realization, reviewed
|
proposition graph, articulation target, deterministic realization, reviewed
|
||||||
memory, and replayable trace.
|
memory, eval/calibration replay, and traceable evidence.
|
||||||
|
|
|
||||||
83
CLAUDE.md
83
CLAUDE.md
|
|
@ -22,6 +22,7 @@ CognitiveTurnPipeline
|
||||||
-> deterministic realizer
|
-> deterministic realizer
|
||||||
-> generation walk telemetry
|
-> generation walk telemetry
|
||||||
-> reviewed teaching loop
|
-> reviewed teaching loop
|
||||||
|
-> deterministic eval/calibration replay
|
||||||
-> deterministic trace hash
|
-> deterministic trace hash
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -99,8 +100,71 @@ runtime path. Vault recall is exact and deterministic.
|
||||||
- `generate/realizer.py` and `generate/templates.py` — deterministic surface realization.
|
- `generate/realizer.py` and `generate/templates.py` — deterministic surface realization.
|
||||||
- `teaching/correction.py`, `teaching/review.py`, `teaching/store.py` — reviewed teaching loop.
|
- `teaching/correction.py`, `teaching/review.py`, `teaching/store.py` — reviewed teaching loop.
|
||||||
- `language_packs/data/en_core_cognition_v1` — core cognition semantic seed pack.
|
- `language_packs/data/en_core_cognition_v1` — core cognition semantic seed pack.
|
||||||
|
- `evals/*` — deterministic cognition eval harness.
|
||||||
|
- `calibration/*` — bounded replay-based calibration.
|
||||||
- `docs/runtime_contracts.md` — response, telemetry, memory, identity, and testing contracts.
|
- `docs/runtime_contracts.md` — response, telemetry, memory, identity, and testing contracts.
|
||||||
|
|
||||||
|
## Efficiency and Performance Doctrine
|
||||||
|
|
||||||
|
Performance is part of correctness for this project because slow feedback hides
|
||||||
|
regressions and encourages unsafe shortcuts. Do not defer obvious hot-path or
|
||||||
|
validation-lane issues until “later.”
|
||||||
|
|
||||||
|
Before changing hot paths, identify whether the change touches:
|
||||||
|
|
||||||
|
- algebra backend dispatch
|
||||||
|
- versor application / closure
|
||||||
|
- propagation
|
||||||
|
- injection / OOV grounding
|
||||||
|
- vault recall/storage
|
||||||
|
- session turn loop
|
||||||
|
- runtime/eval loops
|
||||||
|
|
||||||
|
Required approach:
|
||||||
|
|
||||||
|
1. Prefer semantics-preserving cleanup before new knobs.
|
||||||
|
2. Use `algebra.backend` for hot-path algebra when semantics are identical.
|
||||||
|
3. Hoist repeated imports and repeated structure-building out of tight loops.
|
||||||
|
4. Cache deterministic immutable data only, or return safe copies.
|
||||||
|
5. Keep exact CGA recall exact; use batching/vectorization, not approximation.
|
||||||
|
6. Validate speed-oriented changes through CLI lanes and `core eval cognition`.
|
||||||
|
|
||||||
|
Never improve speed by weakening closure thresholds, skipping construction
|
||||||
|
checks, adding hot-path repair, replacing exact CGA with approximate metrics, or
|
||||||
|
mutating shared cached state unsafely.
|
||||||
|
|
||||||
|
For test speed, prefer curated CLI lanes, small-case eval tests, safe fixture
|
||||||
|
reuse, and immutable pack/load caching. Do not delete meaningful tests just
|
||||||
|
because the full suite is slow.
|
||||||
|
|
||||||
|
## Security and Trust Boundaries
|
||||||
|
|
||||||
|
Any change that touches user-controlled text, filesystem paths, dynamic imports,
|
||||||
|
reports, pack validators, or logs must state the trust boundary.
|
||||||
|
|
||||||
|
High-risk surfaces:
|
||||||
|
|
||||||
|
- `core pack validate` dynamic validator execution.
|
||||||
|
- language/source pack loading.
|
||||||
|
- OOV token grounding and error messages.
|
||||||
|
- CLI commands that echo user content.
|
||||||
|
- eval/report output paths.
|
||||||
|
- pack mutation proposals.
|
||||||
|
- future file/network/database integrations.
|
||||||
|
|
||||||
|
Required approach:
|
||||||
|
|
||||||
|
1. Make arbitrary-code execution explicit and opt-in.
|
||||||
|
2. Reject path traversal and unsafe pack IDs before filesystem access.
|
||||||
|
3. Centralize safe display/log handling before increasing logging.
|
||||||
|
4. Keep pack mutation proposal-only unless a reviewed path applies it.
|
||||||
|
5. Avoid leaking raw sensitive tokens unless the command is explicitly local/debug.
|
||||||
|
6. Preserve deterministic replay evidence for security-relevant decisions.
|
||||||
|
|
||||||
|
Do not add hidden background execution, dynamic imports from untrusted paths,
|
||||||
|
shell passthroughs, or broad filesystem writes without tests and a documented
|
||||||
|
trust boundary.
|
||||||
|
|
||||||
## Runtime Surface Contract
|
## Runtime Surface Contract
|
||||||
|
|
||||||
Keep these distinct:
|
Keep these distinct:
|
||||||
|
|
@ -155,6 +219,7 @@ core test --suite packs -q
|
||||||
core test --suite runtime -q
|
core test --suite runtime -q
|
||||||
core test --suite algebra -q
|
core test --suite algebra -q
|
||||||
core test --suite full -q
|
core test --suite full -q
|
||||||
|
core eval cognition
|
||||||
```
|
```
|
||||||
|
|
||||||
Run the smallest relevant suite first, then `full` before merge when practical.
|
Run the smallest relevant suite first, then `full` before merge when practical.
|
||||||
|
|
@ -163,11 +228,12 @@ Run the smallest relevant suite first, then `full` before merge when practical.
|
||||||
|
|
||||||
Current near-term sequence:
|
Current near-term sequence:
|
||||||
|
|
||||||
1. Keep CLI lanes green.
|
1. Keep CLI lanes and `core eval cognition` green.
|
||||||
2. Integrate semantic seed relations into realizer/cognition quality.
|
2. Tighten hot-path backend consistency and semantics-preserving performance.
|
||||||
3. Add cognitive eval harness.
|
3. Harden pack/OOV/logging trust boundaries.
|
||||||
4. Add deterministic operator calibration from replay evidence.
|
4. Add exact vault recall indexing/batching without approximate search.
|
||||||
5. Expand curriculum teaching after the loop is stable.
|
5. Add Rust backend parity only after Python semantics are locked by tests.
|
||||||
|
6. Expand curriculum teaching after replay/eval/calibration remain deterministic.
|
||||||
|
|
||||||
Avoid broad docs-first churn, dashboard work, or large infrastructure unless it
|
Avoid broad docs-first churn, dashboard work, or large infrastructure unless it
|
||||||
unlocks one of these steps.
|
unlocks one of these steps.
|
||||||
|
|
@ -177,10 +243,11 @@ unlocks one of these steps.
|
||||||
Before opening or merging, answer:
|
Before opening or merging, answer:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
What capability did this add or protect?
|
What capability, performance property, or security boundary did this add/protect?
|
||||||
Which invariant proves the field remains valid?
|
Which invariant proves the field remains valid?
|
||||||
Which CLI suite proves the lane?
|
Which CLI suite/eval proves the lane?
|
||||||
Did this avoid hidden normalization, stochastic fallback, and unreviewed mutation?
|
Did this avoid hidden normalization, stochastic fallback, approximate recall, and unreviewed mutation?
|
||||||
|
If it touches user input, files, dynamic imports, or logs, what trust boundary was enforced?
|
||||||
```
|
```
|
||||||
|
|
||||||
Prefer small, load-bearing PRs with clear evidence.
|
Prefer small, load-bearing PRs with clear evidence.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue