Commit graph

20 commits

Author SHA1 Message Date
Shay
24daebf3c1 feat(pack-resolver): gloss resolver with lexicon-residency + dual-checksum hardening
Lands the gloss-loader scaffolding from feat/pack-glosses-wip onto
main, with every hardening item from the 2026-05-19 design review
built in from the start.  No glosses ship in this commit — only the
infrastructure that will consume them safely.

Hardening items (each pinned by a test):

1. Lexicon-residency check in resolve_gloss()
   chat/pack_resolver.py — resolve_gloss now requires the lemma to be
   present in the same pack's lexicon.jsonl BEFORE consulting
   glosses.jsonl.  Without this, glosses.jsonl would become a parallel
   surface-authoring channel that bypasses the lexicon's checksum
   seal: someone could ship a gloss for a lemma the pack never
   ratified, and the runtime would emit it as if it were pack content.

   Test: TestLexiconResidencyEnforced::test_gloss_for_unratified_lemma_is_rejected
   authors a gloss for ``gamma`` (a lemma not in the lexicon) and
   asserts resolve_gloss returns None.

2. Dual-checksum manifest support
   language_packs/schema.py — LanguagePackManifest gains an OPTIONAL
   ``glosses_checksum: str | None`` field.  Glosses are an additive
   overlay; bumping the glosses_checksum does NOT perturb the
   immutable lexicon checksum.
   language_packs/compiler.py — _load_pack_cached now verifies
   bytes-on-disk of glosses.jsonl against the manifest's
   glosses_checksum when present.  Missing field on legacy packs is
   back-compat (no verification, no raise).  Mismatch raises
   ValueError exactly like the lexicon checksum gate.

   Tests:
     test_matching_glosses_checksum_loads_clean — happy path
     test_checksum_mismatch_raises — tampered file rejected
     test_missing_glosses_checksum_is_back_compat — legacy packs OK

3. clear_resolver_cache() clears BOTH lexicon AND glosses LRU caches
   Previously only cleared _pack_lexicon_for, so test fixtures that
   wrote glosses.jsonl mid-process would see stale (empty) gloss data
   on subsequent resolve_gloss calls.

   Test: TestClearResolverCacheClearsBoth proves the issue exists
   without the clear, then proves the new code fixes it.

4. Malformed JSONL lines silently skipped
   A single bad line in glosses.jsonl must not break resolution for
   the rest of the pack.  Same defensive parsing as _pack_lexicon_for.
   Entries missing required fields (lemma, gloss, or empty values)
   are also skipped.

   Tests:
     test_malformed_line_skipped — invalid JSON between valid lines
     test_entry_missing_required_field_skipped — 4 bad shapes filtered

5. Missing glosses.jsonl is back-compat
   _pack_glosses_for returns an empty dict when the file is absent.
   resolve_gloss returns None.  No exception.  All 9 currently-
   ratified English packs ship with no glosses.jsonl — they must
   continue to load cleanly.

   Tests:
     test_pack_with_no_glosses_returns_empty
     test_resolve_gloss_on_lemma_without_gloss_file_returns_none

Files:
  chat/pack_resolver.py
    + _pack_glosses_for (cached loader)
    + resolve_gloss (lexicon-residency-gated lookup)
    * clear_resolver_cache now clears both caches
  language_packs/schema.py
    + LanguagePackManifest.glosses_checksum field (optional)
  language_packs/compiler.py
    + dual-checksum verification block in _load_pack_cached
    + glosses_checksum field passed through to the manifest dataclass
  tests/test_pack_resolver_glosses.py
    11 tests covering all five hardening items

Verification:
  11/11 new tests green.
  Full cognition eval byte-identical.
  All currently-ratified packs continue to load without glosses.
2026-05-19 07:24:36 -07:00
Shay
140b6fea37 feat(adr-0051): trust-boundary hardening pass 2026-05-18 07:09:55 -07:00
Shay
64c5bc4619 feat(epistemic): truth-seeking schema audit — 3 leaks closed, 4 new lanes, 3 new invariants
Audit of the one-mutation-path invariant (ADR-0021 §3) found three leaks
where pack authority or session-state writes could substitute for coherence
judgment. All three landed fixes or partial closures in this push.

Leaks closed:
- Leak A: pack vocab defaulted to COHERENT — flipped to SPECULATIVE in
  language_packs/{compiler,schema}.py; docstring corrected to align with
  ADR-0021 (it was rationalizing the leak).
- Leak B: vault.recall was epistemic-blind — VaultStore.store() now stamps
  every entry with EpistemicStatus (default SPECULATIVE); recall(min_status=)
  filters to admissible-as-evidence tier. All 4 vault-write sites updated.
- Leak C (write-side): generate/proposition.py:198 stored articulated
  propositions unmarked — now stamps SPECULATIVE, breaking the
  fabrication-feedback loop in principle. Read-side audit of 5 call sites
  is the residual.

New architectural invariants (tests/test_architectural_invariants.py):
- INV-21: one-mutation-path allowlist (caught Leak C on first run)
- INV-22: pack lexicon default is SPECULATIVE (Leak A guard)
- INV-23: vault recall epistemic-aware (Leak B guard)

New eval lanes:
- teaching_injection_resistance — ships GREEN at 1.00/1.00/0 (the
  structural anti-injection claim is real and measurable)
- refusal_calibration — honest gap: 0% refusal, 0% fabrication
- contradiction_detection — honest gap: 50% flag via versor-delta heuristic,
  100% false-positive; motivates the proper coherence-checker
- articulation_of_status — honest gap: 0% speculative articulation, 60%
  false certainty; output-side leak surface

New benchmarks:
- benchmarks/footprint.py — total deployed runtime is 7.06 MiB
  (109,358x smaller than Llama 3.1 405B, runs offline, no GPU)
- benchmarks/learning_curve.py — monotonic + replay-deterministic curve
  per lane

Documentation:
- docs/truth_seeking_schema.md — foundational architectural commitment,
  five rules, mapped to human failure modes, leaks published openly
- evals/CLAIMS.md — five-tier public claims doc; Tier 4.5 publishes
  known gaps with named fixes; verification contract at top
- README.md — new pillar between algebraic substrate and language pillar

Includes in-flight formation pipeline scaffolding (formation/, tests/formation/,
docs/formation_pipeline_plan.md) and minor CLI/contracts/gitignore edits
that were already in the working tree at session start.

Verification: 798 passed, 2 skipped, 1 deselected (pre-existing pack-count
test drift unrelated to schema changes).
2026-05-17 07:27:41 -07:00
Shay
ef95d3e609 feat(adr-0021): epistemic_status surface wired across teaching + trace
ADR-0021 v1 schema land. epistemic_status is a position in the revision
graph, not a source-trust tier — coherence is the only admission signal.

Surfaces:
- teaching/epistemic.py: EpistemicStatus enum (COHERENT, CONTESTED,
  SPECULATIVE, FALSIFIED); ADMISSIBLE_AS_EVIDENCE = {COHERENT}.
- PackMutationProposal.epistemic_status (default SPECULATIVE) + immutable
  with_status() updater.
- ReviewedTeachingExample.epistemic_status (default SPECULATIVE);
  orthogonal to acceptance per ADR §Schema impact.
- LexicalEntry.epistemic_status (default "coherent" for seed; absent in
  JSONL is treated as the seed default — no retroactive tagging).
- compute_trace_hash + trace_hash_from_result + pipeline.py fold the
  load-bearing proposal's epistemic_status into the trace hash so
  replay detects different epistemic frames.

Non-hardening invariant (ADR-0021 §2): tests/test_epistemic_invariants.py
asserts no final/frozen/axiom/permanent flag on PackMutationProposal or
ReviewedTeachingExample, and EpistemicStatus contains no source-trust
tier names.

Docs: docs/runtime_contracts.md gains an Epistemic surface section.

Lanes green: smoke 27/27, teaching 10/10, packs 6/6, runtime 19/19,
cognition eval 100%.
2026-05-16 20:20:35 -07:00
Shay
40cabdec09
Speed up validation lanes and pack loading
- add core test --suite fast for lightweight iteration validation
- cache parsed pack entries and compiled/mounted language packs
- return defensive manifold/list copies so transient mutations cannot leak through caches
- add CLI fast-suite coverage and pack cache isolation tests
- preserve exact recall, backend dispatch, and runtime semantics
2026-05-15 08:26:11 -07:00
Shay
6bad4189d2 Implement core physics and pack validation 2026-05-14 12:35:19 -07:00
Shay
fbbd7c52e3 Fix fail-closed versor construction 2026-05-14 12:13:04 -07:00
Shay
30757ccc63
Add runtime output-language policy
Add RuntimeConfig with English default output policy, wire output language through runtime/frame selection/generation/CLI, preserve language metadata in mounted manifolds, and add runtime/CLI policy tests.
2026-05-13 21:29:43 -07:00
Shay
531acfd40b Implement trilingual field coherence 2026-05-13 19:53:37 -07:00
Shay
6bb2eb348f
Fix language-pack versor hemisphere canonicalization
Compile language-pack features into even-grade rotors, apply canonicalization after alignment nudges. Preserves holonomy parity across token counts. 231 tests passing.
2026-05-13 19:41:31 -07:00
Shay
747289ace7 Fix morphology weight hierarchy: root/stem dominate, inflection/suffix are perturbations
Root was 0.17, stem 0.10 — not heavy enough to anchor same-root forms
against the diverging suffix rotor (ים landing on a different axis).

New hierarchy:
  triliteral root  0.13 → 0.22  (shared identity, strongest anchor)
  root             0.17 → 0.30  (primary root geometry)
  stem             0.10 → 0.18  (secondary, same-root forms cluster here)
  inflection role  0.02 → 0.015 (key label, minimal)
  inflection value 0.05 → 0.03  (number/gender/etc, perturbation only)
  prefix           0.05 → 0.03  (per-position decay preserved)
  suffix           0.04 → 0.02  (per-position decay preserved)

This ensures דבר and דברים both orbit the D-B-R root point closely
enough that cga_inner(singular, plural) > cga_inner(singular, unrelated),
while still encoding morphological distinctions as measurable offsets.
2026-05-13 19:16:38 -07:00
Shay
db7eabd62f Wire alignment graph into load_pack as post-compilation correction pass
- VocabManifold.update(): replace a stored versor in-place (grade-norm
  enforced, KeyError if word missing)
- compile_entries_to_manifold() now also returns entry_id->surface map
  as a second return value; callers that only need the manifold use [0]
- _alignment_nudge_rotor(): slerp-style rotor that rotates source toward
  target by edge.weight * ALIGNMENT_NUDGE_STRENGTH (0.06) — small enough
  to preserve intra-pack geometry, large enough to pull cross-lang pairs
  into proximity
- load_pack(): after both manifolds are built, loads alignment.jsonl,
  resolves entry_id->surface on each side, applies nudge to source versor
  using the already-compiled target versor as the attractor
- _is_hebrew_root() guard retained; _triliteral_root() Hebrew-only
2026-05-13 15:40:33 -07:00
Shay
5e75d46323 Fix Greek triliteral rotor bleed and update structured-morphology baseline test
- Guard _triliteral_root() rotor to Hebrew-script roots only; Greek/other
  scripts now use the root: rotor alone (0.17) — no spurious uppercase
  Unicode blade collision from the romanization fallback path
- test_structured_morphology_improves_same_root_hebrew_resonance: replace
  tag_only baseline (now empty since legacy tags stripped) with no_morphology
  baseline (domain+pos+lemma+surface only), which is the honest comparison
  for what structured morphology contributes post-migration
2026-05-13 15:24:38 -07:00
Shay
400cc031ce Remove legacy root-tag guard; compiler now single-path structured morphology 2026-05-13 15:21:14 -07:00
Shay
5c952a9f9d Migrate Hebrew lexicon to structured morphology 2026-05-13 15:15:15 -07:00
Shay
3168e73ace Wire morphology operator composition 2026-05-13 15:02:52 -07:00
Shay
8d09c2a8c1 Add morphology registry for language packs 2026-05-13 14:50:36 -07:00
Shay
4e7c29b84a Fix field state introspection and pack manifold geometry 2026-05-13 14:24:29 -07:00
Shay
a87c7a9c6f Fix full test suite after cognitive runtime 2026-05-13 13:52:11 -07:00
Shay
71e99c5c51 Build first cognitive response path 2026-05-13 13:40:06 -07:00