Commit graph

3 commits

Author SHA1 Message Date
Shay
45272a7bb2 feat: ADR-0111 physics expert-demo promotion (second successful)
Second worked promotion exercising the ADR-0106 + ADR-0109 contract
on a domain distinct from mathematics_logic. No contract change.

Evidence:
- foundational_physics_ood: accuracy=1.0 (117/117 public, 39/39 holdout)
- inference_closure: all_pass_rate=1.0 (shared with math, distinct digest via domain_id)
- fabrication_control: refused=n, fabricated=0 across all classes (shared)

Signed claim digest: a104cad136f3219df05dc7ce6a78437c02f7b5827cd3cdce568db3acda6a43ed

Bridge landed: cases_plaintext.jsonl dev-mode fallback for
foundational_physics_ood (matches ADR-0105 convention; analogous to the
math/inference bridges in ADR-0110). One small file, not a contract change.

Tests:
- tests/test_adr_0111_physics_expert_demo.py — 4 invariants, 6 cases
- tests/test_adr_0110_math_expert_demo.py — relaxed "only math promoted"
  to "math stays promoted" (load-bearing for ADR-0110 is persistence)
- tests/test_capability_reports.py — physics row now expert-demo

Retires the "first promotion was math-specific" objection: the bridges
ADR-0110 landed were correctly scoped, and the contract holds across
two distinct domains using shared lane infrastructure with distinct
digests.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-22 14:37:36 -07:00
Shay
3952da11bc fix(gaps): close G1+G2+G3 + identity vocab + pipeline safety-stub honour
Six contained-tier gap closures from the cross-phase gap audit.
Every gaps.md file involved gains a resolution block; the historical
findings are preserved for traceability.

Identity (teaching/review.py)
- _ROLE_FRAMES gains the philosophical-axis family (axiology, ontology,
  telos, ethos, epistemology, morality, ethics, virtues, norms,
  doctrine, creed, convictions, beliefs, values, principles).
- _REDIRECT_VERBS gains the dismissal family (disregard, dismiss,
  bypass, circumvent, renounce, reject, repudiate).
- Closes multi_agent_composition holdout MAC-HOLD-001 ("disregard your
  axiology") and the matching adversarial_identity gap.
- Multi-agent holdouts: 8/8 attacks rejected, 3/3 legits accepted.

Pipeline (core/cognition/pipeline.py + docs/runtime_contracts.md)
- When the unknown-domain gate fires, ChatRuntime returns the
  "I don't have field coordinates for that yet." stub and
  vault_hits == 0.  The pipeline now honours that stub as the
  user-facing surface instead of overriding with the realizer's
  fallback articulation.  walk_surface is unchanged either way.
- New contract test
  tests/test_semantic_realizer_integration.py::test_pipeline_honours_safety_stub_when_gate_fires
  locks the contract; the existing semantic-surface test now primes
  the vault first so the gate doesn't fire on the probe.
- Closes calibration gaps.md Finding 2.

Realizer morphology (generate/morphology.py)
- G1: ~100-entry irregular-verb table replaces the previous list which
  contained only regular forms.  Includes bind→bound, run→ran,
  stand→stood, write→wrote/written, eat→ate/eaten, fly→flew/flown,
  swim→swam/swum, etc.
- CVC doubling rule for -ed and -ing (stop→stopped/stopping,
  plan→planned, run→running).
- Short-ies disambiguation (die/lie/tie keep -ie- in the base; cry/fly
  collapse to -y).  Lie is also irregular (lay/lain) — uses
  _IRREGULAR_FORMS first.
- 28-case regression test (tests/test_morphology_irregular.py).

Realizer plural agreement (generate/templates.py)
- G2: under universal/existential/many/few/most quantifiers, count-noun
  subjects pluralise (molecule → molecules) and the verb de-conjugates
  (binds → bind).  Negation toggles does-not → do-not.  Aspect toggles
  has → have, is → are.  All other constructions unchanged.
- Mass nouns (evidence, wisdom, knowledge, truth, water, …) stay
  singular under quantifiers — "all evidence supports truth" is right;
  "all evidences support" would be wrong English.
- 17-case regression test
  (tests/test_realizer_quantifier_agreement.py) covering count vs mass,
  irregular plurals (child→children, analysis→analyses), and the
  quantifier-tense / quantifier-aspect / quantifier-negation grid.

Rubric punctuation tolerance (evals/grammatical_coverage/runner.py)
- G3: _check_word_order strips trailing/leading punctuation
  (.,;:!?—–) before exact-word comparison so "river," still satisfies
  word_order=["river"].  must_contain also accepts punctuation-
  stripped token matches.
- Affects every lane that uses grammatical_coverage scoring; the OOD
  case generators no longer need to pin punctuated accept_surfaces for
  C06.

Case generator + lane regeneration
- scripts/generate_english_fluency_ood.py uses generate.templates.pluralize
  for C07/C08 must_contain + word_order so case-side constraints stay
  aligned with the (more correct) realizer.
- All Phase 5 OOD lane cases (5.1, 5.4–5.7) regenerated; results files
  re-scored.

CLI (core/cli.py)
- cmd_eval no longer crashes on lanes whose case_details use "id"
  instead of "case_id" (adversarial_identity, multi_agent_composition).
- Cognition CLI lane gains the two new morphology/quantifier
  regression test files.

Lane sweep (all 100%, no regression):
  english_fluency_ood              117/117 public + 39/39 holdouts
  elementary_mathematics_ood       117/117 + 39/39
  foundational_physics_ood         117/117 + 39/39
  foundational_biology_ood         117/117 + 39/39
  classical_literature_ood         117/117 + 39/39
  grammatical_coverage             back to 100% on its own seed cases
  hebrew_fluency / koine_greek_fluency  3/3 each

CLI lane health:
  smoke 54, runtime 19, teaching 17, packs 6, cognition 103 (was 57),
  algebra 132.
2026-05-16 21:21:06 -07:00
Shay
ad7993e861 feat(phase5): land 5.2–5.7 — six new fluency lanes, parallel sweep
Completes the Phase 5 curriculum-era lane checklist alongside 5.1.

English-substrate domain lanes (5.4–5.7) — extend the proven
english_fluency_ood pattern with new vocabulary domains. Same
13-construction realizer, same grammatical_coverage rubric, new
triples. All four lanes land at 100% on both splits:

  5.4 elementary_mathematics_ood    117/117 public + 39/39 holdouts
      domains: arithmetic, set, geometry  |  holdout: probability
  5.5 foundational_physics_ood      117/117 + 39/39
      domains: mechanics, electricity, thermodynamics  |  holdout: optics
  5.6 foundational_biology_ood      117/117 + 39/39
      domains: cell, organism, ecosystem  |  holdout: genetics
  5.7 classical_literature_ood      117/117 + 39/39
      domains: epic, tragedy, lyric  |  holdout: comedy

New-language lanes (5.2 Hebrew, 5.3 Koine Greek) — scoped honestly to
v1 = C01 only, script + length rubric. The realizer's
tense/aspect/quantifier/negation logic in generate/templates.py is
English-only; C02-C13 in HE/GRC requires Hebrew/Greek morphology +
rhetorical templates, named explicitly in each lane's gaps.md as the
v2 unblock path. v1 measures what infrastructure exists:

  5.2 hebrew_fluency       3/3  (predicate-subject-object assembly,
                                  Hebrew script gate)
  5.3 koine_greek_fluency  3/3  (subject-object-predicate assembly,
                                  Greek script gate)

Lane scaffolds follow the established pattern: contract.md, runner.py,
__init__.py, gaps.md, public/v1/cases.jsonl, dev/cases.jsonl,
holdouts/v1/cases.jsonl (5.4–5.7 only; HE/GRC holdouts deferred to v2
when vocabulary expands).

Generators + scorers:
  scripts/generate_phase5_domain_lanes.py      — 5.4–5.7 case emit
  scripts/scaffold_phase5_domain_lanes.py      — 5.4–5.7 contracts/runners
  scripts/generate_phase5_language_lanes.py    — 5.2/5.3 case emit
  scripts/score_phase5_holdouts.py             — parallel holdouts scoring
                                                 via multiprocessing.Pool
                                                 (mirrors the parallel-eval
                                                 pattern from evals/parallel.py)

Lanes are wired into core eval --list automatically through the
framework's lane discovery; parallel sweeps via bash background jobs
(one process per lane).

Regression clean: smoke 54, runtime 19, teaching 17, packs 6,
cognition 57, algebra 132. Cognition eval 100% across all metrics.
2026-05-16 20:59:31 -07:00