148 lines
11 KiB
Markdown
148 lines
11 KiB
Markdown
<!-- FILE: docs/analysis/solver-operation-coverage.md -->
|
|
|
|
# Solver Operation Coverage Audit
|
|
|
|
Status: Proposed analysis draft. No serving behavior is changed. This is a
|
|
read-only structural audit to de-risk ADR-0174 Phase 5b / Phase-4-style
|
|
composition work. Verdicts below are code-reading conclusions only; they must be
|
|
verified in the Claude lane with executable solver/binding-graph cases before
|
|
any promotion claim.
|
|
|
|
## Scope
|
|
|
|
Read surfaces:
|
|
|
|
- `generate/math_problem_graph.py`
|
|
- `generate/math_solver.py`
|
|
- `generate/binding_graph/{model,adapter,admissibility,allocation,question_target,units}.py`
|
|
- ADR-0116, ADR-0117, ADR-0132, ADR-0133, ADR-0134, ADR-0135
|
|
- Skimmed ADR-0174 Phase 5b and ADR-0203/0204/0205 to confirm current
|
|
composition/proof-DAG framing.
|
|
|
|
Important correction to the relay: the current solver/graph vocabulary is not
|
|
only `{add, subtract, transfer, multiply, divide}`. `MathProblemGraph` and
|
|
`math_solver` both include the eight operation kinds:
|
|
|
|
```text
|
|
add, subtract, transfer, multiply, divide,
|
|
apply_rate, compare_additive, compare_multiplicative
|
|
```
|
|
|
|
ADR-0174 Phase 5b states the same: the solver is already waiting for these
|
|
operations; the gap is reader -> injector -> `Operation` front-end wiring plus
|
|
composition.
|
|
|
|
## Existing Operation Substrate
|
|
|
|
| Surface | Evidence | Consequence |
|
|
|---|---|---|
|
|
| Closed operation vocabulary | `generate/math_problem_graph.py` defines `VALID_OPERATION_KINDS` with eight kinds. | New arithmetic verbs are not needed for ordinary multiply/divide/rate/comparison chains. |
|
|
| Pack-bound solver dispatch | `generate/math_solver.py` maps all eight kinds to `en_arithmetic_v1` lemmas before solving. | Missing pack lemmas fail loudly; no hidden operation fallback. |
|
|
| Stateful solver semantics | `_apply` mutates `(actor, unit)` terminal state for `add`, `subtract`, `transfer`, `multiply`, `divide`; `_apply_rate` produces numerator-unit state from denominator-unit state; comparisons derive an actor state from a reference actor. | The solver is good at forward state trajectories, but weak at keeping multiple same-unit derived intermediates alive under one actor. |
|
|
| Unknown shape | `Unknown(entity, unit)` resolves either terminal state for one entity or total-across all entities with that unit. | Target questions that ask for a missing operand, number of iterations, or intermediate state are not represented by `Unknown` alone. |
|
|
| Binding-graph equation/data model | `BoundEquation(operation_kind=...)`, `BoundUnknown(question_form=...)`, semantic roles include `rate`, `duration`, `difference`, `ratio`. | The graph can name richer forms than the solver's final `Unknown`, but current adapter still comes from existing `MathProblemGraph` operation chains. |
|
|
| Unit admissibility | `check_admissibility` covers additive, multiplicative, divide, `apply_rate`, and comparison kinds. | Dimensional proofs exist for the current eight operation kinds; new node types would need explicit admissibility rules. |
|
|
| Question-target binding | `infer_question_form` recognizes `ratio`, `difference`, `rate`, `total`, and `count` from operation kinds touching the unknown. | It can label answer form, but it does not solve inverse targets or select intermediate operation indices yet. |
|
|
| Acyclicity | ADR-0203 adds `circular_dependency` refusal to the shared binding-graph constructor. | Any new equation/intermediate-node extension must remain a DAG, not a cyclic algebra system hidden inside the graph. |
|
|
|
|
## Phase-4 Target Chains
|
|
|
|
| Target chain | Verdict | Existing operations that can carry it | What is missing / risk |
|
|
|---|---|---|---|
|
|
| Multi-step rate-sum | Expressible via composition for straight-line rate applications and sums. Piecewise tariffs/conditionals need scoped selection before admission. | `apply_rate` computes `X/Y * Y -> X`; `add`/`subtract` can aggregate generated same-unit totals. Binding admissibility has an `apply_rate` rule requiring one rate dep plus one duration/count dep. | The reader must emit the base duration/count quantities, rate hypotheses, and sum operations without clobbering unrelated same-unit state. Piecewise tariffs such as "$50/day or $500/14 days" need a tariff/choice scope or explicit branch-disagreement gate; that is a binding/composition problem, not a missing arithmetic verb. |
|
|
| Ratio chain | Expressible via composition for forward ratio chains. Inverse ratio equations need a target/equation extension. | `compare_multiplicative` supports "actor = factor * reference"; `multiply`/`divide` support scalar transformations; `infer_question_form` maps touching `compare_multiplicative` to `ratio`. | Forward chains like `A`, `B = 2A`, `C = 1/2(A+B)` can be represented if the reader emits the right reference actors and order. Inverse forms such as "ducks are 10 more than 4x chickens; ducks = 150; find total birds" require solving for an unknown reference, not just applying a forward operation. That needs equation/target solving or a new binding node shape, not a new `ratio` operation kind. |
|
|
| Accumulate-against-target | Needs new binding-graph target/equation capability; not expressible as a fixed existing operation chain in the current solver. | `add`, `subtract`, `multiply`, `divide`, and `apply_rate` can express the arithmetic once the iteration count or missing operand is known. | The current solver consumes a fully specified graph in source order, then resolves terminal `Unknown(entity, unit)`. It cannot represent "after how many weeks", "how many cups to sell to reach profit", or "how long to make back cost" as an unknown operand/iteration count. This needs a first-class target-slot/equation node or bounded inverse solver with refusal/disagreement, plus proof that no cyclic dependency is introduced. |
|
|
| Percent/fraction mutation | Mostly expressible via composition for forward mutations; needs intermediate-symbol/scope support for same-unit derived amounts and event subsets. | Percent/fraction values can be scalar `multiply`/`divide` factors, with `add`/`subtract` for mutation and `compare_multiplicative` for relative quantities. Binding admissibility already covers multiply/divide dimensions. | Some cases are safe forward mutations ("eat 75% of a pan", "lose half"). Others require original and derived same-unit quantities to coexist, e.g. principal plus interest, insured vs uninsured portions, operating cost as percent of startup cost. The current solver overwrites `(actor, unit)` for multiply/divide and `apply_rate`, so these need derived intermediate symbols/event scopes or separate binding nodes. A new `percent` operation kind is not structurally necessary; a new scoped intermediate/equation representation may be. |
|
|
|
|
## Structural Verdict
|
|
|
|
Phase 4/5b is mostly **not** blocked by missing arithmetic operation kinds. The
|
|
current operation vocabulary already covers the primitive arithmetic field:
|
|
addition/subtraction/transfer, scalar multiply/divide, rate application, and
|
|
additive/multiplicative comparison.
|
|
|
|
The real scope risk is representation:
|
|
|
|
- Can the reader emit a chain of typed operations from scattered clauses while
|
|
preserving all grounded quantities?
|
|
- Can the graph retain intermediate derived quantities when they share the same
|
|
actor/unit as their source?
|
|
- Can a question bind to a missing operand, iteration count, or intermediate
|
|
state instead of only terminal `Unknown(entity, unit)`?
|
|
- Can inverse/equation targets be solved under a disagreement rule and the
|
|
ADR-0203 acyclicity invariant?
|
|
|
|
That makes the likely build a **binding-target / intermediate-symbol /
|
|
derivation-composer extension**, not a broad new solver primitive pack.
|
|
|
|
## Chain-Specific Notes
|
|
|
|
### Multi-step rate-sum
|
|
|
|
`apply_rate` is first-class: the solver's `_apply_rate` reads the actor's
|
|
denominator-unit state and writes numerator-unit state. The binding-graph
|
|
adapter synthesizes a rate symbol with composite unit `<num>_per_<denom>`, and
|
|
admissibility checks that the denominator cancels. Therefore simple rate-sum is
|
|
expressible as:
|
|
|
|
```text
|
|
duration/count fact -> apply_rate -> produced total
|
|
produced totals -> add/subtract -> final total
|
|
```
|
|
|
|
The unsafe part is not the operation. It is branch selection and scope:
|
|
overtime, tariffs, and "including today" require deciding which rate applies to
|
|
which event subset.
|
|
|
|
### Ratio chain
|
|
|
|
`compare_multiplicative` already gives a forward ratio operation. It refuses
|
|
when the reference actor has no quantity or multiple ambiguous units, which is
|
|
the right wrong=0 boundary. Binding target can label ratio-form questions.
|
|
|
|
The gap is inverse ratio. Current `Operation` is directional: it mutates the
|
|
actor from a known reference. If the reference is the unknown and the actor is
|
|
given, the solver has no inverse-equation mode.
|
|
|
|
### Accumulate-against-target
|
|
|
|
This is the clearest "needs new graph shape" target. A terminal state solver can
|
|
answer "what is the total after N weeks"; it cannot answer "what N reaches total
|
|
T" unless N is already present as a quantity and a divide operation has been
|
|
materialized by the reader. A safe implementation needs a target slot that names
|
|
the missing operand/iteration count and an admissibility/disagreement rule for
|
|
the inverse derivation.
|
|
|
|
### Percent/fraction mutation
|
|
|
|
Percent/fraction does not need a new arithmetic verb. It needs:
|
|
|
|
- scalar extraction (`75% -> 0.75`, `1/4 -> 0.25`);
|
|
- complement derivation when the text says "covers 80%" but the cost asks for
|
|
the uncovered part;
|
|
- event/subset scoping ("after the first appointment", "subsequent visits");
|
|
- intermediate symbols when original principal/cost and derived interest/cost
|
|
must both remain available.
|
|
|
|
The binding graph is the natural home for these intermediates because it already
|
|
has `BoundEquation`, dependency sets, unit proofs, question forms, and
|
|
acyclicity checks.
|
|
|
|
## Open Questions for the Claude Lane
|
|
|
|
- Does `apply_rate` overwriting `(actor, numerator_unit)` create a concrete
|
|
hazard in rate-sum cases where the actor already holds money? If yes, the
|
|
build needs derived result symbols before promotion.
|
|
- Should inverse target solving be introduced as a new `BoundUnknown` form, a
|
|
new `BoundEquation` operation kind, or a separate proof/derivation rule over
|
|
existing equations?
|
|
- Can forward ratio chains be admitted through `compare_multiplicative` without
|
|
widening the parser's reference-actor ambiguity beyond the existing refusal
|
|
discipline?
|
|
- What is the smallest executable probe set that distinguishes "new operation
|
|
kind required" from "same operation, new intermediate symbol required" for
|
|
percent/fraction mutations?
|
|
- When proof-DAG consumers and math binding graphs share
|
|
`SemanticSymbolicBindingGraph`, should proposition-specific operation kinds
|
|
remain isolated from math admissibility, or should the admissibility dispatcher
|
|
split into math/proof entrypoints before more equation kinds are added?
|