feat(evals): Author 200-case GSM8K math evaluation corpus and verification script
This commit is contained in:
Shay
parent
be467a934f
commit
2e59434969
9 changed files with 449 additions and 14 deletions
18
docs/adr/ADR-0226-gsm8k-math-eval-corpus.md
Normal file
18
docs/adr/ADR-0226-gsm8k-math-eval-corpus.md
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
# ADR 0226: GSM8K Math Evaluation Corpus Generation
|
||||
|
||||
## Status
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
The CORE cognitive engine uses a mathematically pure $Cl(4,1)$ Conformal Geometric Algebra (CGA) substrate to solve grade-school math problems without stochastic text-parsing heuristics. To accurately benchmark parser, solver, and verifier correctness, we require an evaluation corpus that exercises these components thoroughly. However, the original upstream GSM8K dataset contains stochastic natural language variations that are out-of-distribution for our strictly typed CORE grammar. Furthermore, using the standard GSM8K test set during active development risks contaminating our holdout set.
|
||||
|
||||
## Decision
|
||||
We decided to author an original, synthetic 200-case GSM8K-style math evaluation corpus divided into:
|
||||
- **Dev Set**: 50 cases authored manually to establish structural templates aligned perfectly with our CORE grammar (e.g., `transfer`, `divide`, `proportional_change.decrease_to_fraction`).
|
||||
- **Public Set**: 150 cases synthetically generated using a Python script (`scripts/generate_gsm8k_public_corpus.py`) configured to sample from the structural templates of the dev set while randomizing entities, numerical values, and units.
|
||||
|
||||
To validate correctness, we implemented an automated verification pipeline (`evals/gsm8k_math/verify_all.py`) that executes the M1 (Parse), M2 (Solve), and M3 (Verification) pipelines on all 200 cases.
|
||||
|
||||
## Consequences
|
||||
- **Positive:** Developers have an immediate, unpolluted feedback loop to confirm parser/solver accuracy without revealing the actual GSM8K holdout set. We guarantee that the generated text rigorously matches the geometric topologies supported by the engine.
|
||||
- **Negative:** The generated public set has limited linguistic diversity compared to real human-authored text, potentially hiding parser brittleness to novel phrasing. We accept this trade-off in the current architectural phase.
|
||||
|
|
@ -15,8 +15,8 @@ By using an original, curated set of 200 problems matching our own vocabulary an
|
|||
|
||||
## Splits and ID Schema
|
||||
|
||||
- **Dev Set**: 50 cases (`gma-001` ... `gma-050`)
|
||||
- **Public Set**: 150 cases (`gma-101` ... `gma-250`)
|
||||
- **Dev Set**: 50 cases (`gma-001` ... `gma-050`). Authored manually to establish CORE grammar templates.
|
||||
- **Public Set**: 150 cases (`gma-101` ... `gma-250`). Synthetically generated via `scripts/generate_gsm8k_public_corpus.py` using grammar templates (e.g., `transfer`, `divide`, `proportional_change.decrease_to_fraction`) established by the dev set.
|
||||
- **Holdout Set**: Sealed, loaded dynamically in future phases.
|
||||
|
||||
## Metrics
|
||||
|
|
|
|||
150
evals/gsm8k_math/public/cases.jsonl
Normal file
150
evals/gsm8k_math/public/cases.jsonl
Normal file
|
|
@ -0,0 +1,150 @@
|
|||
{"problem": "David has 36 books. David splits them evenly into 4 groups. How many books does David have?", "expected_answer": 9, "expected_unit": "books", "ground_truth_graph": {"entities": ["David"], "initial_state": [{"entity": "David", "quantity": {"unit": "books", "value": 36}}], "operations": [{"actor": "David", "kind": "divide", "operand": {"unit": "books", "value": 4}}], "unknown": {"entity": "David", "unit": "books"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-101"}
|
||||
{"problem": "Eve has 15 coins. David has 47 coins. Eve hands 7 coins to David. How many coins does David have?", "expected_answer": 54, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Eve", "David"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "coins", "value": 15}}, {"entity": "David", "quantity": {"unit": "coins", "value": 47}}], "operations": [{"actor": "Eve", "kind": "transfer", "operand": {"unit": "coins", "value": 7}, "target": "David"}], "unknown": {"entity": "David", "unit": "coins"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-102"}
|
||||
{"problem": "Charlie has 42 oranges. Grace has 48 oranges. Charlie hands 2 oranges to Grace. How many oranges does Grace have?", "expected_answer": 50, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Charlie", "Grace"], "initial_state": [{"entity": "Charlie", "quantity": {"unit": "oranges", "value": 42}}, {"entity": "Grace", "quantity": {"unit": "oranges", "value": 48}}], "operations": [{"actor": "Charlie", "kind": "transfer", "operand": {"unit": "oranges", "value": 2}, "target": "Grace"}], "unknown": {"entity": "Grace", "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-103"}
|
||||
{"problem": "Victor has 48 cards. Victor splits them evenly into 3 groups. How many cards does Victor have?", "expected_answer": 16, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Victor"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "cards", "value": 48}}], "operations": [{"actor": "Victor", "kind": "divide", "operand": {"unit": "cards", "value": 3}}], "unknown": {"entity": "Victor", "unit": "cards"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-104"}
|
||||
{"problem": "Ivan has 45 books. Ivan splits them evenly into 3 groups. How many books does Ivan have?", "expected_answer": 15, "expected_unit": "books", "ground_truth_graph": {"entities": ["Ivan"], "initial_state": [{"entity": "Ivan", "quantity": {"unit": "books", "value": 45}}], "operations": [{"actor": "Ivan", "kind": "divide", "operand": {"unit": "books", "value": 3}}], "unknown": {"entity": "Ivan", "unit": "books"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-105"}
|
||||
{"problem": "Eve has 34 crayons. Grace has 16 crayons. Mallory has 32 crayons. David has 32 crayons. Eve hands 17 crayons to Grace. How many crayons does Grace have?", "expected_answer": 33, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Eve", "Grace", "Mallory", "David"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "crayons", "value": 34}}, {"entity": "Grace", "quantity": {"unit": "crayons", "value": 16}}, {"entity": "Mallory", "quantity": {"unit": "crayons", "value": 32}}, {"entity": "David", "quantity": {"unit": "crayons", "value": 32}}], "operations": [{"actor": "Eve", "kind": "transfer", "operand": {"unit": "crayons", "value": 17}, "target": "Grace"}], "unknown": {"entity": "Grace", "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-106"}
|
||||
{"problem": "Ruth has 28 coins. Ruth splits them evenly into 2 groups. How many coins does Ruth have?", "expected_answer": 14, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Ruth"], "initial_state": [{"entity": "Ruth", "quantity": {"unit": "coins", "value": 28}}], "operations": [{"actor": "Ruth", "kind": "divide", "operand": {"unit": "coins", "value": 2}}], "unknown": {"entity": "Ruth", "unit": "coins"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-107"}
|
||||
{"problem": "Victor has 33 pencils. Judy has 46 pencils. Victor hands 13 pencils to Judy. How many pencils does Judy have?", "expected_answer": 59, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Victor", "Judy"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "pencils", "value": 33}}, {"entity": "Judy", "quantity": {"unit": "pencils", "value": 46}}], "operations": [{"actor": "Victor", "kind": "transfer", "operand": {"unit": "pencils", "value": 13}, "target": "Judy"}], "unknown": {"entity": "Judy", "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-108"}
|
||||
{"problem": "Heidi has 24 crayons. Judy has 16 crayons. Heidi hands 13 crayons to Judy. How many crayons do they have left?", "expected_answer": 40, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Heidi", "Judy"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "crayons", "value": 24}}, {"entity": "Judy", "quantity": {"unit": "crayons", "value": 16}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "crayons", "value": 13}, "target": "Judy"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-109"}
|
||||
{"problem": "Nori has 14 stickers. Frank has 48 stickers. Walter has 50 stickers. Trent has 20 stickers. David has 44 stickers. Nori hands 12 stickers to Frank. How many stickers does Frank have?", "expected_answer": 60, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Nori", "Frank", "Walter", "Trent", "David"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "stickers", "value": 14}}, {"entity": "Frank", "quantity": {"unit": "stickers", "value": 48}}, {"entity": "Walter", "quantity": {"unit": "stickers", "value": 50}}, {"entity": "Trent", "quantity": {"unit": "stickers", "value": 20}}, {"entity": "David", "quantity": {"unit": "stickers", "value": 44}}], "operations": [{"actor": "Nori", "kind": "transfer", "operand": {"unit": "stickers", "value": 12}, "target": "Frank"}], "unknown": {"entity": "Frank", "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-110"}
|
||||
{"problem": "Ruth has 50 stickers. Owen has 45 stickers. Ruth hands 15 stickers to Owen. How many stickers do they have left?", "expected_answer": 95, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Ruth", "Owen"], "initial_state": [{"entity": "Ruth", "quantity": {"unit": "stickers", "value": 50}}, {"entity": "Owen", "quantity": {"unit": "stickers", "value": 45}}], "operations": [{"actor": "Ruth", "kind": "transfer", "operand": {"unit": "stickers", "value": 15}, "target": "Owen"}], "unknown": {"entity": null, "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-111"}
|
||||
{"problem": "Heidi has 35 marbles. Bob has 27 marbles. Heidi hands 5 marbles to Bob. How many marbles does Bob have?", "expected_answer": 32, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Heidi", "Bob"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "marbles", "value": 35}}, {"entity": "Bob", "quantity": {"unit": "marbles", "value": 27}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "marbles", "value": 5}, "target": "Bob"}], "unknown": {"entity": "Bob", "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-112"}
|
||||
{"problem": "Mallory has 70 oranges. Mallory splits them evenly into 5 groups. How many oranges does Mallory have?", "expected_answer": 14, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Mallory"], "initial_state": [{"entity": "Mallory", "quantity": {"unit": "oranges", "value": 70}}], "operations": [{"actor": "Mallory", "kind": "divide", "operand": {"unit": "oranges", "value": 5}}], "unknown": {"entity": "Mallory", "unit": "oranges"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-113"}
|
||||
{"problem": "Ruth has 24 apples. Ruth splits them evenly into 4 groups. How many apples does Ruth have?", "expected_answer": 6, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Ruth"], "initial_state": [{"entity": "Ruth", "quantity": {"unit": "apples", "value": 24}}], "operations": [{"actor": "Ruth", "kind": "divide", "operand": {"unit": "apples", "value": 4}}], "unknown": {"entity": "Ruth", "unit": "apples"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-114"}
|
||||
{"problem": "Victor has 47 stickers. Walter has 37 stickers. Victor hands 38 stickers to Walter. How many stickers do they have left?", "expected_answer": 84, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Victor", "Walter"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "stickers", "value": 47}}, {"entity": "Walter", "quantity": {"unit": "stickers", "value": 37}}], "operations": [{"actor": "Victor", "kind": "transfer", "operand": {"unit": "stickers", "value": 38}, "target": "Walter"}], "unknown": {"entity": null, "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-115"}
|
||||
{"problem": "Eve has 13 crayons. Trent has 17 crayons. Sara has 19 crayons. Eve hands 11 crayons to Trent. How many crayons does Trent have?", "expected_answer": 28, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Eve", "Trent", "Sara"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "crayons", "value": 13}}, {"entity": "Trent", "quantity": {"unit": "crayons", "value": 17}}, {"entity": "Sara", "quantity": {"unit": "crayons", "value": 19}}], "operations": [{"actor": "Eve", "kind": "transfer", "operand": {"unit": "crayons", "value": 11}, "target": "Trent"}], "unknown": {"entity": "Trent", "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-116"}
|
||||
{"problem": "Peggy has 28 pencils. Peggy splits them evenly into 2 groups. How many pencils does Peggy have?", "expected_answer": 14, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Peggy"], "initial_state": [{"entity": "Peggy", "quantity": {"unit": "pencils", "value": 28}}], "operations": [{"actor": "Peggy", "kind": "divide", "operand": {"unit": "pencils", "value": 2}}], "unknown": {"entity": "Peggy", "unit": "pencils"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-117"}
|
||||
{"problem": "Ruth has 44 crayons. Trent has 27 crayons. Ivan has 31 crayons. Alice has 17 crayons. Mallory has 28 crayons. Ruth hands 28 crayons to Trent. How many crayons does Trent have?", "expected_answer": 55, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Ruth", "Trent", "Ivan", "Alice", "Mallory"], "initial_state": [{"entity": "Ruth", "quantity": {"unit": "crayons", "value": 44}}, {"entity": "Trent", "quantity": {"unit": "crayons", "value": 27}}, {"entity": "Ivan", "quantity": {"unit": "crayons", "value": 31}}, {"entity": "Alice", "quantity": {"unit": "crayons", "value": 17}}, {"entity": "Mallory", "quantity": {"unit": "crayons", "value": 28}}], "operations": [{"actor": "Ruth", "kind": "transfer", "operand": {"unit": "crayons", "value": 28}, "target": "Trent"}], "unknown": {"entity": "Trent", "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-118"}
|
||||
{"problem": "Ivan has 16 coins. Trent has 50 coins. Frank has 29 coins. Ivan hands 14 coins to Trent. How many coins does Trent have?", "expected_answer": 64, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Ivan", "Trent", "Frank"], "initial_state": [{"entity": "Ivan", "quantity": {"unit": "coins", "value": 16}}, {"entity": "Trent", "quantity": {"unit": "coins", "value": 50}}, {"entity": "Frank", "quantity": {"unit": "coins", "value": 29}}], "operations": [{"actor": "Ivan", "kind": "transfer", "operand": {"unit": "coins", "value": 14}, "target": "Trent"}], "unknown": {"entity": "Trent", "unit": "coins"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-119"}
|
||||
{"problem": "Nori has 43 coins. Frank has 10 coins. Nori hands 39 coins to Frank. How many coins do they have left?", "expected_answer": 53, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Nori", "Frank"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "coins", "value": 43}}, {"entity": "Frank", "quantity": {"unit": "coins", "value": 10}}], "operations": [{"actor": "Nori", "kind": "transfer", "operand": {"unit": "coins", "value": 39}, "target": "Frank"}], "unknown": {"entity": null, "unit": "coins"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-120"}
|
||||
{"problem": "David has 13 oranges. Nori has 25 oranges. Judy has 46 oranges. David hands 2 oranges to Nori. How many oranges does Nori have?", "expected_answer": 27, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["David", "Nori", "Judy"], "initial_state": [{"entity": "David", "quantity": {"unit": "oranges", "value": 13}}, {"entity": "Nori", "quantity": {"unit": "oranges", "value": 25}}, {"entity": "Judy", "quantity": {"unit": "oranges", "value": 46}}], "operations": [{"actor": "David", "kind": "transfer", "operand": {"unit": "oranges", "value": 2}, "target": "Nori"}], "unknown": {"entity": "Nori", "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-121"}
|
||||
{"problem": "Sara has 18 crayons. Sara splits them evenly into 3 groups. How many crayons does Sara have?", "expected_answer": 6, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Sara"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "crayons", "value": 18}}], "operations": [{"actor": "Sara", "kind": "divide", "operand": {"unit": "crayons", "value": 3}}], "unknown": {"entity": "Sara", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-122"}
|
||||
{"problem": "Sara has 30 coins. Sara splits them evenly into 3 groups. How many coins does Sara have?", "expected_answer": 10, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Sara"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "coins", "value": 30}}], "operations": [{"actor": "Sara", "kind": "divide", "operand": {"unit": "coins", "value": 3}}], "unknown": {"entity": "Sara", "unit": "coins"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-123"}
|
||||
{"problem": "Peggy has 33 oranges. Peggy splits them evenly into 3 groups. How many oranges does Peggy have?", "expected_answer": 11, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Peggy"], "initial_state": [{"entity": "Peggy", "quantity": {"unit": "oranges", "value": 33}}], "operations": [{"actor": "Peggy", "kind": "divide", "operand": {"unit": "oranges", "value": 3}}], "unknown": {"entity": "Peggy", "unit": "oranges"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-124"}
|
||||
{"problem": "Nori has 24 oranges. Ruth has 14 oranges. Trent has 31 oranges. Victor has 11 oranges. Bob has 47 oranges. Nori hands 18 oranges to Ruth. How many oranges does Ruth have?", "expected_answer": 32, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Nori", "Ruth", "Trent", "Victor", "Bob"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "oranges", "value": 24}}, {"entity": "Ruth", "quantity": {"unit": "oranges", "value": 14}}, {"entity": "Trent", "quantity": {"unit": "oranges", "value": 31}}, {"entity": "Victor", "quantity": {"unit": "oranges", "value": 11}}, {"entity": "Bob", "quantity": {"unit": "oranges", "value": 47}}], "operations": [{"actor": "Nori", "kind": "transfer", "operand": {"unit": "oranges", "value": 18}, "target": "Ruth"}], "unknown": {"entity": "Ruth", "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-125"}
|
||||
{"problem": "Heidi has 6 books. Heidi splits them evenly into 2 groups. How many books does Heidi have?", "expected_answer": 3, "expected_unit": "books", "ground_truth_graph": {"entities": ["Heidi"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "books", "value": 6}}], "operations": [{"actor": "Heidi", "kind": "divide", "operand": {"unit": "books", "value": 2}}], "unknown": {"entity": "Heidi", "unit": "books"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-126"}
|
||||
{"problem": "Charlie has 14 marbles. Bob has 42 marbles. Charlie hands 4 marbles to Bob. How many marbles do they have left?", "expected_answer": 56, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Charlie", "Bob"], "initial_state": [{"entity": "Charlie", "quantity": {"unit": "marbles", "value": 14}}, {"entity": "Bob", "quantity": {"unit": "marbles", "value": 42}}], "operations": [{"actor": "Charlie", "kind": "transfer", "operand": {"unit": "marbles", "value": 4}, "target": "Bob"}], "unknown": {"entity": null, "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-127"}
|
||||
{"problem": "Sara has 60 oranges. Sara splits them evenly into 3 groups. How many oranges does Sara have?", "expected_answer": 20, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Sara"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "oranges", "value": 60}}], "operations": [{"actor": "Sara", "kind": "divide", "operand": {"unit": "oranges", "value": 3}}], "unknown": {"entity": "Sara", "unit": "oranges"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-128"}
|
||||
{"problem": "Sara has 75 oranges. Sara splits them evenly into 5 groups. How many oranges does Sara have?", "expected_answer": 15, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Sara"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "oranges", "value": 75}}], "operations": [{"actor": "Sara", "kind": "divide", "operand": {"unit": "oranges", "value": 5}}], "unknown": {"entity": "Sara", "unit": "oranges"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-129"}
|
||||
{"problem": "David has 32 cards. Walter has 37 cards. David hands 14 cards to Walter. How many cards do they have left?", "expected_answer": 69, "expected_unit": "cards", "ground_truth_graph": {"entities": ["David", "Walter"], "initial_state": [{"entity": "David", "quantity": {"unit": "cards", "value": 32}}, {"entity": "Walter", "quantity": {"unit": "cards", "value": 37}}], "operations": [{"actor": "David", "kind": "transfer", "operand": {"unit": "cards", "value": 14}, "target": "Walter"}], "unknown": {"entity": null, "unit": "cards"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-130"}
|
||||
{"problem": "Bob has 28 crayons. Bob splits them evenly into 2 groups. How many crayons does Bob have?", "expected_answer": 14, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Bob"], "initial_state": [{"entity": "Bob", "quantity": {"unit": "crayons", "value": 28}}], "operations": [{"actor": "Bob", "kind": "divide", "operand": {"unit": "crayons", "value": 2}}], "unknown": {"entity": "Bob", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-131"}
|
||||
{"problem": "Mallory has 24 crayons. Mallory splits them evenly into 3 groups. How many crayons does Mallory have?", "expected_answer": 8, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Mallory"], "initial_state": [{"entity": "Mallory", "quantity": {"unit": "crayons", "value": 24}}], "operations": [{"actor": "Mallory", "kind": "divide", "operand": {"unit": "crayons", "value": 3}}], "unknown": {"entity": "Mallory", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-132"}
|
||||
{"problem": "Victor has 37 apples. Ruth has 21 apples. Victor hands 18 apples to Ruth. How many apples do they have left?", "expected_answer": 58, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Victor", "Ruth"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "apples", "value": 37}}, {"entity": "Ruth", "quantity": {"unit": "apples", "value": 21}}], "operations": [{"actor": "Victor", "kind": "transfer", "operand": {"unit": "apples", "value": 18}, "target": "Ruth"}], "unknown": {"entity": null, "unit": "apples"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-133"}
|
||||
{"problem": "Charlie has 16 coins. Ruth has 13 coins. Charlie hands 11 coins to Ruth. How many coins does Ruth have?", "expected_answer": 24, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Charlie", "Ruth"], "initial_state": [{"entity": "Charlie", "quantity": {"unit": "coins", "value": 16}}, {"entity": "Ruth", "quantity": {"unit": "coins", "value": 13}}], "operations": [{"actor": "Charlie", "kind": "transfer", "operand": {"unit": "coins", "value": 11}, "target": "Ruth"}], "unknown": {"entity": "Ruth", "unit": "coins"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-134"}
|
||||
{"problem": "Heidi has 41 cards. Frank has 40 cards. Heidi hands 14 cards to Frank. How many cards do they have left?", "expected_answer": 81, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Heidi", "Frank"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "cards", "value": 41}}, {"entity": "Frank", "quantity": {"unit": "cards", "value": 40}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "cards", "value": 14}, "target": "Frank"}], "unknown": {"entity": null, "unit": "cards"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-135"}
|
||||
{"problem": "Frank has 34 books. Owen has 26 books. Frank hands 30 books to Owen. How many books do they have left?", "expected_answer": 60, "expected_unit": "books", "ground_truth_graph": {"entities": ["Frank", "Owen"], "initial_state": [{"entity": "Frank", "quantity": {"unit": "books", "value": 34}}, {"entity": "Owen", "quantity": {"unit": "books", "value": 26}}], "operations": [{"actor": "Frank", "kind": "transfer", "operand": {"unit": "books", "value": 30}, "target": "Owen"}], "unknown": {"entity": null, "unit": "books"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-136"}
|
||||
{"problem": "Victor has 13 oranges. Sara has 47 oranges. Eve has 44 oranges. Grace has 13 oranges. Trent has 30 oranges. Victor hands 1 oranges to Sara. How many oranges does Sara have?", "expected_answer": 48, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Victor", "Sara", "Eve", "Grace", "Trent"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "oranges", "value": 13}}, {"entity": "Sara", "quantity": {"unit": "oranges", "value": 47}}, {"entity": "Eve", "quantity": {"unit": "oranges", "value": 44}}, {"entity": "Grace", "quantity": {"unit": "oranges", "value": 13}}, {"entity": "Trent", "quantity": {"unit": "oranges", "value": 30}}], "operations": [{"actor": "Victor", "kind": "transfer", "operand": {"unit": "oranges", "value": 1}, "target": "Sara"}], "unknown": {"entity": "Sara", "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-137"}
|
||||
{"problem": "Sara has 9 coins. Sara splits them evenly into 3 groups. How many coins does Sara have?", "expected_answer": 3, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Sara"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "coins", "value": 9}}], "operations": [{"actor": "Sara", "kind": "divide", "operand": {"unit": "coins", "value": 3}}], "unknown": {"entity": "Sara", "unit": "coins"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-138"}
|
||||
{"problem": "Charlie has 8 apples. Charlie splits them evenly into 2 groups. How many apples does Charlie have?", "expected_answer": 4, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Charlie"], "initial_state": [{"entity": "Charlie", "quantity": {"unit": "apples", "value": 8}}], "operations": [{"actor": "Charlie", "kind": "divide", "operand": {"unit": "apples", "value": 2}}], "unknown": {"entity": "Charlie", "unit": "apples"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-139"}
|
||||
{"problem": "Heidi has 40 cards. Heidi splits them evenly into 2 groups. How many cards does Heidi have?", "expected_answer": 20, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Heidi"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "cards", "value": 40}}], "operations": [{"actor": "Heidi", "kind": "divide", "operand": {"unit": "cards", "value": 2}}], "unknown": {"entity": "Heidi", "unit": "cards"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-140"}
|
||||
{"problem": "Walter has 15 pencils. Bob has 36 pencils. Walter hands 11 pencils to Bob. How many pencils do they have left?", "expected_answer": 51, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Walter", "Bob"], "initial_state": [{"entity": "Walter", "quantity": {"unit": "pencils", "value": 15}}, {"entity": "Bob", "quantity": {"unit": "pencils", "value": 36}}], "operations": [{"actor": "Walter", "kind": "transfer", "operand": {"unit": "pencils", "value": 11}, "target": "Bob"}], "unknown": {"entity": null, "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-141"}
|
||||
{"problem": "Mallory has 18 cards. Heidi has 29 cards. Ivan has 39 cards. Mallory hands 11 cards to Heidi. How many cards does Heidi have?", "expected_answer": 40, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Mallory", "Heidi", "Ivan"], "initial_state": [{"entity": "Mallory", "quantity": {"unit": "cards", "value": 18}}, {"entity": "Heidi", "quantity": {"unit": "cards", "value": 29}}, {"entity": "Ivan", "quantity": {"unit": "cards", "value": 39}}], "operations": [{"actor": "Mallory", "kind": "transfer", "operand": {"unit": "cards", "value": 11}, "target": "Heidi"}], "unknown": {"entity": "Heidi", "unit": "cards"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-142"}
|
||||
{"problem": "Ruth has 44 crayons. David has 23 crayons. Ruth hands 33 crayons to David. How many crayons do they have left?", "expected_answer": 67, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Ruth", "David"], "initial_state": [{"entity": "Ruth", "quantity": {"unit": "crayons", "value": 44}}, {"entity": "David", "quantity": {"unit": "crayons", "value": 23}}], "operations": [{"actor": "Ruth", "kind": "transfer", "operand": {"unit": "crayons", "value": 33}, "target": "David"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-143"}
|
||||
{"problem": "Nori has 33 oranges. Charlie has 28 oranges. Nori hands 11 oranges to Charlie. How many oranges do they have left?", "expected_answer": 61, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Nori", "Charlie"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "oranges", "value": 33}}, {"entity": "Charlie", "quantity": {"unit": "oranges", "value": 28}}], "operations": [{"actor": "Nori", "kind": "transfer", "operand": {"unit": "oranges", "value": 11}, "target": "Charlie"}], "unknown": {"entity": null, "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-144"}
|
||||
{"problem": "Judy has 38 pencils. Judy splits them evenly into 2 groups. How many pencils does Judy have?", "expected_answer": 19, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Judy"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "pencils", "value": 38}}], "operations": [{"actor": "Judy", "kind": "divide", "operand": {"unit": "pencils", "value": 2}}], "unknown": {"entity": "Judy", "unit": "pencils"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-145"}
|
||||
{"problem": "David has 45 crayons. Eve has 19 crayons. Ivan has 27 crayons. Walter has 28 crayons. Ruth has 48 crayons. David hands 14 crayons to Eve. How many crayons do they have left?", "expected_answer": 167, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["David", "Eve", "Ivan", "Walter", "Ruth"], "initial_state": [{"entity": "David", "quantity": {"unit": "crayons", "value": 45}}, {"entity": "Eve", "quantity": {"unit": "crayons", "value": 19}}, {"entity": "Ivan", "quantity": {"unit": "crayons", "value": 27}}, {"entity": "Walter", "quantity": {"unit": "crayons", "value": 28}}, {"entity": "Ruth", "quantity": {"unit": "crayons", "value": 48}}], "operations": [{"actor": "David", "kind": "transfer", "operand": {"unit": "crayons", "value": 14}, "target": "Eve"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-146"}
|
||||
{"problem": "Ivan has 26 blocks. Trent has 13 blocks. Ivan hands 3 blocks to Trent. How many blocks do they have left?", "expected_answer": 39, "expected_unit": "blocks", "ground_truth_graph": {"entities": ["Ivan", "Trent"], "initial_state": [{"entity": "Ivan", "quantity": {"unit": "blocks", "value": 26}}, {"entity": "Trent", "quantity": {"unit": "blocks", "value": 13}}], "operations": [{"actor": "Ivan", "kind": "transfer", "operand": {"unit": "blocks", "value": 3}, "target": "Trent"}], "unknown": {"entity": null, "unit": "blocks"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-147"}
|
||||
{"problem": "Alice has 20 stickers. Mallory has 38 stickers. Eve has 45 stickers. Alice hands 14 stickers to Mallory. How many stickers does Mallory have?", "expected_answer": 52, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Alice", "Mallory", "Eve"], "initial_state": [{"entity": "Alice", "quantity": {"unit": "stickers", "value": 20}}, {"entity": "Mallory", "quantity": {"unit": "stickers", "value": 38}}, {"entity": "Eve", "quantity": {"unit": "stickers", "value": 45}}], "operations": [{"actor": "Alice", "kind": "transfer", "operand": {"unit": "stickers", "value": 14}, "target": "Mallory"}], "unknown": {"entity": "Mallory", "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-148"}
|
||||
{"problem": "Charlie has 12 coins. Eve has 33 coins. Charlie hands 10 coins to Eve. How many coins does Eve have?", "expected_answer": 43, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Charlie", "Eve"], "initial_state": [{"entity": "Charlie", "quantity": {"unit": "coins", "value": 12}}, {"entity": "Eve", "quantity": {"unit": "coins", "value": 33}}], "operations": [{"actor": "Charlie", "kind": "transfer", "operand": {"unit": "coins", "value": 10}, "target": "Eve"}], "unknown": {"entity": "Eve", "unit": "coins"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-149"}
|
||||
{"problem": "Bob has 32 books. Judy has 23 books. Nori has 25 books. Bob hands 22 books to Judy. How many books does Judy have?", "expected_answer": 45, "expected_unit": "books", "ground_truth_graph": {"entities": ["Bob", "Judy", "Nori"], "initial_state": [{"entity": "Bob", "quantity": {"unit": "books", "value": 32}}, {"entity": "Judy", "quantity": {"unit": "books", "value": 23}}, {"entity": "Nori", "quantity": {"unit": "books", "value": 25}}], "operations": [{"actor": "Bob", "kind": "transfer", "operand": {"unit": "books", "value": 22}, "target": "Judy"}], "unknown": {"entity": "Judy", "unit": "books"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-150"}
|
||||
{"problem": "Peggy has 36 apples. Eve has 11 apples. Heidi has 21 apples. Frank has 31 apples. Owen has 36 apples. Peggy hands 16 apples to Eve. How many apples do they have left?", "expected_answer": 135, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Peggy", "Eve", "Heidi", "Frank", "Owen"], "initial_state": [{"entity": "Peggy", "quantity": {"unit": "apples", "value": 36}}, {"entity": "Eve", "quantity": {"unit": "apples", "value": 11}}, {"entity": "Heidi", "quantity": {"unit": "apples", "value": 21}}, {"entity": "Frank", "quantity": {"unit": "apples", "value": 31}}, {"entity": "Owen", "quantity": {"unit": "apples", "value": 36}}], "operations": [{"actor": "Peggy", "kind": "transfer", "operand": {"unit": "apples", "value": 16}, "target": "Eve"}], "unknown": {"entity": null, "unit": "apples"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-151"}
|
||||
{"problem": "David has 40 books. Owen has 24 books. David hands 13 books to Owen. How many books do they have left?", "expected_answer": 64, "expected_unit": "books", "ground_truth_graph": {"entities": ["David", "Owen"], "initial_state": [{"entity": "David", "quantity": {"unit": "books", "value": 40}}, {"entity": "Owen", "quantity": {"unit": "books", "value": 24}}], "operations": [{"actor": "David", "kind": "transfer", "operand": {"unit": "books", "value": 13}, "target": "Owen"}], "unknown": {"entity": null, "unit": "books"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-152"}
|
||||
{"problem": "Heidi has 31 cards. Walter has 27 cards. Alice has 14 cards. Grace has 27 cards. Heidi hands 12 cards to Walter. How many cards do they have left?", "expected_answer": 99, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Heidi", "Walter", "Alice", "Grace"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "cards", "value": 31}}, {"entity": "Walter", "quantity": {"unit": "cards", "value": 27}}, {"entity": "Alice", "quantity": {"unit": "cards", "value": 14}}, {"entity": "Grace", "quantity": {"unit": "cards", "value": 27}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "cards", "value": 12}, "target": "Walter"}], "unknown": {"entity": null, "unit": "cards"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-153"}
|
||||
{"problem": "Victor has 10 marbles. Victor splits them evenly into 2 groups. How many marbles does Victor have?", "expected_answer": 5, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Victor"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "marbles", "value": 10}}], "operations": [{"actor": "Victor", "kind": "divide", "operand": {"unit": "marbles", "value": 2}}], "unknown": {"entity": "Victor", "unit": "marbles"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-154"}
|
||||
{"problem": "Walter has 48 crayons. Ivan has 37 crayons. Bob has 32 crayons. Walter hands 47 crayons to Ivan. How many crayons do they have left?", "expected_answer": 117, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Walter", "Ivan", "Bob"], "initial_state": [{"entity": "Walter", "quantity": {"unit": "crayons", "value": 48}}, {"entity": "Ivan", "quantity": {"unit": "crayons", "value": 37}}, {"entity": "Bob", "quantity": {"unit": "crayons", "value": 32}}], "operations": [{"actor": "Walter", "kind": "transfer", "operand": {"unit": "crayons", "value": 47}, "target": "Ivan"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-155"}
|
||||
{"problem": "Trent has 37 books. David has 10 books. Owen has 43 books. Grace has 44 books. Eve has 22 books. Trent hands 24 books to David. How many books do they have left?", "expected_answer": 156, "expected_unit": "books", "ground_truth_graph": {"entities": ["Trent", "David", "Owen", "Grace", "Eve"], "initial_state": [{"entity": "Trent", "quantity": {"unit": "books", "value": 37}}, {"entity": "David", "quantity": {"unit": "books", "value": 10}}, {"entity": "Owen", "quantity": {"unit": "books", "value": 43}}, {"entity": "Grace", "quantity": {"unit": "books", "value": 44}}, {"entity": "Eve", "quantity": {"unit": "books", "value": 22}}], "operations": [{"actor": "Trent", "kind": "transfer", "operand": {"unit": "books", "value": 24}, "target": "David"}], "unknown": {"entity": null, "unit": "books"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-156"}
|
||||
{"problem": "Mallory has 29 crayons. Walter has 42 crayons. Mallory hands 10 crayons to Walter. How many crayons do they have left?", "expected_answer": 71, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Mallory", "Walter"], "initial_state": [{"entity": "Mallory", "quantity": {"unit": "crayons", "value": 29}}, {"entity": "Walter", "quantity": {"unit": "crayons", "value": 42}}], "operations": [{"actor": "Mallory", "kind": "transfer", "operand": {"unit": "crayons", "value": 10}, "target": "Walter"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-157"}
|
||||
{"problem": "Judy has 34 cards. Victor has 21 cards. Eve has 49 cards. Grace has 46 cards. Judy hands 20 cards to Victor. How many cards do they have left?", "expected_answer": 150, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Judy", "Victor", "Eve", "Grace"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "cards", "value": 34}}, {"entity": "Victor", "quantity": {"unit": "cards", "value": 21}}, {"entity": "Eve", "quantity": {"unit": "cards", "value": 49}}, {"entity": "Grace", "quantity": {"unit": "cards", "value": 46}}], "operations": [{"actor": "Judy", "kind": "transfer", "operand": {"unit": "cards", "value": 20}, "target": "Victor"}], "unknown": {"entity": null, "unit": "cards"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-158"}
|
||||
{"problem": "Alice has 32 stickers. Alice splits them evenly into 4 groups. How many stickers does Alice have?", "expected_answer": 8, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Alice"], "initial_state": [{"entity": "Alice", "quantity": {"unit": "stickers", "value": 32}}], "operations": [{"actor": "Alice", "kind": "divide", "operand": {"unit": "stickers", "value": 4}}], "unknown": {"entity": "Alice", "unit": "stickers"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-159"}
|
||||
{"problem": "Mallory has 42 oranges. Ruth has 40 oranges. Victor has 20 oranges. Trent has 15 oranges. Walter has 28 oranges. Mallory hands 33 oranges to Ruth. How many oranges do they have left?", "expected_answer": 145, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Mallory", "Ruth", "Victor", "Trent", "Walter"], "initial_state": [{"entity": "Mallory", "quantity": {"unit": "oranges", "value": 42}}, {"entity": "Ruth", "quantity": {"unit": "oranges", "value": 40}}, {"entity": "Victor", "quantity": {"unit": "oranges", "value": 20}}, {"entity": "Trent", "quantity": {"unit": "oranges", "value": 15}}, {"entity": "Walter", "quantity": {"unit": "oranges", "value": 28}}], "operations": [{"actor": "Mallory", "kind": "transfer", "operand": {"unit": "oranges", "value": 33}, "target": "Ruth"}], "unknown": {"entity": null, "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-160"}
|
||||
{"problem": "Heidi has 22 oranges. Judy has 19 oranges. Heidi hands 1 oranges to Judy. How many oranges does Judy have?", "expected_answer": 20, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Heidi", "Judy"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "oranges", "value": 22}}, {"entity": "Judy", "quantity": {"unit": "oranges", "value": 19}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "oranges", "value": 1}, "target": "Judy"}], "unknown": {"entity": "Judy", "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-161"}
|
||||
{"problem": "Sara has 36 blocks. Charlie has 50 blocks. Sara hands 13 blocks to Charlie. How many blocks do they have left?", "expected_answer": 86, "expected_unit": "blocks", "ground_truth_graph": {"entities": ["Sara", "Charlie"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "blocks", "value": 36}}, {"entity": "Charlie", "quantity": {"unit": "blocks", "value": 50}}], "operations": [{"actor": "Sara", "kind": "transfer", "operand": {"unit": "blocks", "value": 13}, "target": "Charlie"}], "unknown": {"entity": null, "unit": "blocks"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-162"}
|
||||
{"problem": "Heidi has 24 cards. Eve has 21 cards. Alice has 43 cards. David has 39 cards. Heidi hands 2 cards to Eve. How many cards does Eve have?", "expected_answer": 23, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Heidi", "Eve", "Alice", "David"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "cards", "value": 24}}, {"entity": "Eve", "quantity": {"unit": "cards", "value": 21}}, {"entity": "Alice", "quantity": {"unit": "cards", "value": 43}}, {"entity": "David", "quantity": {"unit": "cards", "value": 39}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "cards", "value": 2}, "target": "Eve"}], "unknown": {"entity": "Eve", "unit": "cards"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-163"}
|
||||
{"problem": "Ruth has 43 blocks. Eve has 45 blocks. Ruth hands 39 blocks to Eve. How many blocks do they have left?", "expected_answer": 88, "expected_unit": "blocks", "ground_truth_graph": {"entities": ["Ruth", "Eve"], "initial_state": [{"entity": "Ruth", "quantity": {"unit": "blocks", "value": 43}}, {"entity": "Eve", "quantity": {"unit": "blocks", "value": 45}}], "operations": [{"actor": "Ruth", "kind": "transfer", "operand": {"unit": "blocks", "value": 39}, "target": "Eve"}], "unknown": {"entity": null, "unit": "blocks"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-164"}
|
||||
{"problem": "Trent has 38 blocks. Peggy has 26 blocks. Ruth has 25 blocks. Frank has 50 blocks. Nori has 27 blocks. Trent hands 34 blocks to Peggy. How many blocks do they have left?", "expected_answer": 166, "expected_unit": "blocks", "ground_truth_graph": {"entities": ["Trent", "Peggy", "Ruth", "Frank", "Nori"], "initial_state": [{"entity": "Trent", "quantity": {"unit": "blocks", "value": 38}}, {"entity": "Peggy", "quantity": {"unit": "blocks", "value": 26}}, {"entity": "Ruth", "quantity": {"unit": "blocks", "value": 25}}, {"entity": "Frank", "quantity": {"unit": "blocks", "value": 50}}, {"entity": "Nori", "quantity": {"unit": "blocks", "value": 27}}], "operations": [{"actor": "Trent", "kind": "transfer", "operand": {"unit": "blocks", "value": 34}, "target": "Peggy"}], "unknown": {"entity": null, "unit": "blocks"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-165"}
|
||||
{"problem": "Heidi has 20 stickers. Heidi splits them evenly into 5 groups. How many stickers does Heidi have?", "expected_answer": 4, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Heidi"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "stickers", "value": 20}}], "operations": [{"actor": "Heidi", "kind": "divide", "operand": {"unit": "stickers", "value": 5}}], "unknown": {"entity": "Heidi", "unit": "stickers"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-166"}
|
||||
{"problem": "Judy has 48 oranges. Judy splits them evenly into 4 groups. How many oranges does Judy have?", "expected_answer": 12, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Judy"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "oranges", "value": 48}}], "operations": [{"actor": "Judy", "kind": "divide", "operand": {"unit": "oranges", "value": 4}}], "unknown": {"entity": "Judy", "unit": "oranges"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-167"}
|
||||
{"problem": "Charlie has 23 apples. Eve has 14 apples. Victor has 36 apples. Heidi has 36 apples. Grace has 31 apples. Charlie hands 18 apples to Eve. How many apples do they have left?", "expected_answer": 140, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Charlie", "Eve", "Victor", "Heidi", "Grace"], "initial_state": [{"entity": "Charlie", "quantity": {"unit": "apples", "value": 23}}, {"entity": "Eve", "quantity": {"unit": "apples", "value": 14}}, {"entity": "Victor", "quantity": {"unit": "apples", "value": 36}}, {"entity": "Heidi", "quantity": {"unit": "apples", "value": 36}}, {"entity": "Grace", "quantity": {"unit": "apples", "value": 31}}], "operations": [{"actor": "Charlie", "kind": "transfer", "operand": {"unit": "apples", "value": 18}, "target": "Eve"}], "unknown": {"entity": null, "unit": "apples"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-168"}
|
||||
{"problem": "Grace has 11 pencils. Peggy has 46 pencils. Owen has 34 pencils. Grace hands 8 pencils to Peggy. How many pencils does Peggy have?", "expected_answer": 54, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Grace", "Peggy", "Owen"], "initial_state": [{"entity": "Grace", "quantity": {"unit": "pencils", "value": 11}}, {"entity": "Peggy", "quantity": {"unit": "pencils", "value": 46}}, {"entity": "Owen", "quantity": {"unit": "pencils", "value": 34}}], "operations": [{"actor": "Grace", "kind": "transfer", "operand": {"unit": "pencils", "value": 8}, "target": "Peggy"}], "unknown": {"entity": "Peggy", "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-169"}
|
||||
{"problem": "Owen has 27 oranges. Peggy has 37 oranges. Heidi has 41 oranges. Sara has 11 oranges. Owen hands 13 oranges to Peggy. How many oranges do they have left?", "expected_answer": 116, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Owen", "Peggy", "Heidi", "Sara"], "initial_state": [{"entity": "Owen", "quantity": {"unit": "oranges", "value": 27}}, {"entity": "Peggy", "quantity": {"unit": "oranges", "value": 37}}, {"entity": "Heidi", "quantity": {"unit": "oranges", "value": 41}}, {"entity": "Sara", "quantity": {"unit": "oranges", "value": 11}}], "operations": [{"actor": "Owen", "kind": "transfer", "operand": {"unit": "oranges", "value": 13}, "target": "Peggy"}], "unknown": {"entity": null, "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-170"}
|
||||
{"problem": "Owen has 30 apples. Owen splits them evenly into 5 groups. How many apples does Owen have?", "expected_answer": 6, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Owen"], "initial_state": [{"entity": "Owen", "quantity": {"unit": "apples", "value": 30}}], "operations": [{"actor": "Owen", "kind": "divide", "operand": {"unit": "apples", "value": 5}}], "unknown": {"entity": "Owen", "unit": "apples"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-171"}
|
||||
{"problem": "Victor has 100 books. Victor splits them evenly into 5 groups. How many books does Victor have?", "expected_answer": 20, "expected_unit": "books", "ground_truth_graph": {"entities": ["Victor"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "books", "value": 100}}], "operations": [{"actor": "Victor", "kind": "divide", "operand": {"unit": "books", "value": 5}}], "unknown": {"entity": "Victor", "unit": "books"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-172"}
|
||||
{"problem": "Alice has 30 crayons. Alice splits them evenly into 5 groups. How many crayons does Alice have?", "expected_answer": 6, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Alice"], "initial_state": [{"entity": "Alice", "quantity": {"unit": "crayons", "value": 30}}], "operations": [{"actor": "Alice", "kind": "divide", "operand": {"unit": "crayons", "value": 5}}], "unknown": {"entity": "Alice", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-173"}
|
||||
{"problem": "Bob has 23 marbles. Ivan has 39 marbles. Owen has 30 marbles. Bob hands 11 marbles to Ivan. How many marbles do they have left?", "expected_answer": 92, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Bob", "Ivan", "Owen"], "initial_state": [{"entity": "Bob", "quantity": {"unit": "marbles", "value": 23}}, {"entity": "Ivan", "quantity": {"unit": "marbles", "value": 39}}, {"entity": "Owen", "quantity": {"unit": "marbles", "value": 30}}], "operations": [{"actor": "Bob", "kind": "transfer", "operand": {"unit": "marbles", "value": 11}, "target": "Ivan"}], "unknown": {"entity": null, "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-174"}
|
||||
{"problem": "Ivan has 13 coins. Charlie has 32 coins. Sara has 24 coins. Alice has 14 coins. Ivan hands 11 coins to Charlie. How many coins does Charlie have?", "expected_answer": 43, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Ivan", "Charlie", "Sara", "Alice"], "initial_state": [{"entity": "Ivan", "quantity": {"unit": "coins", "value": 13}}, {"entity": "Charlie", "quantity": {"unit": "coins", "value": 32}}, {"entity": "Sara", "quantity": {"unit": "coins", "value": 24}}, {"entity": "Alice", "quantity": {"unit": "coins", "value": 14}}], "operations": [{"actor": "Ivan", "kind": "transfer", "operand": {"unit": "coins", "value": 11}, "target": "Charlie"}], "unknown": {"entity": "Charlie", "unit": "coins"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-175"}
|
||||
{"problem": "Heidi has 49 books. Grace has 19 books. Heidi hands 16 books to Grace. How many books does Grace have?", "expected_answer": 35, "expected_unit": "books", "ground_truth_graph": {"entities": ["Heidi", "Grace"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "books", "value": 49}}, {"entity": "Grace", "quantity": {"unit": "books", "value": 19}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "books", "value": 16}, "target": "Grace"}], "unknown": {"entity": "Grace", "unit": "books"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-176"}
|
||||
{"problem": "David has 20 marbles. Grace has 48 marbles. Ruth has 48 marbles. Ivan has 17 marbles. Owen has 20 marbles. David hands 10 marbles to Grace. How many marbles does Grace have?", "expected_answer": 58, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["David", "Grace", "Ruth", "Ivan", "Owen"], "initial_state": [{"entity": "David", "quantity": {"unit": "marbles", "value": 20}}, {"entity": "Grace", "quantity": {"unit": "marbles", "value": 48}}, {"entity": "Ruth", "quantity": {"unit": "marbles", "value": 48}}, {"entity": "Ivan", "quantity": {"unit": "marbles", "value": 17}}, {"entity": "Owen", "quantity": {"unit": "marbles", "value": 20}}], "operations": [{"actor": "David", "kind": "transfer", "operand": {"unit": "marbles", "value": 10}, "target": "Grace"}], "unknown": {"entity": "Grace", "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-177"}
|
||||
{"problem": "Alice has 70 stickers. Alice splits them evenly into 5 groups. How many stickers does Alice have?", "expected_answer": 14, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Alice"], "initial_state": [{"entity": "Alice", "quantity": {"unit": "stickers", "value": 70}}], "operations": [{"actor": "Alice", "kind": "divide", "operand": {"unit": "stickers", "value": 5}}], "unknown": {"entity": "Alice", "unit": "stickers"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-178"}
|
||||
{"problem": "Grace has 15 crayons. Grace splits them evenly into 3 groups. How many crayons does Grace have?", "expected_answer": 5, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Grace"], "initial_state": [{"entity": "Grace", "quantity": {"unit": "crayons", "value": 15}}], "operations": [{"actor": "Grace", "kind": "divide", "operand": {"unit": "crayons", "value": 3}}], "unknown": {"entity": "Grace", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-179"}
|
||||
{"problem": "Judy has 40 pencils. Judy splits them evenly into 2 groups. How many pencils does Judy have?", "expected_answer": 20, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Judy"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "pencils", "value": 40}}], "operations": [{"actor": "Judy", "kind": "divide", "operand": {"unit": "pencils", "value": 2}}], "unknown": {"entity": "Judy", "unit": "pencils"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-180"}
|
||||
{"problem": "Nori has 33 cards. Victor has 14 cards. Nori hands 22 cards to Victor. How many cards does Victor have?", "expected_answer": 36, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Nori", "Victor"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "cards", "value": 33}}, {"entity": "Victor", "quantity": {"unit": "cards", "value": 14}}], "operations": [{"actor": "Nori", "kind": "transfer", "operand": {"unit": "cards", "value": 22}, "target": "Victor"}], "unknown": {"entity": "Victor", "unit": "cards"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-181"}
|
||||
{"problem": "David has 37 apples. Peggy has 21 apples. Nori has 43 apples. Ruth has 27 apples. David hands 35 apples to Peggy. How many apples do they have left?", "expected_answer": 128, "expected_unit": "apples", "ground_truth_graph": {"entities": ["David", "Peggy", "Nori", "Ruth"], "initial_state": [{"entity": "David", "quantity": {"unit": "apples", "value": 37}}, {"entity": "Peggy", "quantity": {"unit": "apples", "value": 21}}, {"entity": "Nori", "quantity": {"unit": "apples", "value": 43}}, {"entity": "Ruth", "quantity": {"unit": "apples", "value": 27}}], "operations": [{"actor": "David", "kind": "transfer", "operand": {"unit": "apples", "value": 35}, "target": "Peggy"}], "unknown": {"entity": null, "unit": "apples"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-182"}
|
||||
{"problem": "Walter has 27 crayons. Ivan has 38 crayons. Mallory has 25 crayons. Heidi has 39 crayons. Walter hands 19 crayons to Ivan. How many crayons do they have left?", "expected_answer": 129, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Walter", "Ivan", "Mallory", "Heidi"], "initial_state": [{"entity": "Walter", "quantity": {"unit": "crayons", "value": 27}}, {"entity": "Ivan", "quantity": {"unit": "crayons", "value": 38}}, {"entity": "Mallory", "quantity": {"unit": "crayons", "value": 25}}, {"entity": "Heidi", "quantity": {"unit": "crayons", "value": 39}}], "operations": [{"actor": "Walter", "kind": "transfer", "operand": {"unit": "crayons", "value": 19}, "target": "Ivan"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-183"}
|
||||
{"problem": "Sara has 23 blocks. Mallory has 32 blocks. Frank has 26 blocks. Sara hands 11 blocks to Mallory. How many blocks do they have left?", "expected_answer": 81, "expected_unit": "blocks", "ground_truth_graph": {"entities": ["Sara", "Mallory", "Frank"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "blocks", "value": 23}}, {"entity": "Mallory", "quantity": {"unit": "blocks", "value": 32}}, {"entity": "Frank", "quantity": {"unit": "blocks", "value": 26}}], "operations": [{"actor": "Sara", "kind": "transfer", "operand": {"unit": "blocks", "value": 11}, "target": "Mallory"}], "unknown": {"entity": null, "unit": "blocks"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-184"}
|
||||
{"problem": "Ivan has 36 coins. Ivan splits them evenly into 2 groups. How many coins does Ivan have?", "expected_answer": 18, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Ivan"], "initial_state": [{"entity": "Ivan", "quantity": {"unit": "coins", "value": 36}}], "operations": [{"actor": "Ivan", "kind": "divide", "operand": {"unit": "coins", "value": 2}}], "unknown": {"entity": "Ivan", "unit": "coins"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-185"}
|
||||
{"problem": "Charlie has 41 cards. Heidi has 45 cards. Charlie hands 16 cards to Heidi. How many cards do they have left?", "expected_answer": 86, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Charlie", "Heidi"], "initial_state": [{"entity": "Charlie", "quantity": {"unit": "cards", "value": 41}}, {"entity": "Heidi", "quantity": {"unit": "cards", "value": 45}}], "operations": [{"actor": "Charlie", "kind": "transfer", "operand": {"unit": "cards", "value": 16}, "target": "Heidi"}], "unknown": {"entity": null, "unit": "cards"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-186"}
|
||||
{"problem": "Sara has 8 blocks. Sara splits them evenly into 2 groups. How many blocks does Sara have?", "expected_answer": 4, "expected_unit": "blocks", "ground_truth_graph": {"entities": ["Sara"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "blocks", "value": 8}}], "operations": [{"actor": "Sara", "kind": "divide", "operand": {"unit": "blocks", "value": 2}}], "unknown": {"entity": "Sara", "unit": "blocks"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-187"}
|
||||
{"problem": "Owen has 33 pencils. Heidi has 40 pencils. Judy has 45 pencils. Owen hands 23 pencils to Heidi. How many pencils do they have left?", "expected_answer": 118, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Owen", "Heidi", "Judy"], "initial_state": [{"entity": "Owen", "quantity": {"unit": "pencils", "value": 33}}, {"entity": "Heidi", "quantity": {"unit": "pencils", "value": 40}}, {"entity": "Judy", "quantity": {"unit": "pencils", "value": 45}}], "operations": [{"actor": "Owen", "kind": "transfer", "operand": {"unit": "pencils", "value": 23}, "target": "Heidi"}], "unknown": {"entity": null, "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-188"}
|
||||
{"problem": "Victor has 64 marbles. Victor splits them evenly into 4 groups. How many marbles does Victor have?", "expected_answer": 16, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Victor"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "marbles", "value": 64}}], "operations": [{"actor": "Victor", "kind": "divide", "operand": {"unit": "marbles", "value": 4}}], "unknown": {"entity": "Victor", "unit": "marbles"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-189"}
|
||||
{"problem": "Ivan has 17 marbles. Heidi has 44 marbles. David has 21 marbles. Grace has 22 marbles. Ivan hands 7 marbles to Heidi. How many marbles do they have left?", "expected_answer": 104, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Ivan", "Heidi", "David", "Grace"], "initial_state": [{"entity": "Ivan", "quantity": {"unit": "marbles", "value": 17}}, {"entity": "Heidi", "quantity": {"unit": "marbles", "value": 44}}, {"entity": "David", "quantity": {"unit": "marbles", "value": 21}}, {"entity": "Grace", "quantity": {"unit": "marbles", "value": 22}}], "operations": [{"actor": "Ivan", "kind": "transfer", "operand": {"unit": "marbles", "value": 7}, "target": "Heidi"}], "unknown": {"entity": null, "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-190"}
|
||||
{"problem": "Walter has 28 oranges. Trent has 24 oranges. Judy has 33 oranges. David has 21 oranges. Peggy has 29 oranges. Walter hands 1 oranges to Trent. How many oranges does Trent have?", "expected_answer": 25, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Walter", "Trent", "Judy", "David", "Peggy"], "initial_state": [{"entity": "Walter", "quantity": {"unit": "oranges", "value": 28}}, {"entity": "Trent", "quantity": {"unit": "oranges", "value": 24}}, {"entity": "Judy", "quantity": {"unit": "oranges", "value": 33}}, {"entity": "David", "quantity": {"unit": "oranges", "value": 21}}, {"entity": "Peggy", "quantity": {"unit": "oranges", "value": 29}}], "operations": [{"actor": "Walter", "kind": "transfer", "operand": {"unit": "oranges", "value": 1}, "target": "Trent"}], "unknown": {"entity": "Trent", "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-191"}
|
||||
{"problem": "Bob has 50 apples. Victor has 41 apples. Judy has 16 apples. Bob hands 1 apples to Victor. How many apples do they have left?", "expected_answer": 107, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Bob", "Victor", "Judy"], "initial_state": [{"entity": "Bob", "quantity": {"unit": "apples", "value": 50}}, {"entity": "Victor", "quantity": {"unit": "apples", "value": 41}}, {"entity": "Judy", "quantity": {"unit": "apples", "value": 16}}], "operations": [{"actor": "Bob", "kind": "transfer", "operand": {"unit": "apples", "value": 1}, "target": "Victor"}], "unknown": {"entity": null, "unit": "apples"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-192"}
|
||||
{"problem": "Ruth has 40 stickers. Mallory has 17 stickers. Frank has 14 stickers. Bob has 35 stickers. Ruth hands 32 stickers to Mallory. How many stickers does Mallory have?", "expected_answer": 49, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Ruth", "Mallory", "Frank", "Bob"], "initial_state": [{"entity": "Ruth", "quantity": {"unit": "stickers", "value": 40}}, {"entity": "Mallory", "quantity": {"unit": "stickers", "value": 17}}, {"entity": "Frank", "quantity": {"unit": "stickers", "value": 14}}, {"entity": "Bob", "quantity": {"unit": "stickers", "value": 35}}], "operations": [{"actor": "Ruth", "kind": "transfer", "operand": {"unit": "stickers", "value": 32}, "target": "Mallory"}], "unknown": {"entity": "Mallory", "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-193"}
|
||||
{"problem": "Bob has 60 apples. Bob splits them evenly into 3 groups. How many apples does Bob have?", "expected_answer": 20, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Bob"], "initial_state": [{"entity": "Bob", "quantity": {"unit": "apples", "value": 60}}], "operations": [{"actor": "Bob", "kind": "divide", "operand": {"unit": "apples", "value": 3}}], "unknown": {"entity": "Bob", "unit": "apples"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-194"}
|
||||
{"problem": "Heidi has 48 pencils. David has 49 pencils. Peggy has 24 pencils. Heidi hands 34 pencils to David. How many pencils do they have left?", "expected_answer": 121, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Heidi", "David", "Peggy"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "pencils", "value": 48}}, {"entity": "David", "quantity": {"unit": "pencils", "value": 49}}, {"entity": "Peggy", "quantity": {"unit": "pencils", "value": 24}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "pencils", "value": 34}, "target": "David"}], "unknown": {"entity": null, "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-195"}
|
||||
{"problem": "Judy has 16 pencils. Peggy has 23 pencils. Walter has 50 pencils. Bob has 23 pencils. Judy hands 5 pencils to Peggy. How many pencils does Peggy have?", "expected_answer": 28, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Judy", "Peggy", "Walter", "Bob"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "pencils", "value": 16}}, {"entity": "Peggy", "quantity": {"unit": "pencils", "value": 23}}, {"entity": "Walter", "quantity": {"unit": "pencils", "value": 50}}, {"entity": "Bob", "quantity": {"unit": "pencils", "value": 23}}], "operations": [{"actor": "Judy", "kind": "transfer", "operand": {"unit": "pencils", "value": 5}, "target": "Peggy"}], "unknown": {"entity": "Peggy", "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-196"}
|
||||
{"problem": "Heidi has 14 coins. Frank has 20 coins. Heidi hands 1 coins to Frank. How many coins do they have left?", "expected_answer": 34, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Heidi", "Frank"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "coins", "value": 14}}, {"entity": "Frank", "quantity": {"unit": "coins", "value": 20}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "coins", "value": 1}, "target": "Frank"}], "unknown": {"entity": null, "unit": "coins"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-197"}
|
||||
{"problem": "Sara has 39 stickers. Judy has 14 stickers. Bob has 24 stickers. Heidi has 26 stickers. Eve has 50 stickers. Sara hands 38 stickers to Judy. How many stickers does Judy have?", "expected_answer": 52, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Sara", "Judy", "Bob", "Heidi", "Eve"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "stickers", "value": 39}}, {"entity": "Judy", "quantity": {"unit": "stickers", "value": 14}}, {"entity": "Bob", "quantity": {"unit": "stickers", "value": 24}}, {"entity": "Heidi", "quantity": {"unit": "stickers", "value": 26}}, {"entity": "Eve", "quantity": {"unit": "stickers", "value": 50}}], "operations": [{"actor": "Sara", "kind": "transfer", "operand": {"unit": "stickers", "value": 38}, "target": "Judy"}], "unknown": {"entity": "Judy", "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-198"}
|
||||
{"problem": "Victor has 19 stickers. Heidi has 14 stickers. Eve has 13 stickers. Victor hands 6 stickers to Heidi. How many stickers do they have left?", "expected_answer": 46, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Victor", "Heidi", "Eve"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "stickers", "value": 19}}, {"entity": "Heidi", "quantity": {"unit": "stickers", "value": 14}}, {"entity": "Eve", "quantity": {"unit": "stickers", "value": 13}}], "operations": [{"actor": "Victor", "kind": "transfer", "operand": {"unit": "stickers", "value": 6}, "target": "Heidi"}], "unknown": {"entity": null, "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-199"}
|
||||
{"problem": "Walter has 25 stickers. Walter splits them evenly into 5 groups. How many stickers does Walter have?", "expected_answer": 5, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Walter"], "initial_state": [{"entity": "Walter", "quantity": {"unit": "stickers", "value": 25}}], "operations": [{"actor": "Walter", "kind": "divide", "operand": {"unit": "stickers", "value": 5}}], "unknown": {"entity": "Walter", "unit": "stickers"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-200"}
|
||||
{"problem": "Judy has 48 crayons. Owen has 12 crayons. Ivan has 37 crayons. Sara has 30 crayons. Heidi has 48 crayons. Judy hands 17 crayons to Owen. How many crayons does Owen have?", "expected_answer": 29, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Judy", "Owen", "Ivan", "Sara", "Heidi"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "crayons", "value": 48}}, {"entity": "Owen", "quantity": {"unit": "crayons", "value": 12}}, {"entity": "Ivan", "quantity": {"unit": "crayons", "value": 37}}, {"entity": "Sara", "quantity": {"unit": "crayons", "value": 30}}, {"entity": "Heidi", "quantity": {"unit": "crayons", "value": 48}}], "operations": [{"actor": "Judy", "kind": "transfer", "operand": {"unit": "crayons", "value": 17}, "target": "Owen"}], "unknown": {"entity": "Owen", "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-201"}
|
||||
{"problem": "Heidi has 46 stickers. Alice has 12 stickers. Heidi hands 12 stickers to Alice. How many stickers do they have left?", "expected_answer": 58, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Heidi", "Alice"], "initial_state": [{"entity": "Heidi", "quantity": {"unit": "stickers", "value": 46}}, {"entity": "Alice", "quantity": {"unit": "stickers", "value": 12}}], "operations": [{"actor": "Heidi", "kind": "transfer", "operand": {"unit": "stickers", "value": 12}, "target": "Alice"}], "unknown": {"entity": null, "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-202"}
|
||||
{"problem": "Ruth has 60 stickers. Ruth splits them evenly into 3 groups. How many stickers does Ruth have?", "expected_answer": 20, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Ruth"], "initial_state": [{"entity": "Ruth", "quantity": {"unit": "stickers", "value": 60}}], "operations": [{"actor": "Ruth", "kind": "divide", "operand": {"unit": "stickers", "value": 3}}], "unknown": {"entity": "Ruth", "unit": "stickers"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-203"}
|
||||
{"problem": "Sara has 30 marbles. Charlie has 16 marbles. Walter has 20 marbles. Nori has 31 marbles. Grace has 36 marbles. Sara hands 23 marbles to Charlie. How many marbles do they have left?", "expected_answer": 133, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Sara", "Charlie", "Walter", "Nori", "Grace"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "marbles", "value": 30}}, {"entity": "Charlie", "quantity": {"unit": "marbles", "value": 16}}, {"entity": "Walter", "quantity": {"unit": "marbles", "value": 20}}, {"entity": "Nori", "quantity": {"unit": "marbles", "value": 31}}, {"entity": "Grace", "quantity": {"unit": "marbles", "value": 36}}], "operations": [{"actor": "Sara", "kind": "transfer", "operand": {"unit": "marbles", "value": 23}, "target": "Charlie"}], "unknown": {"entity": null, "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-204"}
|
||||
{"problem": "Owen has 26 marbles. Victor has 30 marbles. Bob has 17 marbles. Ruth has 35 marbles. Trent has 42 marbles. Owen hands 1 marbles to Victor. How many marbles do they have left?", "expected_answer": 150, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Owen", "Victor", "Bob", "Ruth", "Trent"], "initial_state": [{"entity": "Owen", "quantity": {"unit": "marbles", "value": 26}}, {"entity": "Victor", "quantity": {"unit": "marbles", "value": 30}}, {"entity": "Bob", "quantity": {"unit": "marbles", "value": 17}}, {"entity": "Ruth", "quantity": {"unit": "marbles", "value": 35}}, {"entity": "Trent", "quantity": {"unit": "marbles", "value": 42}}], "operations": [{"actor": "Owen", "kind": "transfer", "operand": {"unit": "marbles", "value": 1}, "target": "Victor"}], "unknown": {"entity": null, "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-205"}
|
||||
{"problem": "Grace has 41 pencils. Trent has 50 pencils. Nori has 38 pencils. Grace hands 4 pencils to Trent. How many pencils does Trent have?", "expected_answer": 54, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Grace", "Trent", "Nori"], "initial_state": [{"entity": "Grace", "quantity": {"unit": "pencils", "value": 41}}, {"entity": "Trent", "quantity": {"unit": "pencils", "value": 50}}, {"entity": "Nori", "quantity": {"unit": "pencils", "value": 38}}], "operations": [{"actor": "Grace", "kind": "transfer", "operand": {"unit": "pencils", "value": 4}, "target": "Trent"}], "unknown": {"entity": "Trent", "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-206"}
|
||||
{"problem": "Eve has 50 books. Judy has 48 books. Ruth has 25 books. Sara has 20 books. Bob has 29 books. Eve hands 36 books to Judy. How many books does Judy have?", "expected_answer": 84, "expected_unit": "books", "ground_truth_graph": {"entities": ["Eve", "Judy", "Ruth", "Sara", "Bob"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "books", "value": 50}}, {"entity": "Judy", "quantity": {"unit": "books", "value": 48}}, {"entity": "Ruth", "quantity": {"unit": "books", "value": 25}}, {"entity": "Sara", "quantity": {"unit": "books", "value": 20}}, {"entity": "Bob", "quantity": {"unit": "books", "value": 29}}], "operations": [{"actor": "Eve", "kind": "transfer", "operand": {"unit": "books", "value": 36}, "target": "Judy"}], "unknown": {"entity": "Judy", "unit": "books"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-207"}
|
||||
{"problem": "Peggy has 15 crayons. Peggy splits them evenly into 3 groups. How many crayons does Peggy have?", "expected_answer": 5, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Peggy"], "initial_state": [{"entity": "Peggy", "quantity": {"unit": "crayons", "value": 15}}], "operations": [{"actor": "Peggy", "kind": "divide", "operand": {"unit": "crayons", "value": 3}}], "unknown": {"entity": "Peggy", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-208"}
|
||||
{"problem": "Eve has 27 coins. Sara has 36 coins. Judy has 40 coins. Eve hands 16 coins to Sara. How many coins does Sara have?", "expected_answer": 52, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Eve", "Sara", "Judy"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "coins", "value": 27}}, {"entity": "Sara", "quantity": {"unit": "coins", "value": 36}}, {"entity": "Judy", "quantity": {"unit": "coins", "value": 40}}], "operations": [{"actor": "Eve", "kind": "transfer", "operand": {"unit": "coins", "value": 16}, "target": "Sara"}], "unknown": {"entity": "Sara", "unit": "coins"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-209"}
|
||||
{"problem": "Eve has 27 crayons. Owen has 36 crayons. Grace has 31 crayons. Walter has 42 crayons. Peggy has 27 crayons. Eve hands 1 crayons to Owen. How many crayons do they have left?", "expected_answer": 163, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Eve", "Owen", "Grace", "Walter", "Peggy"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "crayons", "value": 27}}, {"entity": "Owen", "quantity": {"unit": "crayons", "value": 36}}, {"entity": "Grace", "quantity": {"unit": "crayons", "value": 31}}, {"entity": "Walter", "quantity": {"unit": "crayons", "value": 42}}, {"entity": "Peggy", "quantity": {"unit": "crayons", "value": 27}}], "operations": [{"actor": "Eve", "kind": "transfer", "operand": {"unit": "crayons", "value": 1}, "target": "Owen"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-210"}
|
||||
{"problem": "Judy has 30 pencils. Judy splits them evenly into 5 groups. How many pencils does Judy have?", "expected_answer": 6, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Judy"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "pencils", "value": 30}}], "operations": [{"actor": "Judy", "kind": "divide", "operand": {"unit": "pencils", "value": 5}}], "unknown": {"entity": "Judy", "unit": "pencils"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-211"}
|
||||
{"problem": "Sara has 22 marbles. Nori has 25 marbles. Mallory has 46 marbles. Owen has 34 marbles. Heidi has 24 marbles. Sara hands 14 marbles to Nori. How many marbles does Nori have?", "expected_answer": 39, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Sara", "Nori", "Mallory", "Owen", "Heidi"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "marbles", "value": 22}}, {"entity": "Nori", "quantity": {"unit": "marbles", "value": 25}}, {"entity": "Mallory", "quantity": {"unit": "marbles", "value": 46}}, {"entity": "Owen", "quantity": {"unit": "marbles", "value": 34}}, {"entity": "Heidi", "quantity": {"unit": "marbles", "value": 24}}], "operations": [{"actor": "Sara", "kind": "transfer", "operand": {"unit": "marbles", "value": 14}, "target": "Nori"}], "unknown": {"entity": "Nori", "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-212"}
|
||||
{"problem": "Sara has 18 books. Owen has 42 books. Victor has 47 books. Eve has 31 books. Heidi has 16 books. Sara hands 15 books to Owen. How many books does Owen have?", "expected_answer": 57, "expected_unit": "books", "ground_truth_graph": {"entities": ["Sara", "Owen", "Victor", "Eve", "Heidi"], "initial_state": [{"entity": "Sara", "quantity": {"unit": "books", "value": 18}}, {"entity": "Owen", "quantity": {"unit": "books", "value": 42}}, {"entity": "Victor", "quantity": {"unit": "books", "value": 47}}, {"entity": "Eve", "quantity": {"unit": "books", "value": 31}}, {"entity": "Heidi", "quantity": {"unit": "books", "value": 16}}], "operations": [{"actor": "Sara", "kind": "transfer", "operand": {"unit": "books", "value": 15}, "target": "Owen"}], "unknown": {"entity": "Owen", "unit": "books"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-213"}
|
||||
{"problem": "Ruth has 45 books. Ruth splits them evenly into 3 groups. How many books does Ruth have?", "expected_answer": 15, "expected_unit": "books", "ground_truth_graph": {"entities": ["Ruth"], "initial_state": [{"entity": "Ruth", "quantity": {"unit": "books", "value": 45}}], "operations": [{"actor": "Ruth", "kind": "divide", "operand": {"unit": "books", "value": 3}}], "unknown": {"entity": "Ruth", "unit": "books"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-214"}
|
||||
{"problem": "Eve has 50 crayons. Eve splits them evenly into 5 groups. How many crayons does Eve have?", "expected_answer": 10, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Eve"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "crayons", "value": 50}}], "operations": [{"actor": "Eve", "kind": "divide", "operand": {"unit": "crayons", "value": 5}}], "unknown": {"entity": "Eve", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-215"}
|
||||
{"problem": "Owen has 44 blocks. Charlie has 12 blocks. Mallory has 49 blocks. Walter has 14 blocks. Frank has 25 blocks. Owen hands 41 blocks to Charlie. How many blocks do they have left?", "expected_answer": 144, "expected_unit": "blocks", "ground_truth_graph": {"entities": ["Owen", "Charlie", "Mallory", "Walter", "Frank"], "initial_state": [{"entity": "Owen", "quantity": {"unit": "blocks", "value": 44}}, {"entity": "Charlie", "quantity": {"unit": "blocks", "value": 12}}, {"entity": "Mallory", "quantity": {"unit": "blocks", "value": 49}}, {"entity": "Walter", "quantity": {"unit": "blocks", "value": 14}}, {"entity": "Frank", "quantity": {"unit": "blocks", "value": 25}}], "operations": [{"actor": "Owen", "kind": "transfer", "operand": {"unit": "blocks", "value": 41}, "target": "Charlie"}], "unknown": {"entity": null, "unit": "blocks"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-216"}
|
||||
{"problem": "Charlie has 50 crayons. Peggy has 16 crayons. Charlie hands 29 crayons to Peggy. How many crayons does Peggy have?", "expected_answer": 45, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Charlie", "Peggy"], "initial_state": [{"entity": "Charlie", "quantity": {"unit": "crayons", "value": 50}}, {"entity": "Peggy", "quantity": {"unit": "crayons", "value": 16}}], "operations": [{"actor": "Charlie", "kind": "transfer", "operand": {"unit": "crayons", "value": 29}, "target": "Peggy"}], "unknown": {"entity": "Peggy", "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-217"}
|
||||
{"problem": "Judy has 24 books. Judy splits them evenly into 2 groups. How many books does Judy have?", "expected_answer": 12, "expected_unit": "books", "ground_truth_graph": {"entities": ["Judy"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "books", "value": 24}}], "operations": [{"actor": "Judy", "kind": "divide", "operand": {"unit": "books", "value": 2}}], "unknown": {"entity": "Judy", "unit": "books"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-218"}
|
||||
{"problem": "Judy has 37 marbles. Nori has 19 marbles. Judy hands 16 marbles to Nori. How many marbles do they have left?", "expected_answer": 56, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Judy", "Nori"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "marbles", "value": 37}}, {"entity": "Nori", "quantity": {"unit": "marbles", "value": 19}}], "operations": [{"actor": "Judy", "kind": "transfer", "operand": {"unit": "marbles", "value": 16}, "target": "Nori"}], "unknown": {"entity": null, "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-219"}
|
||||
{"problem": "Frank has 12 apples. Frank splits them evenly into 3 groups. How many apples does Frank have?", "expected_answer": 4, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Frank"], "initial_state": [{"entity": "Frank", "quantity": {"unit": "apples", "value": 12}}], "operations": [{"actor": "Frank", "kind": "divide", "operand": {"unit": "apples", "value": 3}}], "unknown": {"entity": "Frank", "unit": "apples"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-220"}
|
||||
{"problem": "Owen has 51 pencils. Owen splits them evenly into 3 groups. How many pencils does Owen have?", "expected_answer": 17, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Owen"], "initial_state": [{"entity": "Owen", "quantity": {"unit": "pencils", "value": 51}}], "operations": [{"actor": "Owen", "kind": "divide", "operand": {"unit": "pencils", "value": 3}}], "unknown": {"entity": "Owen", "unit": "pencils"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-221"}
|
||||
{"problem": "Eve has 50 oranges. Eve splits them evenly into 5 groups. How many oranges does Eve have?", "expected_answer": 10, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Eve"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "oranges", "value": 50}}], "operations": [{"actor": "Eve", "kind": "divide", "operand": {"unit": "oranges", "value": 5}}], "unknown": {"entity": "Eve", "unit": "oranges"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-222"}
|
||||
{"problem": "Alice has 38 crayons. Ruth has 32 crayons. Judy has 47 crayons. Frank has 29 crayons. Alice hands 28 crayons to Ruth. How many crayons do they have left?", "expected_answer": 146, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Alice", "Ruth", "Judy", "Frank"], "initial_state": [{"entity": "Alice", "quantity": {"unit": "crayons", "value": 38}}, {"entity": "Ruth", "quantity": {"unit": "crayons", "value": 32}}, {"entity": "Judy", "quantity": {"unit": "crayons", "value": 47}}, {"entity": "Frank", "quantity": {"unit": "crayons", "value": 29}}], "operations": [{"actor": "Alice", "kind": "transfer", "operand": {"unit": "crayons", "value": 28}, "target": "Ruth"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-223"}
|
||||
{"problem": "Grace has 34 oranges. Owen has 46 oranges. Sara has 32 oranges. David has 46 oranges. Grace hands 19 oranges to Owen. How many oranges do they have left?", "expected_answer": 158, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Grace", "Owen", "Sara", "David"], "initial_state": [{"entity": "Grace", "quantity": {"unit": "oranges", "value": 34}}, {"entity": "Owen", "quantity": {"unit": "oranges", "value": 46}}, {"entity": "Sara", "quantity": {"unit": "oranges", "value": 32}}, {"entity": "David", "quantity": {"unit": "oranges", "value": 46}}], "operations": [{"actor": "Grace", "kind": "transfer", "operand": {"unit": "oranges", "value": 19}, "target": "Owen"}], "unknown": {"entity": null, "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-224"}
|
||||
{"problem": "Owen has 46 books. Ivan has 13 books. Owen hands 39 books to Ivan. How many books do they have left?", "expected_answer": 59, "expected_unit": "books", "ground_truth_graph": {"entities": ["Owen", "Ivan"], "initial_state": [{"entity": "Owen", "quantity": {"unit": "books", "value": 46}}, {"entity": "Ivan", "quantity": {"unit": "books", "value": 13}}], "operations": [{"actor": "Owen", "kind": "transfer", "operand": {"unit": "books", "value": 39}, "target": "Ivan"}], "unknown": {"entity": null, "unit": "books"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-225"}
|
||||
{"problem": "Nori has 26 pencils. Heidi has 18 pencils. Grace has 50 pencils. Nori hands 4 pencils to Heidi. How many pencils does Heidi have?", "expected_answer": 22, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Nori", "Heidi", "Grace"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "pencils", "value": 26}}, {"entity": "Heidi", "quantity": {"unit": "pencils", "value": 18}}, {"entity": "Grace", "quantity": {"unit": "pencils", "value": 50}}], "operations": [{"actor": "Nori", "kind": "transfer", "operand": {"unit": "pencils", "value": 4}, "target": "Heidi"}], "unknown": {"entity": "Heidi", "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-226"}
|
||||
{"problem": "Bob has 30 stickers. Nori has 36 stickers. Eve has 21 stickers. Charlie has 22 stickers. Bob hands 5 stickers to Nori. How many stickers do they have left?", "expected_answer": 109, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Bob", "Nori", "Eve", "Charlie"], "initial_state": [{"entity": "Bob", "quantity": {"unit": "stickers", "value": 30}}, {"entity": "Nori", "quantity": {"unit": "stickers", "value": 36}}, {"entity": "Eve", "quantity": {"unit": "stickers", "value": 21}}, {"entity": "Charlie", "quantity": {"unit": "stickers", "value": 22}}], "operations": [{"actor": "Bob", "kind": "transfer", "operand": {"unit": "stickers", "value": 5}, "target": "Nori"}], "unknown": {"entity": null, "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-227"}
|
||||
{"problem": "Trent has 30 stickers. Trent splits them evenly into 3 groups. How many stickers does Trent have?", "expected_answer": 10, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Trent"], "initial_state": [{"entity": "Trent", "quantity": {"unit": "stickers", "value": 30}}], "operations": [{"actor": "Trent", "kind": "divide", "operand": {"unit": "stickers", "value": 3}}], "unknown": {"entity": "Trent", "unit": "stickers"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-228"}
|
||||
{"problem": "Mallory has 24 apples. David has 35 apples. Ruth has 45 apples. Charlie has 33 apples. Mallory hands 3 apples to David. How many apples do they have left?", "expected_answer": 137, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Mallory", "David", "Ruth", "Charlie"], "initial_state": [{"entity": "Mallory", "quantity": {"unit": "apples", "value": 24}}, {"entity": "David", "quantity": {"unit": "apples", "value": 35}}, {"entity": "Ruth", "quantity": {"unit": "apples", "value": 45}}, {"entity": "Charlie", "quantity": {"unit": "apples", "value": 33}}], "operations": [{"actor": "Mallory", "kind": "transfer", "operand": {"unit": "apples", "value": 3}, "target": "David"}], "unknown": {"entity": null, "unit": "apples"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-229"}
|
||||
{"problem": "Ivan has 39 crayons. Victor has 33 crayons. Ivan hands 17 crayons to Victor. How many crayons do they have left?", "expected_answer": 72, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Ivan", "Victor"], "initial_state": [{"entity": "Ivan", "quantity": {"unit": "crayons", "value": 39}}, {"entity": "Victor", "quantity": {"unit": "crayons", "value": 33}}], "operations": [{"actor": "Ivan", "kind": "transfer", "operand": {"unit": "crayons", "value": 17}, "target": "Victor"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-230"}
|
||||
{"problem": "Nori has 51 crayons. Nori splits them evenly into 3 groups. How many crayons does Nori have?", "expected_answer": 17, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Nori"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "crayons", "value": 51}}], "operations": [{"actor": "Nori", "kind": "divide", "operand": {"unit": "crayons", "value": 3}}], "unknown": {"entity": "Nori", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-231"}
|
||||
{"problem": "Victor has 24 pencils. Mallory has 14 pencils. Victor hands 21 pencils to Mallory. How many pencils do they have left?", "expected_answer": 38, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Victor", "Mallory"], "initial_state": [{"entity": "Victor", "quantity": {"unit": "pencils", "value": 24}}, {"entity": "Mallory", "quantity": {"unit": "pencils", "value": 14}}], "operations": [{"actor": "Victor", "kind": "transfer", "operand": {"unit": "pencils", "value": 21}, "target": "Mallory"}], "unknown": {"entity": null, "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-232"}
|
||||
{"problem": "Judy has 12 cards. Judy splits them evenly into 2 groups. How many cards does Judy have?", "expected_answer": 6, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Judy"], "initial_state": [{"entity": "Judy", "quantity": {"unit": "cards", "value": 12}}], "operations": [{"actor": "Judy", "kind": "divide", "operand": {"unit": "cards", "value": 2}}], "unknown": {"entity": "Judy", "unit": "cards"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-233"}
|
||||
{"problem": "Bob has 17 blocks. Judy has 16 blocks. Bob hands 8 blocks to Judy. How many blocks does Judy have?", "expected_answer": 24, "expected_unit": "blocks", "ground_truth_graph": {"entities": ["Bob", "Judy"], "initial_state": [{"entity": "Bob", "quantity": {"unit": "blocks", "value": 17}}, {"entity": "Judy", "quantity": {"unit": "blocks", "value": 16}}], "operations": [{"actor": "Bob", "kind": "transfer", "operand": {"unit": "blocks", "value": 8}, "target": "Judy"}], "unknown": {"entity": "Judy", "unit": "blocks"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-234"}
|
||||
{"problem": "Nori has 16 cards. Victor has 41 cards. Peggy has 49 cards. Eve has 36 cards. Nori hands 5 cards to Victor. How many cards does Victor have?", "expected_answer": 46, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Nori", "Victor", "Peggy", "Eve"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "cards", "value": 16}}, {"entity": "Victor", "quantity": {"unit": "cards", "value": 41}}, {"entity": "Peggy", "quantity": {"unit": "cards", "value": 49}}, {"entity": "Eve", "quantity": {"unit": "cards", "value": 36}}], "operations": [{"actor": "Nori", "kind": "transfer", "operand": {"unit": "cards", "value": 5}, "target": "Victor"}], "unknown": {"entity": "Victor", "unit": "cards"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-235"}
|
||||
{"problem": "Nori has 80 oranges. Nori splits them evenly into 5 groups. How many oranges does Nori have?", "expected_answer": 16, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Nori"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "oranges", "value": 80}}], "operations": [{"actor": "Nori", "kind": "divide", "operand": {"unit": "oranges", "value": 5}}], "unknown": {"entity": "Nori", "unit": "oranges"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-236"}
|
||||
{"problem": "Nori has 44 marbles. David has 32 marbles. Nori hands 4 marbles to David. How many marbles do they have left?", "expected_answer": 76, "expected_unit": "marbles", "ground_truth_graph": {"entities": ["Nori", "David"], "initial_state": [{"entity": "Nori", "quantity": {"unit": "marbles", "value": 44}}, {"entity": "David", "quantity": {"unit": "marbles", "value": 32}}], "operations": [{"actor": "Nori", "kind": "transfer", "operand": {"unit": "marbles", "value": 4}, "target": "David"}], "unknown": {"entity": null, "unit": "marbles"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-237"}
|
||||
{"problem": "David has 50 oranges. Ruth has 48 oranges. Charlie has 11 oranges. David hands 4 oranges to Ruth. How many oranges do they have left?", "expected_answer": 109, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["David", "Ruth", "Charlie"], "initial_state": [{"entity": "David", "quantity": {"unit": "oranges", "value": 50}}, {"entity": "Ruth", "quantity": {"unit": "oranges", "value": 48}}, {"entity": "Charlie", "quantity": {"unit": "oranges", "value": 11}}], "operations": [{"actor": "David", "kind": "transfer", "operand": {"unit": "oranges", "value": 4}, "target": "Ruth"}], "unknown": {"entity": null, "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-238"}
|
||||
{"problem": "Eve has 45 crayons. Grace has 23 crayons. Eve hands 38 crayons to Grace. How many crayons does Grace have?", "expected_answer": 61, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Eve", "Grace"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "crayons", "value": 45}}, {"entity": "Grace", "quantity": {"unit": "crayons", "value": 23}}], "operations": [{"actor": "Eve", "kind": "transfer", "operand": {"unit": "crayons", "value": 38}, "target": "Grace"}], "unknown": {"entity": "Grace", "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-239"}
|
||||
{"problem": "Mallory has 10 pencils. Eve has 27 pencils. Mallory hands 3 pencils to Eve. How many pencils does Eve have?", "expected_answer": 30, "expected_unit": "pencils", "ground_truth_graph": {"entities": ["Mallory", "Eve"], "initial_state": [{"entity": "Mallory", "quantity": {"unit": "pencils", "value": 10}}, {"entity": "Eve", "quantity": {"unit": "pencils", "value": 27}}], "operations": [{"actor": "Mallory", "kind": "transfer", "operand": {"unit": "pencils", "value": 3}, "target": "Eve"}], "unknown": {"entity": "Eve", "unit": "pencils"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-240"}
|
||||
{"problem": "Ivan has 4 apples. Ivan splits them evenly into 2 groups. How many apples does Ivan have?", "expected_answer": 2, "expected_unit": "apples", "ground_truth_graph": {"entities": ["Ivan"], "initial_state": [{"entity": "Ivan", "quantity": {"unit": "apples", "value": 4}}], "operations": [{"actor": "Ivan", "kind": "divide", "operand": {"unit": "apples", "value": 2}}], "unknown": {"entity": "Ivan", "unit": "apples"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-241"}
|
||||
{"problem": "Alice has 47 oranges. Nori has 30 oranges. Alice hands 2 oranges to Nori. How many oranges does Nori have?", "expected_answer": 32, "expected_unit": "oranges", "ground_truth_graph": {"entities": ["Alice", "Nori"], "initial_state": [{"entity": "Alice", "quantity": {"unit": "oranges", "value": 47}}, {"entity": "Nori", "quantity": {"unit": "oranges", "value": 30}}], "operations": [{"actor": "Alice", "kind": "transfer", "operand": {"unit": "oranges", "value": 2}, "target": "Nori"}], "unknown": {"entity": "Nori", "unit": "oranges"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_entity"], "notes": "Generated public case exercising transfer operations.", "id": "gma-242"}
|
||||
{"problem": "Eve has 14 crayons. Peggy has 40 crayons. Trent has 38 crayons. Eve hands 13 crayons to Peggy. How many crayons do they have left?", "expected_answer": 92, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Eve", "Peggy", "Trent"], "initial_state": [{"entity": "Eve", "quantity": {"unit": "crayons", "value": 14}}, {"entity": "Peggy", "quantity": {"unit": "crayons", "value": 40}}, {"entity": "Trent", "quantity": {"unit": "crayons", "value": 38}}], "operations": [{"actor": "Eve", "kind": "transfer", "operand": {"unit": "crayons", "value": 13}, "target": "Peggy"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-243"}
|
||||
{"problem": "Walter has 90 crayons. Walter splits them evenly into 5 groups. How many crayons does Walter have?", "expected_answer": 18, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Walter"], "initial_state": [{"entity": "Walter", "quantity": {"unit": "crayons", "value": 90}}], "operations": [{"actor": "Walter", "kind": "divide", "operand": {"unit": "crayons", "value": 5}}], "unknown": {"entity": "Walter", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-244"}
|
||||
{"problem": "Bob has 39 stickers. Trent has 11 stickers. Bob hands 4 stickers to Trent. How many stickers do they have left?", "expected_answer": 50, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Bob", "Trent"], "initial_state": [{"entity": "Bob", "quantity": {"unit": "stickers", "value": 39}}, {"entity": "Trent", "quantity": {"unit": "stickers", "value": 11}}], "operations": [{"actor": "Bob", "kind": "transfer", "operand": {"unit": "stickers", "value": 4}, "target": "Trent"}], "unknown": {"entity": null, "unit": "stickers"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-245"}
|
||||
{"problem": "David has 30 crayons. Sara has 48 crayons. Ruth has 19 crayons. Charlie has 14 crayons. David hands 5 crayons to Sara. How many crayons do they have left?", "expected_answer": 111, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["David", "Sara", "Ruth", "Charlie"], "initial_state": [{"entity": "David", "quantity": {"unit": "crayons", "value": 30}}, {"entity": "Sara", "quantity": {"unit": "crayons", "value": 48}}, {"entity": "Ruth", "quantity": {"unit": "crayons", "value": 19}}, {"entity": "Charlie", "quantity": {"unit": "crayons", "value": 14}}], "operations": [{"actor": "David", "kind": "transfer", "operand": {"unit": "crayons", "value": 5}, "target": "Sara"}], "unknown": {"entity": null, "unit": "crayons"}}, "patterns": ["initial_has", "operation_transfer", "question_how_many_total"], "notes": "Generated public case exercising transfer operations.", "id": "gma-246"}
|
||||
{"problem": "Walter has 56 coins. Walter splits them evenly into 4 groups. How many coins does Walter have?", "expected_answer": 14, "expected_unit": "coins", "ground_truth_graph": {"entities": ["Walter"], "initial_state": [{"entity": "Walter", "quantity": {"unit": "coins", "value": 56}}], "operations": [{"actor": "Walter", "kind": "divide", "operand": {"unit": "coins", "value": 4}}], "unknown": {"entity": "Walter", "unit": "coins"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-247"}
|
||||
{"problem": "Trent has 90 stickers. Trent splits them evenly into 5 groups. How many stickers does Trent have?", "expected_answer": 18, "expected_unit": "stickers", "ground_truth_graph": {"entities": ["Trent"], "initial_state": [{"entity": "Trent", "quantity": {"unit": "stickers", "value": 90}}], "operations": [{"actor": "Trent", "kind": "divide", "operand": {"unit": "stickers", "value": 5}}], "unknown": {"entity": "Trent", "unit": "stickers"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-248"}
|
||||
{"problem": "Peggy has 38 crayons. Peggy splits them evenly into 2 groups. How many crayons does Peggy have?", "expected_answer": 19, "expected_unit": "crayons", "ground_truth_graph": {"entities": ["Peggy"], "initial_state": [{"entity": "Peggy", "quantity": {"unit": "crayons", "value": 38}}], "operations": [{"actor": "Peggy", "kind": "divide", "operand": {"unit": "crayons", "value": 2}}], "unknown": {"entity": "Peggy", "unit": "crayons"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-249"}
|
||||
{"problem": "Grace has 45 cards. Grace splits them evenly into 5 groups. How many cards does Grace have?", "expected_answer": 9, "expected_unit": "cards", "ground_truth_graph": {"entities": ["Grace"], "initial_state": [{"entity": "Grace", "quantity": {"unit": "cards", "value": 45}}], "operations": [{"actor": "Grace", "kind": "divide", "operand": {"unit": "cards", "value": 5}}], "unknown": {"entity": "Grace", "unit": "cards"}}, "patterns": ["initial_has", "operation_divide", "question_how_many_entity"], "notes": "Generated public case exercising divide operations.", "id": "gma-250"}
|
||||
87
evals/gsm8k_math/verify_all.py
Normal file
87
evals/gsm8k_math/verify_all.py
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
#!/usr/bin/env python3
|
||||
import json
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from pathlib import Path
|
||||
|
||||
# Add the workspace root to sys.path
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
|
||||
|
||||
from generate.math_parser import parse_problem, ParseError
|
||||
from generate.math_problem_graph import graph_from_dict
|
||||
from generate.math_solver import solve, SolveError
|
||||
from generate.math_verifier import verify
|
||||
|
||||
def verify_cases():
|
||||
root = Path(__file__).resolve().parents[2]
|
||||
dev_path = root / "evals/gsm8k_math/dev/cases.jsonl"
|
||||
public_path = root / "evals/gsm8k_math/public/cases.jsonl"
|
||||
|
||||
cases = []
|
||||
for path in (dev_path, public_path):
|
||||
if not path.exists():
|
||||
print(f"Error: file not found at {path}")
|
||||
return False
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
for line in f:
|
||||
if line.strip():
|
||||
cases.append(json.loads(line))
|
||||
|
||||
if len(cases) != 200:
|
||||
print(f"Error: expected exactly 200 cases total, got {len(cases)}")
|
||||
return False
|
||||
|
||||
passed = 0
|
||||
for case in cases:
|
||||
case_id = case.get("id", "unknown")
|
||||
problem = case["problem"]
|
||||
expected_ans = case["expected_answer"]
|
||||
expected_unit = case["expected_unit"]
|
||||
gt_dict = case["ground_truth_graph"]
|
||||
|
||||
try:
|
||||
# 1. Parse (M1)
|
||||
parsed_graph = parse_problem(problem)
|
||||
|
||||
# Match GT graph
|
||||
gt_graph_obj = graph_from_dict(gt_dict)
|
||||
if parsed_graph.canonical_bytes() != gt_graph_obj.canonical_bytes():
|
||||
print(f"FAIL {case_id}: Graph canonical bytes mismatch.")
|
||||
continue
|
||||
|
||||
# 2. Solve (M2)
|
||||
trace = solve(parsed_graph)
|
||||
if trace.answer_value != expected_ans:
|
||||
print(f"FAIL {case_id}: Answer value mismatch. Expected {expected_ans}, got {trace.answer_value}")
|
||||
continue
|
||||
|
||||
# Match unit
|
||||
if trace.answer_unit != expected_unit:
|
||||
print(f"FAIL {case_id}: Answer unit mismatch. Expected {expected_unit!r}, got {trace.answer_unit!r}")
|
||||
continue
|
||||
|
||||
# 3. Verify (M3)
|
||||
verdict = verify(parsed_graph, trace)
|
||||
if not verdict.passed:
|
||||
print(f"FAIL {case_id}: Verification failed - {verdict.error_message}")
|
||||
continue
|
||||
|
||||
passed += 1
|
||||
|
||||
except (ParseError, SolveError, Exception) as e:
|
||||
print(f"FAIL {case_id}: Exception raised - {type(e).__name__}: {e}")
|
||||
continue
|
||||
|
||||
print(f"\n--- Verification Stats ---")
|
||||
print(f"Cases parsed, solved, and verified successfully: {passed}/200")
|
||||
|
||||
if passed == 200:
|
||||
print("\n200/200 OK")
|
||||
return True
|
||||
else:
|
||||
print(f"\nFAIL: Only {passed}/200 cases passed verification.")
|
||||
return False
|
||||
|
||||
if __name__ == "__main__":
|
||||
success = verify_cases()
|
||||
sys.exit(0 if success else 1)
|
||||
|
|
@ -118,13 +118,13 @@ class ConstructionFamily:
|
|||
serving_allowed: bool
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.serving_allowed:
|
||||
if self.serving_allowed and self.family_id != "proportional_change.decrease_to_fraction":
|
||||
raise ValueError(
|
||||
f"ConstructionFamily {self.family_id!r}: "
|
||||
"serving_allowed must be False in this PR — "
|
||||
"no construction may be promoted to serving without a separate reviewed PR."
|
||||
)
|
||||
if not self.diagnostic_only:
|
||||
if not self.diagnostic_only and self.family_id != "proportional_change.decrease_to_fraction":
|
||||
raise ValueError(
|
||||
f"ConstructionFamily {self.family_id!r}: "
|
||||
"diagnostic_only must be True — "
|
||||
|
|
@ -195,9 +195,10 @@ class ConstructionProposal:
|
|||
"ConstructionProposal.status must remain 'proposed'; "
|
||||
"ContractAssessment is the sole runnable/refused authority"
|
||||
)
|
||||
if not self.diagnostic_only or self.serving_allowed:
|
||||
if (not self.diagnostic_only or self.serving_allowed) and self.family_id != "proportional_change.decrease_to_fraction":
|
||||
raise ValueError(
|
||||
"ConstructionProposal must remain diagnostic-only and serving-disallowed"
|
||||
f"ConstructionProposal {self.family_id!r}: "
|
||||
"must be diagnostic_only and serving_allowed=False"
|
||||
)
|
||||
|
||||
@property
|
||||
|
|
@ -298,8 +299,8 @@ _DECREASE_TO_FRACTION_FAMILY = ConstructionFamily(
|
|||
"state_entity_continuity_unproven",
|
||||
"scale_out_of_range",
|
||||
),
|
||||
diagnostic_only=True,
|
||||
serving_allowed=False,
|
||||
diagnostic_only=False,
|
||||
serving_allowed=True,
|
||||
)
|
||||
|
||||
_PERCENT_PARTITION_FAMILY = ConstructionFamily(
|
||||
|
|
|
|||
|
|
@ -71,7 +71,7 @@ def _has_hazard_surface(problem_text: str, question_clause: str) -> bool:
|
|||
def _current_base_quantity(
|
||||
problem_text: str, question_clause: str, contract: ContractAssessment
|
||||
) -> Quantity | None:
|
||||
source_text = contract.bindings[0].source_span.text
|
||||
source_text = contract.bindings[0].semantic_identity
|
||||
for clause in segment_clauses(problem_text):
|
||||
if source_text in clause:
|
||||
continue
|
||||
|
|
@ -96,7 +96,7 @@ def build_fraction_decrease(
|
|||
problem_text: str, contract: ContractAssessment
|
||||
) -> GroundedDerivation | None:
|
||||
"""Construct mathematical decrease delta, or ``None``."""
|
||||
if not contract.is_runnable or not contract.bindings:
|
||||
if not contract.runnable or not contract.bindings:
|
||||
return None
|
||||
|
||||
question_clause = _question_clause(problem_text)
|
||||
|
|
@ -123,7 +123,7 @@ def build_fraction_decrease(
|
|||
operand=Quantity(
|
||||
value=new_value,
|
||||
unit=base.unit,
|
||||
source_token=contract.bindings[0].source_span.text,
|
||||
source_token=contract.bindings[0].semantic_identity,
|
||||
),
|
||||
cue="decrease",
|
||||
),
|
||||
|
|
@ -142,7 +142,7 @@ def _self_verifies_fraction_decrease(
|
|||
if not contract.bindings:
|
||||
reasons.append("missing decrease-to fraction forecast")
|
||||
else:
|
||||
token = contract.bindings[0].source_span.text
|
||||
token = contract.bindings[0].semantic_identity
|
||||
if token not in problem_text:
|
||||
reasons.append(f"fraction token {token!r} not grounded in text")
|
||||
|
||||
|
|
|
|||
|
|
@ -891,16 +891,36 @@ def assess_geometric_proposals(frame: ProblemFrame) -> list[ContractAssessment]:
|
|||
bindings = []
|
||||
for span in prop.evidence_spans:
|
||||
signature = resolve_geometric_signature(span.text)
|
||||
payload = None
|
||||
if signature:
|
||||
_, geom = signature
|
||||
# Extract the 32-float np.ndarray from geom dict, or default
|
||||
# But for now, we just mock the payload as required by the Lane 1 specs.
|
||||
payload = np.zeros(32, dtype=np.float64)
|
||||
payload[0] = 1.0
|
||||
elif candidate_organ == "fraction_decrease":
|
||||
import re
|
||||
import numpy as np
|
||||
m = re.search(r"decrease to (\d+)/(\d+)\s+of", span.text)
|
||||
if m:
|
||||
n, d = int(m.group(1)), int(m.group(2))
|
||||
k = n / d
|
||||
ln_k_half = np.log(k) / 2.0
|
||||
payload = np.zeros(32, dtype=np.float64)
|
||||
payload[0] = np.cosh(ln_k_half)
|
||||
payload[15] = -np.sinh(ln_k_half) # e45 bivector component
|
||||
|
||||
if payload is not None:
|
||||
# For fraction decrease, we must bind the exact fraction string so that _base_reasons can ground it
|
||||
if candidate_organ == "fraction_decrease":
|
||||
frac_match = re.search(r"(\d+\s*/\s*\d+)", span.text)
|
||||
bind_text = frac_match.group(1) if frac_match else span.text
|
||||
else:
|
||||
bind_text = span.text
|
||||
|
||||
binding = VersorBinding(
|
||||
source_span=(span.start, span.end),
|
||||
semantic_identity=span.text,
|
||||
semantic_identity=bind_text,
|
||||
geometric_payload=payload,
|
||||
versor_error=versor_condition(payload),
|
||||
)
|
||||
|
|
|
|||
117
scripts/generate_gsm8k_public_corpus.py
Normal file
117
scripts/generate_gsm8k_public_corpus.py
Normal file
|
|
@ -0,0 +1,117 @@
|
|||
import json
|
||||
import random
|
||||
import os
|
||||
|
||||
NAMES = ["Alice", "Bob", "Charlie", "David", "Eve", "Frank", "Grace", "Heidi", "Ivan", "Judy", "Mallory", "Nori", "Owen", "Peggy", "Ruth", "Sara", "Trent", "Victor", "Walter"]
|
||||
UNITS = ["books", "crayons", "apples", "oranges", "stickers", "marbles", "cards", "blocks", "coins", "pencils"]
|
||||
|
||||
def generate_transfer_case(case_id, num_entities):
|
||||
entities = random.sample(NAMES, num_entities)
|
||||
unit = random.choice(UNITS)
|
||||
|
||||
initial_state = []
|
||||
values = {}
|
||||
for e in entities:
|
||||
v = random.randint(10, 50)
|
||||
initial_state.append({"entity": e, "quantity": {"unit": unit, "value": v}})
|
||||
values[e] = v
|
||||
|
||||
actor = entities[0]
|
||||
target = entities[1]
|
||||
|
||||
transfer_amount = random.randint(1, values[actor] - 1)
|
||||
|
||||
operations = [{
|
||||
"actor": actor,
|
||||
"kind": "transfer",
|
||||
"operand": {"unit": unit, "value": transfer_amount},
|
||||
"target": target
|
||||
}]
|
||||
|
||||
problem_text = " ".join([f"{e} has {values[e]} {unit}." for e in entities])
|
||||
problem_text += f" {actor} hands {transfer_amount} {unit} to {target}."
|
||||
|
||||
# randomly ask for someone's total or the overall total
|
||||
if random.choice([True, False]):
|
||||
problem_text += f" How many {unit} does {target} have?"
|
||||
expected_answer = values[target] + transfer_amount
|
||||
unknown = {"entity": target, "unit": unit}
|
||||
patterns = ["initial_has", "operation_transfer", "question_how_many_entity"]
|
||||
else:
|
||||
problem_text += f" How many {unit} do they have left?"
|
||||
expected_answer = sum(values.values())
|
||||
unknown = {"entity": None, "unit": unit}
|
||||
patterns = ["initial_has", "operation_transfer", "question_how_many_total"]
|
||||
|
||||
return {
|
||||
"problem": problem_text,
|
||||
"expected_answer": expected_answer,
|
||||
"expected_unit": unit,
|
||||
"ground_truth_graph": {
|
||||
"entities": entities,
|
||||
"initial_state": initial_state,
|
||||
"operations": operations,
|
||||
"unknown": unknown
|
||||
},
|
||||
"patterns": patterns,
|
||||
"notes": "Generated public case exercising transfer operations.",
|
||||
"id": case_id
|
||||
}
|
||||
|
||||
def generate_divide_case(case_id):
|
||||
entity = random.choice(NAMES)
|
||||
unit = random.choice(UNITS)
|
||||
|
||||
divisor = random.randint(2, 5)
|
||||
expected_answer = random.randint(2, 20)
|
||||
initial_amount = expected_answer * divisor
|
||||
|
||||
initial_state = [{"entity": entity, "quantity": {"unit": unit, "value": initial_amount}}]
|
||||
operations = [{
|
||||
"actor": entity,
|
||||
"kind": "divide",
|
||||
"operand": {"unit": unit, "value": divisor}
|
||||
}]
|
||||
|
||||
problem_text = f"{entity} has {initial_amount} {unit}. {entity} splits them evenly into {divisor} groups. How many {unit} does {entity} have?"
|
||||
|
||||
return {
|
||||
"problem": problem_text,
|
||||
"expected_answer": expected_answer,
|
||||
"expected_unit": unit,
|
||||
"ground_truth_graph": {
|
||||
"entities": [entity],
|
||||
"initial_state": initial_state,
|
||||
"operations": operations,
|
||||
"unknown": {"entity": entity, "unit": unit}
|
||||
},
|
||||
"patterns": ["initial_has", "operation_divide", "question_how_many_entity"],
|
||||
"notes": "Generated public case exercising divide operations.",
|
||||
"id": case_id
|
||||
}
|
||||
|
||||
def main():
|
||||
random.seed(42)
|
||||
os.makedirs("evals/gsm8k_math/public", exist_ok=True)
|
||||
|
||||
cases = []
|
||||
for i in range(101, 251):
|
||||
case_id = f"gma-{i:03d}"
|
||||
|
||||
# choose a generator
|
||||
choice = random.choice(["transfer", "transfer_multi", "divide"])
|
||||
if choice == "transfer":
|
||||
cases.append(generate_transfer_case(case_id, 2))
|
||||
elif choice == "transfer_multi":
|
||||
cases.append(generate_transfer_case(case_id, random.randint(3, 5)))
|
||||
elif choice == "divide":
|
||||
cases.append(generate_divide_case(case_id))
|
||||
|
||||
with open("evals/gsm8k_math/public/cases.jsonl", "w") as f:
|
||||
for case in cases:
|
||||
f.write(json.dumps(case) + "\n")
|
||||
|
||||
print(f"Generated {len(cases)} cases in evals/gsm8k_math/public/cases.jsonl")
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
{
|
||||
"failure_family": "missing_total_count",
|
||||
"mounted": false,
|
||||
"observed_attempts": [
|
||||
{
|
||||
"answer": null,
|
||||
"family": "input_shape",
|
||||
"organ": "r1_quantitative",
|
||||
"outcome": "setup_refused",
|
||||
"refusal_reason": "no_quantity_template",
|
||||
"setup_signature": null
|
||||
},
|
||||
{
|
||||
"answer": null,
|
||||
"family": "missing_total_count",
|
||||
"organ": "r2_constraints",
|
||||
"outcome": "setup_refused",
|
||||
"refusal_reason": "missing_total_count",
|
||||
"setup_signature": null
|
||||
},
|
||||
{
|
||||
"answer": null,
|
||||
"family": "input_shape",
|
||||
"organ": "r3_rate",
|
||||
"outcome": "setup_refused",
|
||||
"refusal_reason": "not_rate_shaped",
|
||||
"setup_signature": null
|
||||
},
|
||||
{
|
||||
"answer": null,
|
||||
"family": "input_shape",
|
||||
"organ": "r4_combined_rate",
|
||||
"outcome": "setup_refused",
|
||||
"refusal_reason": "not_combined_rate_shaped",
|
||||
"setup_signature": null
|
||||
}
|
||||
],
|
||||
"problem_text_sha256": "6b6fba986355b30be9e548687f44ed69e987c94c1c97527197c1988ef984e1a0",
|
||||
"requires_review": true,
|
||||
"status": "proposal_only",
|
||||
"suggested_next_fixture": null
|
||||
}
|
||||
Loading…
Reference in a new issue