In mathematics, the keyword strategy involves identifying a keyword and using that keyword to determine the operation needed to find a word problem's solution. We analyzed 747 high-stakes released items across grades 3, 4, 5, 6, 7, and 8. Of these, 57 items did not involve written text. Of the 690 text-based items, we classified 69% as directive word problems and 31% as routineword problems. For all 690 items, we identified any keywords appearing in the text of the word problem. We categorized the 214 routine word problems by schema. We identified keywords within these problems and determined whether a keyword and its implied operation matched the correct problem solution. For single-step routine word problems, we determined that keywords featured within the problem led to a correct problem solution with less than a 50% match rate. For multistep routine word problems, the match rate was less than 10%.