Online credit recovery (OCR) courses are the most common means through which students retake courses required for high school graduation. Yet a growing body of research has raised concerns regarding student learning in these courses, with low quality assessments posited as one contributing factor. To address this concern, we reviewed every assessment item from a widely used OCR Algebra 1 course. We also examined pathways for passing the course mastery tests without learning content. In addition, we identified if and how states regulate OCR. We found OCR assessments as executed lacked rigor and validity. We offer recommendations to improve rigor, close pathways that call into question the validity of results, strengthen implementation procedures, and increase state-level oversight of providers.