This study examines the implementation of "Criterion," an automated writing evaluation system developed by ETS, as a source of diagnostic feedback on learners' linguistic performance in a Vietnamese EFL writing classroom. Thirty-eight second-year English majors had access to "Criterion" for a five-month period. Data include "Criterion" error tags on students' essays from multiple practice sessions, recorded think-aloud protocols as students engaged with the feedback for revisions, and first and revised drafts students submitted to "Criterion." The main findings indicate "Criterion's" satisfactory precision and capacity to trigger various engagement strategies among learners, but reservations remain due to students' modest response accuracy and lack of substantive revisions to their texts. Important implications for formative feedback practices in EFL writing classrooms and the adaptation of "Criterion's" technical capacities are accordingly presented.