This article examines definitions, rationales, and calculations associated with higher education performance measures: persistence, retention rate, attrition rate, drop-out rate, and graduation rate. Strengths and limitations of these measures are scrutinized relative to online master's programs. Outcomes of a sample of students (N = 96) enrolled in multi-university online master's programs sponsored by Great Plains IDEA are tracked over six years. The students' unique characteristics and degree completion patterns suggest modifying performance metrics and increasing the tracking time. The study empirically proves that existing graduation and retention metrics used to compare online to face-to-face programs underreport successful outcomes for online master's students by at least one-third. Such comparisons should be made with caution, using consistent terminology and recognizing the limitations of existing metrics. Standard methods for measuring student persistence in master's degree programs should be universally established at the policy level. (Contains 1 table and 2 figures.)