The purpose of this study was to critically examine the absolute grading system applied to the English section of South Korea's College Scholastic Ability Test (CSAT), in a departure from a relative grading system, for the first time in 2017. English language instructors as well as English education researchers have somewhat differing understandings of the concept of absolute evaluation/grading. This paper investigated the nature of absolute evaluation/grading as presented in theses and academic papers as well as research reports, English language assessment textbooks, and press releases from the Ministry of Education (MOE). The study identified four different testing contexts in which the term absolute evaluation/grading is applied: criterion-referenced testing; grading based on fixed cut scores; measurement against external criteria; and norm-referenced testing. While retaining the same test structures as in the prior norm-referenced testing situation, the CSAT English section corresponds to the second context, that is, interpreting results based on fixed cut scores. Achievement standards essential for producing criterion-referenced tests have so far been undefined, with cut scores being arbitrarily set. The examination concluded that the CSAT English section under an absolute grading scale is essentially a norm-referenced test that produces a significantly larger percentage of higher-level test-takers. There is a fair likelihood that, unlike MOE's claim, converting to an absolute evaluation/grading system may not bring about the intended positive change.