In 1999 a validation study of the 1998 Civics Assessment of the National Assessment of Educational Progress (NAEP) was conducted, combining a similarity classification study and a booklet classification study. The rationale was that if the teachers who had participated in the 5-day achievement levels-setting process could not use the descriptions…

This paper describes the feedback included in operational achievement level setting (ALS) procedures for the National Assessment of Educational Progress (NAEP). It does not describe the feedback form field trials, pilot studies, or other research studies related to the NAEP. The NAEP ALS process includes three rounds of item-by-item ratings, and…

Applied the analytical judgment standard setting method to 90 papers from the 1996 Grade 8 National Assessment of Educational Progress science assessment. Compared sorting versus direct classification, long and short versions of the classification scale, and effects of discussion on cutscores. Results from 17 Georgia teachers and 8 Michigan…

This paper reports on a standard-setting method designed for complex performance assessments with multiple performance categories. The method studied, the Analytical Judgment Method, involves panelists' making analytical classification decisions for each of the test's components individually. It also allows for discussion and reconsideration of…