Historically, AP Potential™ has used PSAT/NMSQT® scores to identify students who are likely to earn a 3 or higher on a specific AP® Exam--based on research showing moderate to strong relationships between PSAT/NMSQT scores and AP Exam scores (Camara & Millsap, 1998; Ewing, Camara & Millsap, 2006; Zhang, Patel & Ewing, 2014a). For most…

Developing a cognitive model of task performance is an important and often overlooked phase in assessment design; failing to establish such a model can threaten the validity of the inferences made from the scores produced by an assessment (e.g., Leighton, 2004). Conducting think aloud interviews (TAIs), where students think aloud while completing…

Presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation provides an overview of ECD. In addition, the presentation describes the benefits of, as well as the challenges that were faced, in implementing ECD in the Advanced Placement Program.

Presented at the Annual Meeting of National Council on Measurement in Education (NCME) in San Diego, CA in April 2009. This presentation describes the methodology that was used with subject-matter experts (SMEs) to articulate the content and skills important in the domain, and then the iterative processes that were used to articulate the claims…

This paper discusses issues related to the design and psychometric work for mixed-format tests --tests containing both multiple-choice (MC) and constructed-response (CR) items. The issues of validity, fairness, reliability and score consistency can be addressed but for mixed-format tests there are many decisions to be made and no examination or…

The psychometric considerations and challenges associated with including constructed response items on tests are discussed along with how these issues affect the form assembly specifications for mixed-format exams. Reliability and validity, security and fairness, pretesting, content and skills coverage, test length and timing, weights, statistical…

Presented at the Annual National Council on Measurement in Education (NCME) in New York in March 2008. This presentation explores how different item weighting can affect the effective weights, validity coefficents and test reliability of composite scores among test takers.

The Advanced Placement Program® (AP®) is dedicated to providing high school students opportunities to enroll in college-level courses while in high school. The advantages of such accelerated opportunities are both financial and educational. The AP Program is available internationally. As a result of its widespread availability, considerable effort…