In this research report, we describe a study to map the scores of the "TOEFL® Essentials"™ test to the Canadian Language Benchmarks (CLB). The TOEFL Essentials test is a four-skills assessment of foundational English language skills and communication abilities in academic and general (daily life) contexts. At the time of writing this…

One common phenomenon in Angoff standard setting is that panelists regress their ratings in toward the middle of the probability scale. This study describes two indices based on taking ratios of standard deviations that can be utilized with a scatterplot of item ratings versus expected probabilities of success to identify whether ratings are…

Purpose: In the last 5 years, many states have introduced science of reading (SOR) reforms that require increased attention to foundational skills instruction in grades K-3. The fast spread of these policies raises questions about the mechanisms that facilitated their rapid adoption. The purpose of this article is to examine how SOR discourses…

In this digital ITEMS module, Dr. Michael Bunch provides an in-depth, step-by-step look at how standard setting is done. It does not focus on any specific procedure or methodology (e.g., modified Angoff, bookmark, and body of work) but on the practical tasks that must be completed for any standard setting activity. Dr. Bunch carries the…

Standard setting is arguably one of the most subjective techniques in test development and psychometrics. The decisions when scores are compared to standards, however, are arguably the most consequential outcomes of testing. Providing licensure to practice in a profession has high stake consequences for the public. Denying graduation or forcing…

A common belief is that the Bookmark method is a cognitively simpler standard-setting method than the modified Angoff method. However, a limited amount of research has investigated panelist's ability to perform well the Bookmark method, and whether some of the challenges panelists face with the Angoff method may also be present in the Bookmark…

This study aims to demonstrate the optimal way to determine the cut-off score to be used to interpret the total scores obtained from an achievement test or scale using the Artificial Neural Networks method. To this end, the multiple-choice item responses in the Booklet-11 Mathematics subtest at the 8th grade level in the TIMSS 2015 Turkey sample…

The study reported on here contributes to the growing body of knowledge on the use of standard setting methods for improving the reporting and utility value of assessment results in South Africa as well as for addressing the conceptual shortcomings of the Curriculum and Assessment Policy Statement (CAPS) reporting framework. Using data from the…

Mealtimes in early learning settings present rich opportunities for connection, learning, and positive food experiences. Make the most of mealtimes--and prevent and address common feeding challenges--with this user-friendly planning guide, the first of its kind for early childhood educators and care providers. Written by a cross-disciplinary team…

This report provides a resource for state and district assessment staff as they develop requests for proposals (RFPs) for interim assessments for all of their students, including those who require accessibility features, to demonstrate their knowledge and skills as well as those who may need an alternate assessment because of their significant…

In this paper we assert that the practice of principled assessment design renders traditional standard-setting methodology redundant at best and contradictory at worst. We describe the rationale for, and methodological details of, Embedded Standard Setting (ESS; previously, Engineered Cut Scores. Lewis, 2016), an approach to establish performance…

In this study, the cut-off scores obtained from the Angoff, Angoff Y/N, Nedelsky and Ebel standard methods were compared with the 50 T score and the current cut-off score in various aspects. Data were collected from 448 students who took Module B1+ English Exit Exam IV and 14 experts. It was seen that while the Nedelsky method gave the lowest…

Standard-setting is an essential component of test development, supporting the meaningfulness and appropriate interpretation of test scores. However, in the high-stakes testing environment of aviation, standard-setting studies are underexplored. To address this gap, we document two stages in the standard-setting procedures for the Overseas Flight…

The purpose of this action research study was to examine grading practices in one military-connected high school and to make recommendations for aligning grading practices to improve student achievement, engagement, and equity. Through iterative research cycles of look, think, and act, teachers from one military-connected high school participated…

This chapter focuses on Education and Training 2020 (ET 2020), a significant European policy framework that provides an integrated approach to education and training. The chapter identifies the main governance mechanisms and policy instruments used by the European Union (EU) institutions within ET 2020 that lead to increased domestic policy…