I review opportunities and threats that widely accessible Artificial Intelligence (AI)-powered services present for educational statistics and measurement. Algorithmic and computational advances continue to improve approaches to item generation, scale maintenance, test security, test scoring, and score reporting. Predictable misuses of AI for…

A key issue of cognitive diagnostic models (CDMs) is the correct identification of Q-matrix which indicates the relationship between attributes and test items. Previous CDMs typically assumed a known Q-matrix provided by domain experts such as those who developed the questions. However, misspecifications of Q-matrix had been discovered in the past…

The "2004/06 Beginning Postsecondary Students Longitudinal Study (BPS:04/06)" is sponsored by the U.S. Department of Education to respond to the need for a national, comprehensive database concerning issues students may face in enrollment, persistence, progress, and attainment in postsecondary education and in consequent early rates of…

Major analyses of the achievement tests used in the Beginning Teacher Evaluation Study were conducted to determine test reactivity to instruction. Reading and mathematics tests were administered to second and fifth grade children. Classroom teachers' records were examined to determine the amount of opportunity students had to learn the content…

This article proposes an item exposure control method, which is the extension of the Sympson and Hetter procedure and can provide item exposure control at both the item and test levels. Item exposure rate and test overlap rate are two indices commonly used to track item exposure in computerized adaptive tests. By considering both indices, item…

This report describes the methods and procedures used for the Beginning Postsecondary Students Longitudinal Study: 1996-2001 (BPS:1996/2001). These students, who started their postsecondary education during the 1995-96 academic year, were first interviewed in 1996 as part of the National Postsecondary Student Aid Study (NPSAS:96). A follow-up…

It is a widely held belief that an anchor test used in equating should be a miniature version (or "minitest") of the tests to be equated; that is, the anchor test should be proportionally representative of the two tests in content and statistical characteristics. This paper examines the scientific foundation of this belief, especially…

Achievement tests that are maximally sensitive to effective instruction in reading and mathematics for grades 2 and 5 were developed and refined. Important considerations regarding the tests' validity were: its coverage of instructional content (opportunity to learn), and its reactivity to instruction. Student ability must be minimally related to…

As a preliminary step in customizing the Stanford Achievement Test, a study was made to compare the test items at each level with the curriculum objectives of a Florida school district, classifying them as either matching, above or below grade level, or unmatching. Results are shown in table form. (TE)

In a preceding research report, ONR/RR-82-1 (Information Loss Caused by Noise in Models for Dichotomous Items), observations were made on the effect of noise accommodated in different types of models on the dichotomous response level. In the present paper, focus is put upon the three-parameter logistic model, which is widely used among…

A study was undertaken to develop guidelines for the interpretation of the parameters of three multidimensional item response theory models and to determine the relationship between the parameters and traditional concepts of item difficulty and discrimination. The three models considered were multidimensional extensions of the one-, two-, and…

In 2003, a trial National Assessment of Educational Progress (NAEP) mathematics assessment was administered in Spanish to public school students at grades 4 and 8 in Puerto Rico. Based on preliminary analyses of the 2003 data, changes were made in administration and translation procedures for the 2005 NAEP administration in Puerto Rico. This…

Because of the recent popularity of the three-parameter logistic model among the researchers who apply latent trait theory, it will be worthwhile to investigate the effect of noise accommodated in different models. In the present paper, four types of models on the dichotomous response level, Types A, B, C and D, are considered. Type A does not…

The Test for Inquiry Social Studies (TISS) is a 40-item multiple-choice instrument designed to measure the application of either inquiry social studies or higher-level cognitive thinking skills in fifth and sixth graders. The test can generally be completed in 40 to 50 minutes. This administrator's guide contains information on the reliability,…

The appropriate analysis of students' responses to an assessment is an essential step in improving the quality of the assessment itself as well as staff teaching and student learning. Many academics are unfamiliar with the formal processes used to analyze assessment results; the standard statistical methods associated with analyzing the validity…