This paper reports on a set of three recent experiments utilizing large-scale speech models to assess the oral reading fluency (ORF) of students in Ghana. While ORF is a well-established measure of foundational literacy, assessing it typically requires one-on-one sessions between a student and a trained rater, a process that is time-consuming and…

The standard methodology for many large-scale assessments in education involves regressing latent variables on numerous contextual variables to estimate proficiency distributions. To reduce the number of contextual variables used in the regression and improve estimation, we propose and evaluate principal component analysis on the covariance matrix…

Errors that occasionally manifest in examination papers and other educational assessment instruments can threaten reliability and validity. For example, a multiple choice question could have two correct response options, or a geography question containing an inaccurate map could be unanswerable. In this paper we explore this oft-neglected element…

There have been significant advances in science and technology in recent years. Therefore, all countries need qualified people who can take on the challenges of life today and compete in the international arena. This has led countries to adopt new approaches to education. STEM education is one of the latest examples of those approaches. This study…

Item response theory (IRT) is a statistical paradigm for developing educational tests and assessing students. IRT, however, currently lacks an established graphical method for examining model fit for the three-parameter logistic model, the most flexible and popular IRT model in educational testing. A method is presented here to do this. The graph,…

English language learners (ELLs) and their teachers, schools, and communities face increasingly high-stakes consequences due to test score gaps between ELLs and non-ELLs. It is essential that the field of educational assessment continue to investigate the meaning of these test score gaps. This article discusses the findings of an exploratory study…

Evidence from recent research suggests that in the UK the public perception of errors in national examinations is that they are simply mistakes; events that are preventable. This perception predominates over the more sophisticated technical view that errors arise from many sources and create an inevitable variability in assessment outcomes. The…

This article characterizes and measures errors in the 2010 National Research Council (NRC) assessment of research-doctorate programs in geography. This article provides a conceptual model for data-based sources of uncertainty and reports on a quantitative assessment of NRC research data uncertainty for a particular geography doctoral program.…

Differential item functioning (DIF) analysis is a statistical technique used for ensuring the equity and fairness of educational assessments. This study formulates a new DIF analysis method using the information similarity index (ISI). ISI compares item information functions when data fits the Rasch model. Through simulations and an international…

In this policy brief, the author explores the problems with attainment measures when it comes to evaluating performance at the school level, and explores the best uses of value-added measures. These value-added measures, the author writes, are useful for sorting out-of-school influences from school influences or from teacher performance, giving…

This is a preliminary report on research into computational errors made by six- and seven-year-olds. (Author/MK)

With the theory that social and personal values influence the conduct of evaluation studies in education, the author discusses the impact of two major sources of such values--contextual factors, including political, social and organizational influences; and the terminology, models, and personal values of evaluators. Alternative purposes for an…

This report was designed to provide information on the effectiveness of the high school mathematics program in Hawaii, including identification of deficiency areas. The Stanford Achievement Test of Academic Skills (TASK) Level 1 was administered to 10,154 tenth-grade students in October 1983. In general, the students did not perform as well as the…

This report was designed to provide information on the effectiveness of the mathematics instructional program in grade 2 in Hawaii, including identification of deficiency areas. The Stanford Achievement Test, Primary Level II, was administered in May 1984 to 12,106 students in grade 2. In general, these students performed better than the national…

Rater errors (rater severity, halo effect, central tendency, and restriction of range) are described, and criteria are presented for evaluating rating quality based on a many-faceted Rasch (FACETS) model. Ratings of 264 compositions from the Eighth Grade Writing Test in Georgia by 15 raters illustrate the discussion. (SLD)