Detection of differential item functioning (DIF) is needed in the development of tests to obtain useful items. The Mantel-Haenszel method and standardization are tools for DIF detection based on classical theory assumptions. The study was conducted to highlight the sensitivity and accuracy between the Mantel-Haenszel method and the standardization…

School STEM Culture--an aspect of culture within a school community--is defined as the beliefs, values, practices, and resources in STEM fields as perceived by students, parents, teachers, and administrators and counselors within a school. This study validates the STEM Culture Assessment Tool (STEM-CAT), an instrument intended to advance the use…

Method: Sixty English-speaking children, 7-14 years old with normal hearing, had a single study visit during which each test was administered twice. Changes on retest were summarized by within-subject standard deviation ( S[subscript w]), compared among tests, and compared with binomial model predictions. Correlates of variance were explored.…

Automatic item generation (AIG)--a means of leveraging technology to create large quantities of items--requires a minimum number of items to offset the sizable upfront investment (i.e., model development and technology deployment) in order to achieve cost savings. In this cost-benefit analysis, we estimated the cost of each step of AIG and manual…

The "Standards for Educational and Psychological Testing" recommend an argument-based approach to validation that involves a clear statement of the intended interpretation and use of test scores, the identification of the underlying assumptions and inferences in that statement--termed the interpretation/use argument, and gathering of…

This article provides a brief introduction to the set of four articles in the special issue. To provide a foundation for the issue, key terms are defined, a brief historical overview of validity is provided, and a description of several different validation approaches used in the issue are explained. Finally, the contribution of the articles to…

This article explores the amount of equating error at a passing score when equating scores from exams with small samples sizes. This article focuses on equating using classical test theory methods of Tucker linear, Levine linear, frequency estimation, and chained equipercentile equating. Both simulation and real data studies were used in the…

Sample representativeness is an important piece to consider when evaluating the quality of a screening assessment. If you are trying to determine whether or not the screening tool accurately measures children's skills, you want to ensure that the sample that is used to validate the tool is representative of your population of interest.

Reliability is the consistency of a set of scores that are designed to measure the same thing. Reliability is a statistical property of scores that must be demonstrated rather than assumed.

In data collected from virtual learning environments (VLEs), item response theory (IRT) models can be used to guide the ongoing measurement of student ability. However, such applications of IRT rely on unbiased item parameter estimates associated with test items in the VLE. Without formal piloting of the items, one can expect a large amount of…

This paper presents results of a score interpretation study for a computer adaptive mathematics assessment. The study purpose was to test the efficacy of item developers' alignment of items to Range Achievement-Level Descriptors (RALDs; Egan et al.) against the empirical achievement-level alignment of items to investigate the use of RALDs as the…

The contemporary era of learning-oriented assessment (LOA) demands teacher professional efforts to appropriately and accurately assess learners' attainment and use the assessment results for the enhancement of learning. In second/foreign language (L2) discipline, this has recently brought language assessment literacy (LAL) to the forefront,…

Burnout is psychological, physical, and emotional suffering that may affect students with low or inadequate resources to face stressful events at school. Although the existing instruments are used worldwide to assess school burnout risk, they show several flaws and mainly focus on the emotional facets of the syndrome. No previous studies have…

With the present study I investigated the sources of score variance and dependability in a local oral English proficiency test for potential international teaching assistants (ITAs) across four first language (L1) groups, and suggested alternative test designs. Using generalizability theory, I examined the relative importance of L1s (i.e., Indian,…

The study examines whether the average performance of the population with dyslexia in a working memory measure can be inferred dimensionally from the characteristics of the typical population. Specifically, we focused on Associative Phonological Working Memory (APWM), an ability that we predicted being impaired in dyslexia due to the relationship…