Three mode comparability studies were conducted on the following Saturday national ACT test dates: October 26, 2019, December 14, 2019, and February 8, 2020. The primary goal of these studies was to evaluate whether ACT scores exhibited mode effects between paper and online testing that would necessitate statistical adjustments to the online…

This study compares two methods of defining groups for the detection of differential item functioning (DIF): (a) pairwise comparisons and (b) composite group comparisons. We aim to emphasize and empirically support the notion that the choice of pairwise versus composite group definitions in DIF is a reflection of how one defines fairness in DIF…

In preparation for online administration of the ACT® test, ACT conducted studies to examine the comparability of scores between online and paper administrations, including a timing study in fall 2013, a mode comparability study in spring 2014, and a second mode comparability study in spring 2015. This report presents major findings from these…

Detecting unusual similarity in the item responses of a pair of examinees usually conditions on the pair's overall test performance (e.g., raw scores). Doing this, however, often requires assumptions about the invariance of other examinee pair characteristics. In this study, we examined the appropriateness of such assumptions about selected…

Composite scores are often formed from test scores on educational achievement test batteries to provide a single index of achievement over two or more content areas or two or more item types on that test. Composite scores are subject to measurement error, and as with scores on individual tests, the amount of error variability typically depends on…

This paper describes a multistep Enhanced Formative Assessment Program (EFAP) that features a Self-Regulated Learning (SRL) component. The program, which teaches students to become more effective learners, has been applied in a wide range of academic disciplines. In this paper we report on how the EFAP-SRL model can be applied to the area of…

ACT college test publisher provided scores. On average, non-homeschoolers performed better than homeschoolers, by about two items, out of sixty items, on the ACT mathematics test that was analyzed. This result may be due to the different teaching/learning media used in teaching each of the two groups, to different teacher/student interaction, or…

The primary concern or focus of a certification or licensure test is to obtain valid criterion-referenced information regarding a candidate's competence to practice. When the test is administered by computer, a valid pass/fail decision can be made with fewer items than an equivalent paper/pencil test by targeting items at the passing score and…

An optimization model is presented that allows test assemblers to control the shape of the observed-score distribution on a test for a population with a known ability distribution. An obvious application is for item response theory-based test assembly in programs where observed scores are reported and operational test forms are required to produce…

This study attempted to control differences in achievement when examining omitting tendencies of examinees. Test data of randomly sampled examinees (7 samples of 2,000 examinees each) from one national administration of the ACT Assessment were used. The number of responses omitted per examinee was examined over all examinees and over only those…

This report builds on and extends existent research on population invariance to new tests and issues. The authors lay the foundation for a deeper understanding of the use of population invariance measures in a wide variety of practical contexts. The invariance of linear, equipercentile and IRT equating methods are examined using data from five…