We illustrate how to analyze complete generalizability theory (GT) designs using structural equation modeling software ("lavaan" in R), compare results to those obtained from numerous ANOVA-based packages, and apply those results in practical ways using data obtained from a large sample of respondents, who completed the Self-Perception…

By far, the most frequently used method of validating (the interpretation and use of) automated essay scores has been to compare them with scores awarded by human raters. Although this practice is questionable, human-machine agreement is still often regarded as the "gold standard." Our objective was to refine this model and apply it to…

This was a study of differential item functioning (DIF) for grades 4, 7, and 10 reading and mathematics items from state criterion-referenced tests. The tests were composed of multiple-choice and constructed-response items. Gender DIF was investigated using POLYSIBTEST and a Rasch procedure. The Rasch procedure flagged more items for DIF than did…

This study examines the effects of two test administration accommodations on the mathematics performance of students within the context of a large-scale statewide assessment. The two test administration accommodations were resource guides and calculators. A stratified random sample of schools was selected to represent the demographic…

This study addressed what standard-setting panelists think about when they make item performance estimates for a barely proficient student. This study extended previous studies by considering the factors that influenced panelists' decisions in an Angoff (1971)-based standard-setting study as a function of their item performance estimates.…

The modified caution index's use in identifying judges whose patterns of item judgment appear aberrant when compared with the pattern produced by the entire group (N=158) of judges was studied. Effects on test standards and passing rates of removing test standards of these judges were also assessed. (TJH)

The feasibility of combining criterion-referenced and norm-referenced tests into a school achievement test score to be used for school effectiveness classification was studied with 361 elementary schools across 2 years and across a subsample of 264 schools. Results support combining scores of different tests to measure school effectiveness. (SLD)

Psychometrically linking a state criterion-referenced test (CRT) and a norm-referenced test (NRT) to yield NRT information through the CRT was studied with samples of 1,500 to 3,000 elementary school students per subject and grade level in Connecticut. A CRT/NRT link can create a focused and coherent assessment system. (SLD)

Four main approaches to customized testing are described, and their resulting scores' valid uses and interpretations are discussed. Customized testing can yield valid normative and curriculum-specific information, although cautious application is needed to avoid misleading inferences about student achievement. (SLD)