The No Child Left Behind Act and the accountability movement in public education caused many states to develop criterion-referenced academic achievement tests. Scores from these tests are often used to make high stakes decisions. Even so, these tests typically do not receive independent psychometric scrutiny. We evaluated the 2005 Arkansas…

A strict equivalence presupposed by Kaiser and Michael to derive the coefficient of "domain validity" is defensible only as a biased approximation. But then, it is far from clear what psychometric significance this coefficient has in the first place. (Author)

This article attempts to present a novel application of a method of measuring accuracy for academic success predictors that could be used as a standard. This procedure is known as the receiver operating characteristic (ROC) curve, which comes from statistical decision techniques. The statistical prediction techniques provide predictor models and…

A framework, or classification scheme, is provided for displaying the spectrum of criterion-referenced tests. This framework illustrates that no single type of test can be identified as the absolute prototype criterion-referenced test. It is shown that over the past 115 years criterion-referenced testing has grown to be a many-faceted concept, its…

Teachers should be evaluated chiefly by the results they produce. Those results will not be properly detected through the use of norm-referenced achievement tests (NRT) due to the following major deficits of NRT: (1) the descriptions of what is measured by NRT are far too loose; (2) evaluative expectations are unclear; (3) the necessity for NRT to…

Research Findings: We examined the reliability and validity of the language, literacy, and mathematics domains of "Work Sampling for Head Start" (WSHS), an observational assessment designed for 3- and 4-year-olds. Participants included 112 children who were enrolled over a two-year period in Head Start and a number of other programs…

Issues related to the evaluation of instructional programs developed under the auspices of the Southwest Educational Development Laboratory are briefly discussed. The Laboratory develops criterion-referenced tests which form an integral part of each instructional program. The importance of examining the reliability and validity of these tests is…

Suggests that there are major issues involved in using criterion-referenced reading tests that are unclear or unresolved, including their place in the system of teaching reading, their validity, their reliability, and how cut-off scores are set and evaluated. (TJ)

There has been a recent emphasis on improving the academic performance of students with emotional and behavioral disorders (EBD). Improving the academic performance of students with EBD is especially important in the current accountability era in which there is much emphasis placed on performance of standardized tests. The purpose of this study…

A computer simulation study was conducted to investigate Subkoviak's index of reliability for criterion-referenced tests, called the coefficient of agreement. Results indicate that the index can be adequately estimated. (JKS)

Four different methods for estimating the proportions of testees properly classified as having mastered or not mastered test content are examined, using data from the Scholastic Aptitude Test. All four methods prove reasonably accurate and all show some bias under certain conditions. (JKS)

Derives formulas for the validity of predictor-criterion tests that hold for all test scores constructed according to the expected-value concept of true score. These more general formulas disclose some paradoxical properties of test validity under conditions where errors are correlated and have some implications for practical testing situations…

Criterion-referenced performance tests (CRPT) provide a realistic method for objectively measuring task proficiency against predetermined attainment standards. This article explains the procedures of constructing, validating, and scoring CRPTs and includes a checklist for a welding test. (JOW)