Decision theory is a useful method for assessing the effectiveness of the components of a course placement system. The effectiveness of placement tests or other variables in identifying underprepared students is described by the conditional probability of success in a standard course. Estimating the conditional probability of success is discussed.…

The present study examined and evaluated the application of linear policy-capturing models to the real-world decision task of graduate admissions. Utility of the policy-capturing models was great enough to be of practical significance, and least-squares weights showed no predictive advantage over equal weights. (Author/CTM)

The reliability and validity of a tailored testing procedure based on the simple logistic model was determined for an achievement test in statistics and measurement. The test was administered on a CRT terminal to students from graduate and undergraduate measurement courses. Equivalent form reliability over a one-week interval was found to be 0.595…

A model is presented using examinee judgements in detecting ambiguous/misinterpreted items on teacher-made criterion-referenced tests. A computational example and guidelines for constructing domain categories and interpreting the indices are presented. (Author/PN)

A multilevel variance component analysis from the pilot year of pretesting an instrument--the GENED--designed to present information about general education outcomes is presented, using data from about 11,000 college students. The analysis addresses the discriminant validity of the subtests and statistical issues in test construction. (SLD)

This study investigated the convergent validity of expert-system scores for four mathematical constructed-response item formats. A five-factor model was proposed comprised of four constructed-response format factors and a Graduate Record Examinations (GRE) General Test quantitative factor. Subjects were drawn from examinees taking a single form of…

Prior research in a single clinical training setting has shown Cason and Cason's (1981) simplified model of their performance rating theory can improve rating reliability and validity through statistical control of rater stringency error. Here, the model was applied to clinical performance ratings of 14 cohorts (about 250 students and 200 raters)…

Previous research has studied the effects of different methods of item option weighting on the reliability and concurrent and predictive validity of achievement tests. Increases in reliability are generally found, but with mixed results for validity. Several methods of producing option weights, (i.e., Guttman internal and external weights and…

Validity evidence for a new instrument, the Situation-Response Measure of Achievement Motivation for analyzing cross-situational consistency of achievement-related behavior, is presented in a study of 246 college students. Exploratory factor analysis indicates the presence of two factors, striving and apprehensiveness. (SLD)

Multidimensional item difficulty (MID) is proposed as a means of describing test items which measure more than one ability. With mathematical story problems, for instance, both mathematical and verbal skills are required to obtain a correct answer. The proposed measure of MID is based upon three general assumptions: (1) the probability of…

Tailored testing procedures for achievement testing were applied in a situation that failed to meet some of the specifications generally considered to be necessary for tailored testing. Discrepancies from the appropriate conditions included the use of small samples for calibrating items, and the use of an item pool that was not designed to be…

To improve the accuracy of college placement, Rasch scoring and person-fit statistics on the Comparative Guidance and Placement test (CGP) was compared to the traditional right-only scoring. Correlations were calculated between English and mathematics course grades and scores of 1,448 entering freshmen on the reading, writing, and mathematics…

The validity of American College Testing Program (ACT) test scores and self-reported high school grades for predicting grades in specific college freshman courses was studied. Specific course grades are typically used to place students in remedial, standard, or advanced classes. These placement decisions, in turn, have immediate implications for…

Three approaches to the construction of achievement tests are compared: construct, operational, and empirical. The construct approach is based upon classical test theory and measures an abstract representation of the instructional objectives. The operational approach specifies instructional intent through instructional objectives, facet design,…