This study examined the effects of certain item selection methods on the classification accuracy and classification consistency of criterion-referenced instruments. Three item response data sets, representing varying situations of instructional effectiveness, were simulated. Five methods of item selection were then applied to each data set for the…

Formulas are presented for decision reliability and for classification validity for mastery/nonmastery decisions based on criterion referenced tests. Two item parameters are used: the probability of a master answering an item correctly, and the probability of a nonmaster answering an item incorrectly. The theory explores the relationships of…

This study examined the appropriateness of the use of criterion referenced tests as a means of controlling an individual student's advancement to the next level of instruction or retention in the current unit in the PLATO Air Force Base Computer-Based Education project at Chanute. The study was also concerned with program evaluation, which…

A model is presented using examinee judgements in detecting ambiguous/misinterpreted items on teacher-made criterion-referenced tests. A computational example and guidelines for constructing domain categories and interpreting the indices are presented. (Author/PN)

The purpose of the investigation was to obtain some relationships among (1) test lengths, (2) shape of domain-score distributions, (3) advancement scores, and (4) several criterion-referenced test score reliability and validity indices. The study was conducted using computer simulation methods. The values of variables under study were set to be…

Reviewing psychometric and statistical developments in criterion- referenced testing, this paper presents six sections: uses of criterion- referenced test scores, reliability of criterion-referenced test scores, determination of test length, determination of cut-off scores, test development and validation, and summary and suggestions for further…

This study investigates two fixed parameter models for calibrating test items for an item bank under local school district control. Mathematics, reading, and language tests administered to students (Grades 7-9) provided the criterion information. Model I used calibrations from the Northwest Evaluation Association Item Bank to calibrate new items…

This material provides a discussion of the construction and analysis of tests prepared for classroom use by teachers. The initial discussion is concerned with the purposes of evaluation and the specification of objectives. This is followed by an examination of theoretical and practical considerations in planning a test. The material on test item…

The criterion-oriented problem of test bias and fairness in selection is compared to the construct-oriented problem of comparability of test scores in cross-cultural research. These problems are shown to have important similarities, and their studies may supplement each other. In a formulation of psychometric criteria for the comparability of test…

This symposium consists of five papers and presents some recent developments in adaptive testing which have applications to several military testing problems. The overview, by James R. McBride, defines adaptive testing and discusses some of its item selection and scoring strategies. Item response theory, or item characteristic curve theory, is…

Three approaches to the construction of achievement tests are compared: construct, operational, and empirical. The construct approach is based upon classical test theory and measures an abstract representation of the instructional objectives. The operational approach specifies instructional intent through instructional objectives, facet design,…

The Beginning Teacher Evaluation Study (BTES), Phase II, was a research project on effective teaching behavior--what teachers do that significantly affects what and how pupils learn. The purposes of Phase II were to (1) develop an assessment system for measuring teacher and pupil behaviors and other factors which could influence each of them and…