This paper examines the development and evaluation of criterion-referenced measures, and elaborates on the distinction between them and norm-referenced measures. The concept of sensitivity is introduced as an appropriate method for evaluating such measures, and a sensitivity index is proposed. The traditional model for the response of a subject to…

This book offers specific how-to advice to college faculty on every stage of the testing process, including planning the test and classifying objectives to be measured, ensuring the validity and reliability of the test, and grading in such a way as to arrive at fair grades based on relevant data. The book examines the strengths and weaknesses of…

In order to look for evidence of possible racial bias in six of the Navy tests for the Enlisted Advancement System, statistical data for a group of black personnel were compared with data for a selected matching group of whites. The first step was to compute item difficulty indices (the percent passing each item) for the black group, for the…

The technology of transforming sentences from prose instruction into test questions was examined by comparing two methods of selecting sentences (keyword vs. rare singleton), two types of question words (nouns vs. adjectives), and two foil construction methods (writer's choice vs. algorithmic). Four item writers created items using each…

To explain the discrepancy between median scores on the 1976 administration of the California Achievement Tests (CAT) and the Sequential Tests of Educational Progress (STEP) in the Austin Independent School District (AISD), ten technical variables typical of achievement tests were considered as explanations. (1) The STEP may measure different…

Twenty multiple-choice vocabulary items and 20 free-response vocabulary items were administered to 660 college students. The free-response items consisted of the stem words of the multiple-choice items. Testees were asked to respond to the free-response items with synonyms. A computer algorithm was developed to transform the numerous…

This volume contains tables of item analysis results obtained by following procedures associated with the Rasch Model for those reading tests used in the Anchor Test Study. Appendix I gives the test names and their corresponding analysis code numbers. Section I (Basic Item Analyses) presents data for the item analysis of each test in a two part…

This study investigated the appropriateness of Item Response Theory (IRT) models to analyze language test data. In a university second language institute, results of typical versions of tests of French as a Second Language (n=831 examinees) and of English as a Second Language (n=617 examinees) were subjected to non-linear factor analysis in…

The existence of criterion-referenced (CR) measurement is questioned in this paper. Despite beliefs that differences exist between two alternative forms of measurement, CR and Norm Referenced (NR), an analysis of philosophical and psychological descriptions of measurement, as well as a growing number of empirical studies, reveal that the common…

This research investigated the use of the Rasch simple logistic model in item and test calibration. Tests employing word, picture, symbol, and number analogies were administered to college students, high school students, civil service clerical employees, and clients of the Minnesota Division of Vocational Rehabilitation. The results suggest that…

Information provided by typical and improved conventional classroom achievement tests was compared with information provided by an adaptive test covering the same subject matter. Both tests were administered to over 700 college students in a general biology course. Using the same scoring method, adaptive testing was found to yield substantially…

An investigation was conducted to determine whether the Parental Attitude and Behavior Inventory (PABI) Form III, a lengthy self-report instrument (577 items each for both parents) for assessing parents' attitudes and behavior toward their children and each other, could be shortened to a more feasible length. This terminal report summarizes the…

The No Child Left Behind Act has brought great attention to the high school graduation rate as one of the mandatory accountability measures for public school systems. However, there is no consensus on how to calculate the high school graduation rate given the lack of longitudinal databases that track individual students. This study reviews…

With the continual progress of computer technologies, computer automated scoring (CAS) has become a popular tool for evaluating writing assessments. Research of applications of these methodologies to new types of performance assessments is still emerging. While research has generally shown a high agreement of CAS system generated scores with those…

In July 2001, the Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA) agreed to the development of assessment instruments and key performance measures for reporting on student skills, knowledge and understandings in primary science. It directed the newly established Performance Measurement and Reporting Taskforce…