Critical to precise quantitative research is reliability estimation. Researchers have limited tools, however, to assess the reliability of evolving instruments. Consequently, cursory assessment is typical and in-depth evaluation is rare. This paper presents a rationale for and description of PIAS, a computerized instrument analysis system. PIAS…

The Test Analysis Program (TAP) is a comprehensive, flexible computer system designed to score and to analyze objective educational tests. The goals of the designers were to construct a program which would be user-oriented, flexible, and clear in structure and in output. (Author/JKS)

The results of data analysis based on the multiplicative binomial model (MLTBIN) are compared with dichotomous data analyzed by using the simple logistic model (SLM). Specifically, the paper focuses on the issues of fit to these Rasch Models and estimates of item difficulty. The data indicate that the SLM and MLTBIN are not comparable in the…

Various models have been proposed for analyzing dichotomous test or questionnaire items which were constructed to reflect an assumed underlying structure (e.g., hierarchical). This paper shows that many such models are special cases of latent class analysis and discusses a currently available computer program to analyze them. (Author/JKS)

Since fall 1976, an undergraduate measurement class has utilized a 32K microcomputer programmed in APL to present replicate sets of questions randomly selected from 80-question item pools representing each of three domains of knowledge based on Sax's Principles of Educational Measurement and Evaluation. Multiple choice questions are presented on a…

The effect of stratified sampling of items on the estimation of test score distribution parameters by multiple matrix sampling was studied. Item difficulty and/or interitem correlations were the bases of stratification. Various item iniverses were created by computer simulation and sampled according to several plans. The results indicate that…

An interactive test analysis system was developed which interfaces a 3M DATRONICS system with a XEROX Sigma 9 computer. The computer programs were written in A Programming Language (APL). The current implementation of the program is characterized by its capability to: read responses from a DATRONIC answer sheet; allow the faculty member the option…

This paper will discuss the results of a series of computer simulations comparing the Rasch logistic model to a series of models departing to various degrees from its assumption of equal discrimination power for all items. The results have implications for test construction and test scoring, indicating how closely the conventional raw score…

The appropriateness of the use of the standardized residual (SR) to assess congruence between sample test item responses and the one parameter latent trait (Rasch) item characteristic curve is investigated. Latent trait theory is reviewed, as well as theory of the SR, the apparent error in calculating the expected distribution of the SR, and…

Procedures are presented for equating simultaneously several tests which have been calibrated by the Rasch Model. Three multiple test equating designs are described. A Full Matrix Design equates each test to all others. A Chain Design links tests sequentially. A Vector Design equates one test to each of the other tests. For each design, the Rasch…

False-positive and false-negative dicisions are the fundamental errors committed with a mastery test; yet the estimation of the likelihood of committing these errors has not been investigated. Accordingly, two methods of estimating the likelihood of committing these errors are described and then investigated using Monte Carlo techniques.…

Item characteristic curve (ICC) theory describes the relationship between the ability of individuals and the probability of their answering a test question correctly; it is useful in estimating test scores, equating the scores of various tests, and scoring responses during adaptive testing. A simulation study of the effectiveness of the following…

A method for developing statistically parallel tests based on the analysis of unique item variance was developed. A test population of 907 basic airmen trainees were required to estimate the angle at which an object in a photograph was viewed, selecting from eight possibilities. A FORTRAN program known as VARSEL was used to rank all the test items…

Two methods of item selection were used to select sets of 40 items from a 50-item verbal analogies test, and the resulting item sets were compared for relative efficiency. The BICAL program was used to select the 40 items having the best mean square fit to the one parameter logistic (Rasch) model. The LOGIST program was used to select the 40 items…

PRIME is a library of 44 batch-oriented computer routines: 20 major package programs, which use 12 statistical utility routines, and 12 other utility routines for input/output and data manipulation. This manual contains a general description of data preparation and coding, standard control cards, input deck arrangement, standard options, and…