The application of the Mantel-Haenszel test statistic (and other popular DIF-detection methods) to determine DIF requires large samples, but test administrators often need to detect DIF with small samples. There is no universally agreed upon statistical approach for performing DIF analysis with small samples; hence there is substantial scope of…

Since the primary purpose of classical testing is to rank order examinees consistently, the absolute value of the true score has been relatively unimportant. However, the major purpose of criterion referenced testing is to estimate the true capabilities of examinees to perform specific tasks. Hence, the problems of true score determination assume…

An approximation procedure is proposed for the posterior means and standard deviation of the ability parameter in an item response model. The method is illustrated for the two-parameter logistic model using data from a 39-item American College Testing mathematics test. The effect of sample size is considered. (SLD)

Simultaneous estimation of many parameters can often be improved, sometimes dramatically so, if it is reasonable to consider one or more subsets of parameters as exchangeable members of corresponding populations. While each observation may provide limited information about the parameters it is modeled directly in terms of, it also contributes…

The predictive analysis approach to adaptive testing originated in the idea of statistical predictive analysis suggested by J. Aitchison and I.R. Dunsmore (1975). The adaptive testing model proposed is based on parameter-free predictive distribution. Aitchison and Dunsmore define statistical prediction analysis as the use of data obtained from an…

The major advantage of sequential classification, a technique for automatically classifying documents into previously selected categories, is that the entire document need not be processed before it is classified. This method assumes the availability of a priori categories, a selection of keywords representative of these categories, and the a…

The effect of prior information in Bayesian estimation is considered, mainly from the standpoint of objective testing. In the estimation of a parameter belonging to an individual, the prior information is, in most cases, the density function of the population to which the individual belongs. Bayesian estimation was compared with maximum likelihood…

A linear classification rule (used with equal covariance matrices) was contrasted with a quadratic rule (used with unequal covariance matrices) for accuracy of internal and external classification. The comparisons were made for seven situations which resulted from combining three data conditions (equal and unequal covariance matrices, minimal and…

The feasibility and effectiveness of a Bayesian method for estimating regressions in m groups is studied by application of the method to data from the Basic Research Service of The American College Testing Program. Evidence supports the belief that in many testing applications the collateral information obtained from each subset of m-1 colleges…

Educational assessment concerns inference about student knowledge, skills, and accomplishments. Test theory has evolved in part to address questions of weight, coverage, and import of data. Resulting concepts and techniques can be viewed as applications of more general principles for inference in the presence of uncertainty. (SLD)

False-positive and false-negative dicisions are the fundamental errors committed with a mastery test; yet the estimation of the likelihood of committing these errors has not been investigated. Accordingly, two methods of estimating the likelihood of committing these errors are described and then investigated using Monte Carlo techniques.…

The feasibility of using a distance measure, called the Bayesian distance, for automatic sequential document classification was studied. Results indicate that, by observing the variation of this distance measure as keywords are extracted sequentially from a document, the occurrence of noisy keywords may be detected. This property of the distance…

A decision-theoretic procedure is outlined which provides a framework within which Bayesian statistical methods can be employed with criterion-referenced tests to improve the quality of decision making in objectives based instructional programs. (Author/DEP)

The Computer-Assisted Data Analysis (CADA) Monitor is a set of conversational-language interactive computer programs that permit relatively inexperienced persons to perform relatively complex statistical data analysis. The Monitor leads the user through an analysis on a step-by-step basis providing the necessary direction, information, and…