Diagnostic classification models (DCMs) are psychometric models that can be used to estimate the presence or absence of psychological traits, or proficiency on fine-grained skills. Critical to the use of any psychometric model in practice, including DCMs, is an evaluation of model fit. Traditionally, DCMs have been estimated with maximum…

Baeysian Network Meta-Analysis represents a rather unique challenge in assessing the quality of included studies. Prior efforts to synthesize computer based scaffolding are in need of a closer examination of research quality. This study examines two quality metrics for meta-analysis, study design, and risk of bias (Higgins et al., 2011). Lower…

Early prediction of student difficulty during long-duration learning activities allows a tutoring system to intervene by providing needed support, such as a hint, or by alerting an instructor. To be effective, these predictions must come early and be highly accurate, but such predictions are difficult for open-ended programming problems. In this…

The usage of data mining has dramatically increased over the past few years and the education sector is leveraging this field in order to analyze and gain intuitive knowledge in terms of the vast accumulated data within its confines. The primary objective of this study is to compare the results of different classification techniques such as Naïve…

A growing body of research suggests that accounting for student specific variability in educational data can improve modeling accuracy and may have implications for individualizing instruction. The Additive Factors Model (AFM), a logistic regression model used to fit educational data and discover/refine skill models of learning, contains a…

Fisher's two-group discriminant function has been generalized in two different ways for the case of three or more groups, leading to confusion in the literature. The precise functional relation between the two functions is derived, and the interpretation of the two functions is discussed. An example is provided. (Author/CTM)

The classification problem in educational testing is a decision problem. One must assign subjects to one of several available treatments on the basis of test scores, where the success of each treatment is measured by a different criterion. Examples of classification decisions include individualized instruction, counseling, and clinical settings.…

This paper describes a study, now in progress, of new methods for representing the sampling variability of Mantel-Haenszel differential item functioning (DIF) results, based on the system for categorizing the severity of DIF that is now in place at the Educational Testing Service. The methods, which involve a Bayesian elaboration of procedures…

A linear classification rule (used with equal covariance matrices) was contrasted with a quadratic rule (used with unequal covariance matrices) for accuracy of internal and external classification. The comparisons were made for seven situations which resulted from combining three data conditions (equal and unequal covariance matrices, minimal and…

A procedure for predicting categorical outcomes using categorical predictor variables was described by Moonan. This paper describes a related technique which uses prior probabilities, updated by joint likelihoods, as classification criteria. The procedure differs from Moonan's in that the outcome having the greatest posterior probability is…

This paper reviews recent research in the Netherlands on the application of decision theory to test-based decision making about personnel selection and student placement. The review is based on an earlier model proposed for the classification of decision problems, and emphasizes an empirical Bayesian framework. Classification decisions with…

The problems associated with misclassifying students when pass-fail decisions are based on test scores are discussed. One protection against misclassification is to set a confidence interval around the cutting score. Those whose scores fall above the interval are passed; those whose scores fall below the interval are failed; and those whose scores…