ERIC Number: ED482930
Record Type: Non-Journal
Publication Date: 2002-Nov
Pages: 45
Abstractor: N/A
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Modeling Conditional Probabilities in Complex Educational Assessments. CSE Technical Report.
Mislevy, Robert J.; Almond, Russell; Dibello, Lou; Jenkins, Frank; Steinberg, Linda; Yan, Duanli; Senturk, Deniz
An active area in psychometric research is coordinated task design and statistical analysis built around cognitive models. Compared with classical test theory and item response theory, there is often less information from observed data about the measurement-model parameters. On the other hand, there is more information from the grounding psychological theory, and the task designer's insights into which patterns of skills lead to which patterns of performance. The paper describes a Bayesian approach to modeling these situations, which uses experts' judgments to produce prior distributions for the conditional probabilities in a multivariate latent-variable model, and Monte Carlo Markov Chain estimation to refine the estimated. Task-design schemes and expert judgments are used in the first phase to structure the conditional probability tablethat is, conjunctive, compensatory, or disjunctive models, or combinations thereof. Machinery form graded-response item response theory is used to translate experts' judgments about task requirements into prior distributions for model parameters, which in turn imply values for all the conditional probabilities. Bayesian estimation methods are then used to update the distributions for the model parameters in response to observed data. The approach is illustrated with examples from the Biomass biology assessment prototype. (Contains 9 figures, 12 tables, and 25 references.) (Author/SLD)
Publication Type: Reports - Descriptive
Education Level: N/A
Audience: N/A
Language: English
Sponsor: Office of Educational Research and Improvement (ED), Washington, DC.
Authoring Institution: California Univ., Los Angeles. Center for the Study of Evaluation.; National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.
Grant or Contract Numbers: N/A
Author Affiliations: N/A