Performance assessments rely on the expert judgment of raters for the measurement of the quality of responses, and raters unavoidably introduce error in the scoring process. Defined as the tendency of a rater to assign higher or lower ratings, on average, than those assigned by other raters, even after accounting for differences in examinee…

In the psycholinguistic literature, reaction times and accuracy can be analyzed separately using mixed (logistic) effects models with crossed random effects for item and subject. Given the potential correlation between these two outcomes, a joint model for the reaction time and accuracy may provide further insight. In this paper, a Bayesian…

In a provocative and influential paper, Jesse Rothstein (2010) finds that standard value-added models (VAMs) suggest implausible future teacher effects on past student achievement, a finding that obviously cannot be viewed as causal. This is the basis of a falsification test (the Rothstein falsification test) that appears to indicate bias in VAM…

Estimation problems associated with the correlated-trait correlated-method (CTCM) parameterization of a multitrait-multimethod (MTMM) matrix are widely documented: the model often fails to converge; even when convergence is achieved, one or more of the parameter estimates are outside the admissible parameter space. In this study, the authors…

University career counsellors invest a lot of time and effort in the development of the skills and knowledge associated with self-reliance in job interviews. The resource-intensive nature of this approach needs to be reconsidered in the context of overall careers service delivery. More specifically, the key barriers to achieving a competent degree…

The Indian Institute of Science is proud to host the fully in-person sixteenth iteration of the International Conference on Educational Data Mining (EDM) during July 11-14, 2023. EDM is the annual flagship conference of the International Educational Data Mining Society. The theme of this year's conference is "Educational data mining for…

In this study we present the structure and implementation of a model-based inquiry teaching-learning sequence (TLS) integrating expressive, experimental and exploratory modelling pedagogies in a cyclic manner, with the aim of enhancing primary education student teachers' epistemological beliefs about the aspects, nature, purpose and change of…

This article investigates the use of teacher value-added estimates to assess the distribution of effective teaching across students of varying socioeconomic disadvantage in the presence of classroom composition effects. We examine, via simulations, how accurately commonly used teacher value-added estimators recover the rank correlation between…

This paper reports on a project in which students designed, constructed and tested a model of an existing early warning system with simulation of debris flow in a context of a landslide. Students also assessed rural community members' knowledge of this system and subsequently taught them to estimate the time needed for evacuation of the community…

Effective design of training-related games (games for training and/or assessment) requires synergy between the mechanisms for delivering instructional content and the mechanisms for learning game play and game functionality (Becker, 2006). The learning domain must be embedded as a core game mechanic: that is, the game cannot be advanced or won…

This paper examines the retention of team cognition with changes in team membership. Hypotheses are developed from shared cognition and interactive team cognition theories. We report a study of the effects of Short (3-6 weeks) versus Long (10-13 weeks) retention intervals and change (Mixed) versus no change (Intact) in team membership during the…

Sensitivity analysis is the study of how the different input variations of a mathematical model influence the variability of its output. In this paper, we review the principle of global and local sensitivity analyses of a complex black-box system. A simulated case of application is given at the end of this paper to compare both approaches.…

A new metric, Word Maturity, estimates the development by individual students of knowledge of every word in a large corpus. The metric is constructed by Latent Semantic Analysis modeling of word knowledge as a function of the reading that a simulated learner has done and is calibrated by its developing closeness in information content to that of a…

Different random or purposive allocations of items to parcels within a single sample are thought not to alter structural parameter estimates as long as items are unidimensional and congeneric. If, additionally, numbers of items per parcel and parcels per factor are held fixed across allocations, different allocations of items to parcels within a…

Students conducted inquiry using simulations within a rich learning environment for 4 science topics. By applying educational data mining to students' log data, assessment metrics were generated for two key inquiry skills, testing stated hypotheses and designing controlled experiments. Three models were then developed to analyze the transfer of…