Conversations in collaborative problem-solving activities can be used to probe the collaboration skills of the team members. Annotating the conversations into different collaboration skills by human raters is laborious and time consuming. In this report, we report our work on developing an automated annotation system, CPS-rater, for conversational…

Collaborative problem solving (CPS) is a critical competency in a variety of contexts, including the workplace, school, and home. However, only recently have assessment and curriculum reformers begun to focus to a greater extent on the acquisition and development of CPS skill. One of the major challenges in psychometric modeling of CPS is…

This study discusses the development of a basic electronics knowledge (BEK) assessment as a pretest activity for undergraduate students in engineering and related fields. The 28 BEK items represent 12 key concepts, including properties of serial circuits, knowledge of electrical laws (e.g., Kirchhoff 's and Ohm's laws), and properties of digital…

Simulation or game-based assessments produce outcome data and process data. In this article, some statistical models that can potentially be used to analyze data from simulation or game-based assessments are introduced. Specifically, cognitive diagnostic models that can be used to estimate latent skills from outcome data so as to scale these…

To reveal what is being learned during the gaming experience, this report proposes an approach for embedding assessments in immersive games, drawing on recent advances in assessment design. Key to this approach are formative assessment to guide instructional experiences and evidence-centered design to systematically analyze the assessment argument…