The Social Shapes Test (SST) is a measure of social intelligence which does not use human faces or rely on extensive verbal ability. The SST has shown promising validity among adults without autism spectrum disorder (ASD), but it is uncertain whether it is suitable for adults with ASD. We find measurement invariance between adults with (n = 229)…

Problem decomposition into sub-problems or subgoals and recomposition of the solutions to the subgoals into one complete solution is a common strategy to reduce difficulties in structured problem solving. In this study, we use a datadriven graph-mining-based method to decompose historical student solutions of logic-proof problems into Chunks. We…

This article reviews the evidence on the extent to which experts' perceptions of item difficulties, captured using comparative judgement, can predict empirical item difficulties. This evidence is drawn from existing published studies on this topic and also from statistical analysis of data held by Cambridge Assessment. Having reviewed the…

Reading comprehension is one of the most valuable academic skills taught in school. Selecting the appropriate assessment instrument to ensure early identification and intervention is important as there is an amalgam of cognitive abilities and academic skills involved in reading comprehension. The GORT-5 is the most recent edition of a test that…

Testing English writing skills could be multi-dimensional; thus, the study aimed to compare students' writing scores calculated according to Classical Test Theory (CTT) and Multi-Facet Rasch Model (MFRM). The research was carried out in 2019 with 100 university students studying at a foreign language preparatory class and four experienced…

In Comparative Judgement (CJ) exercises, examiners are asked to look at a selection of candidate scripts (with marks removed) and order them in terms of which they believe display the best quality. By including scripts from different examination sessions, the results of these exercises can be used to help with maintaining standards. Results from…

Assessing students' (mis)conceptions is a challenging task for teachers as well as for researchers. While individual assessment, for example through interviews, can provide deep insights into students' thinking, this is very time-consuming and therefore not feasible for whole classes or even larger settings. For those settings, automatically…

Background: Engineers operate in an increasingly global environment, making it important that engineering students develop global engineering competency to prepare them for success in the workplace. To understand this learning, we need assessment approaches that go beyond traditional self-report surveys. A previous study (Jesiek et al.,…

As a research field geared toward understanding and improving learning, Learning Analytics (LA) must be able to provide empirical support for causal claims. However, as a highly applied field, tightly controlled randomized experiments are not always feasible nor desirable. Instead, researchers often rely on observational data, based on which they…

The purpose of this study was to investigate the Type I Error findings and power rates of the methods used to determine dimensionality in unidimensional and bidimensional psychological constructs for various conditions (characteristic of the distribution, sample size, length of the test, and interdimensional correlation) and to examine the joint…

Despite the increasing demand for diagnostic information, observed subscores have been often reported to lack adequate psychometric qualities such as reliability, distinctiveness, and validity. Therefore, several statistical techniques based on CTT and IRT frameworks have been proposed to improve the quality of subscores. More recently, DCM has…

Learning analytics researchers have been diligently integrating trace data to study Self-Regulated Learning (SRL). Compared to traditionally used survey data, trace data, such as log or clickstream data designed and interpreted to understand a certain SRL construct, are considered to be more effective in capturing dynamic SRL as fine-grained…

School districts increasingly gauge school quality with surveys that ask about school climate and student engagement. We use data from New York City's middle and high schools to compare the long-run predictive validity of surveys with that of conventional test score value-added models (VAMs). Our analysis leverages the New York school match, which…

Organization, time management, and planning (OTMP) skills are behavioral manifestations of executive functioning linked to academic outcomes. Interventions to improve OTMP skills have shown favorable outcomes. The Children's Organizational Skills Scale parent and teacher forms (COSS-P, COSS-T) are widely used for assessing OTMP skills, but there…

Organization, time management, and planning (OTMP) skills are behavioral manifestations of executive functioning linked to academic outcomes. Interventions to improve OTMP skills have shown favorable outcomes. The Children's Organizational Skills Scale parent and teacher forms (COSS-P, COSS-T) are widely used for assessing OTMP skills, but there…