Whenever classification decisions are made on educational tests, such as pass/fail, or basic, proficient, or advanced, the consistency and accuracy of those decisions should be estimated and reported. Methods for estimating the reliability of classification decisions made on the basis of educational tests are well-established (e.g., Rudner, 2001;…

A national survey of physics faculty was conducted to investigate the prevalence and nature of computational instruction in physics courses across the United States. 1246 faculty from 357 unique institutions responded to the survey. The results suggest that more faculty have some form of computational teaching experience than a decade ago, but it…

Parallel analysis (PA) assesses the number of factors in exploratory factor analysis. Traditionally PA compares the eigenvalues for a sample correlation matrix with the eigenvalues for correlation matrices for 100 comparison datasets generated such that the variables are independent, but this approach uses the wrong reference distribution. The…

Contrast analysis is a relatively simple but effective statistical method for testing theoretical predictions about differences between group means against the empirical data. Despite its advantages, contrast analysis is hardly used to date, perhaps because it is not implemented in a convenient manner in many statistical software packages. This…

The purpose of this study is to assess the impact of aberrant responses on the estimation accuracy in forced-choice format assessments. To that end, a wide range of aberrant response behaviors (e.g., fake, random, or mechanical responses) affecting upward of 20%--30% of the responses was manipulated under the multi-unidimensional pairwise…

Purpose: The purpose of this paper is to present computational algorithms as potential bearers of biases and also tools for reflection, and to advance strategies for engaging with them in education. Design/methodology/approach: Three example games (Dwarf Fortress, RimWorld, FreeCiv) are provided for introducing algorithms as meaningful and…

This methodological paper examines current conceptions of reliability in chemistry education research (CER) and provides recommendations for moving beyond the current reliance on reporting coefficient alpha (a) as reliability evidence without regard to its appropriateness for the research context. To help foster a better understanding of…

In many prekindergarten classes, children begin the day by singing a song as part of their opening routine. In reflecting on opening routines in public prekindergarten classrooms across Michigan, the authors have been struck by how often small changes in teachers' routines or materials could enrich the mathematical engagement of children. The goal…

Design-based methods have recently been developed as a way to analyze randomized controlled trial (RCT) data for designs with a single treatment and control group. This article builds on this framework to develop design-based estimators for evaluations with multiple research groups. Results are provided for a wide range of designs used in…

The aim of this article is to provide an overview of the opportunities for developing computational thinking in young learners. It includes a review of empirical studies on the educational technologies used to develop computational thinking in young learners, and analyses and descriptions of a selection of commercially available technologies for…

Design-based methods have recently been developed as a way to analyze data from impact evaluations of interventions, programs, and policies (Freedman, 2008; Lin, 2013; Imbens and Rubin, 2015; Schochet, 2013, 2016; Yang and Tsiatis, 2001). The non-parametric estimators are derived using the building blocks of experimental designs with minimal…

In the core of computational thinking (CT) practices is problem-solving. However, there is little research on connecting CT with problem solving processes. We developed a computer-based assessment on CT that can log students' problem-solving processes and administered it to a group of fifth graders as a pre- and post- measure of a robotics…

In Huggins-Manley & Han (2017), it was shown that WLSMV global model fit indices used in structural equating modeling practice are sensitive to person parameter estimate RMSE and item difficulty parameter estimate RMSE that results from local dependence in 2-PL IRT models, particularly when conditioning on number of test items and sample size.…

The use of learner modelling approaches is critical for providing adaptive support in educational computer games, with predictive learner modelling being among the key approaches. While adaptive supports have been shown to improve the effectiveness of educational games, improperly customized support can have negative effects on learning outcomes.…

The aim of this study was to investigate current trends in research of math anxiety (MA) through bibliometric perspective. Three main clusters were formed based on author keywords: cognitive correlates (working memory, attention, numerical cognition, mental arithmetic), psychological factors and effects (self-concept and self-efficacy, motivation,…