When comparing social science phenomena through a time perspective, absolute and relative difference (RD) are the two typical presentation formats used to communicate interpretations to the audience, while time distance (TD) is the least frequently used of such formats. This article argues that the chosen presentation format is extremely important…

Recent research on learning analytics dashboards has focused on designing user interfaces that offer various forms of "visualization guidance" (often referring to notions such as "data storytelling" or "narrative visualization") to teachers (e.g., emphasizing data points or trends with colour and adding annotations),…

The gold-standard for evaluating the effect of an educational intervention on student outcomes is running a randomized controlled trial (RCT). However, RCTs may often be small due to logistical considerations, and resulting treatment effect estimates may lack precision. Recent methods improve experimental precision by incorporating information…

The gold-standard for evaluating the effect of an educational intervention on student outcomes is running a randomized controlled trial (RCT). However, RCTs may often be small due to logistical considerations, and resulting treatment effect estimates may lack precision. Recent methods improve experimental precision by incorporating information…

Qualitative comparative analysis (QCA) is gaining ground in evaluation circles, but the number of applications is still limited. In this article, we consider the challenges that can emerge during a QCA evaluation by drawing on our experience of conducting one in the field of development cooperation. For each stage of the evaluation process, we…

This applied research methods report is a guide for state and local education agency policymakers and their analysts who are interested in studying teacher mobility and retention. This report is the second in a two-part set and builds on the foundational information in report 1. This report presents guidance on how to interpret differences in…

K-12 students are expected to acquire competence in data display as part of developing statistical literacy. To support research, assessment design, and instruction, we developed a hypothesized learning progression (LP) using existing empirical literature in the fields of mathematics and statistics education. The data display LP posits a…

There are no absolute standards regarding what teaching evaluation ratings are satisfactory. It is also problematic to compare teaching evaluation ratings with the average or with a cutoff number to determine whether they are adequate. In this paper, we use average and standard deviation charts (X[overbar]-S charts), which are based on the theory…

Given growing interest in K-12 data and data science education, new approaches are needed to help students develop robust understandings of and familiarity with data. The model of the "quantified self"--in which data about one's own activities are collected and made into objects of study--provides inspiration for one such approach. By…

Hierarchical linear modeling (HLM) is a useful tool when analyzing data collected from groups. There are many decisions to be made when constructing and estimating a model in HLM including which estimation technique to use. Three of the estimation techniques available when analyzing data with HLM are maximum likelihood, restricted maximum…

Developmental theory has long emphasized the importance of linking perception, cognition, and action. Techniques designed to record the spatial and temporal characteristics of hand movements (i.e., "manual dynamics") present new opportunities to study the nature of these links across development by providing a window into how perceptual,…

The 2017 NAPLAN results were followed by political and media attention focusing on a decline in achievement in writing. This article analyses NAPLAN data to identify the locus of the decline. Contrary to media reports in which it was suggested Year 3 was problematic, the article systematically compares NAPLAN results between 2011 and 2017 to…

This study aims to examine the effectiveness of metacognitive scaffolding in different inquiry tasks related to optics. Two high school classes participated in this study. One class, the treatment group (n = 33), which integrated metacognitive prompts into the simulation-based inquiry, was compared to the other class, the control group (n = 34),…

New technologies have completely altered the ways that citizens can access data. Indeed, emerging online data sources give citizens access to an enormous amount of numerical information that provides new sorts of evidence used to influence public opinion. In this new environment, two trends have had a significant impact on our increasingly…

We present considerations for validity when a population outside of a normed sample is assessed and those data are interpreted. Using a career group counseling example exploring life satisfaction changes as evidenced by the Quality of Life Inventory (Frisch, 1994), we showcase qualitative and quantitative approaches to explore how normative data…