Identifying effective literacy interventions for teachers of very young children is important, but it is not easy. In this case study, we share our experiences implementing a quasi-experimental evaluation of an early literacy program. We describe the design and context of the study and discuss three key issues that we encountered with data…

Although social scientists devote considerable effort to mitigating measurement error during data collection, they often ignore the issue during data analysis. And although many statistical methods have been proposed for reducing measurement error-induced biases, few have been widely used because of implausible assumptions, high levels of model…

Evolution of technology and its tremendous use in education has changed the ways of educational services in higher education around the world. There is worldwide access to higher education through virtual learning environments. This is a new avenue for 21st century education and within a short time, it has been able to establish new culture of…

This paper examines sources of potential bias in systematic reviews and meta-analyses which can distort their findings, leading to problems with interpretation and application by practitioners and policymakers. It follows from an article that was published in the "Canadian Journal of Communication" in 1990, "Integrating Research…

Purpose: This policy brief, the third in the AERI-NEPC eBrief series "Understanding validity issues around the world", discusses validity issues surrounding International Large Scale Assessment (ILSA) programs. ILSA programs, such as the well-known Programme of International Student Assessment (PISA) and the Trends in International…

In the current article, we consider the influential position of the Programme for International Student Assessment (PISA) and discuss several methodological areas that demonstrate the need for caution when using and interpreting PISA results. We motivate our argument by briefly describing the program's increased influence in educational policy…

The use of "radar charts" is an increasingly popular way to present spatial data in a visually interesting format. Some authors recommend using "filled radar charts" to compare the performance of observational units. Filled radar charts are not appropriate for such comparisons because the size of the area within the polygon is not invariant to the…

Overlap-based measures are increasingly applied in the synthesis of single-subject research. This article considers two criticisms of overlap-based metrics, specifically that they do not measure magnitude of effect and do not adequately correspond with visual analysis. It is argued that these criticisms are based on fundamental misconceptions…

In 1916 Robert Frost published his famous poem, "The Road Not Taken," in which he muses about what might have been had he chosen a different path, made a different choice. While counterfactual arguments in general can often lead to vacuous nowheres, frequently in statistics the data that are not presented actually exist, in a sense,…

Background: Our ability to look at structure and function of a living brain has increased exponentially since the early 1970s. Many studies of developmental disorders now routinely include a brain imaging or electrophysiological component. Amid current enthusiasm for applications of neuroscience to educational interventions, we need to pause to…

Cross-language qualitative research in education continues to increase. However, there has been inadequate discussion in the literature concerning the translation process that ensures research trustworthiness applicable for bilingual researchers. Informed by the literature on evaluation criteria for qualitative data translation, this paper…

The authors of the study "The Long-Term Impact of Teachers" claim that their study shows that increases in teacher value-added lead to significant and lasting increases in test scores and significant increases in income that will last throughout adulthood. Instead, I show that these claims are false because they are contradicted by the…

Our target article (Robinson-Cimpian, Lubienski, Ganley, & Copur-Gencturk, 2014) used nationally representative data to examine the development of gender gaps in math achievement. We found that when boys and girls demonstrate equivalent math test performance and are perceived by their teachers to be equally well behaved and engaged with the…

This article presents a reflection on an aspect of research methodology, particularly on the interpretation strategy of data from a Science and Indigenous Knowledge Systems Project (SIKSP) in a South African university. The data interpretation problem arose while we were analysing the effects of a series of SIKSP-based workshops on the views of a…

Shortly after we published "Addressing Flawed Research in Developmental Education" (2012) in the "Journal of Developmental Education," Thomas Bailey, Shanna Smith Jaggars, and Judith Scott- Clayton from the Community College Research Center (CCRC) wrote a response rebutting several of our claims. Though their response corrects…