RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two…

Student self-assessment is a central component of current conceptions of formative and classroom assessment. The research on self-assessment has focused on its efficacy in promoting both academic achievement and self-regulated learning, with little concern for issues of validity. Because reliability of testing is considered a sine qua non for the…

This book explores the Capability Approach (CA) as an alternative critical lens through which to regard early childhood education (ECE) curricula. The CA framework is a counter narrative to the narrow instrumentalism that reduces education to a mere process of academic skills acquisition for a future workplace. Primarily the book draws on the…

The Hughes et al. paper is critiqued generally and in specific areas. The weak nature of the authors' empirical work is discussed along with their enigmatic writing and vague and incorrect use of references, and their simultaneous use of sweeping statements of opinion and narrow analytical focus. This review examines the authors' errors…

This reaction article highlights areas of agreement and disagreement with the study conducted by Hughes, Rycus, Saunders-Adams, Hughes, and Hughes on the current state of research and practice in differential response (DR). Overall, we agree with several of the arguments put forth by Hughes et al. regarding the limitations of DR research and the…

In this article, the authors responded to nine commentaries by 17 contributors to their article, "Issues in Differential Response." The authors found that a majority of the respondents agreed with the major conclusions of "Issues in Differential Response." However, there were varying degrees of disagreement regarding the…

The purpose of this paper is to reflect on our efforts to balance consistency in our multi-year participatory action research study with the need to adapt our research protocol to what we are learning along the way. While both are important, we share several examples of how our flexibility and openness to adapt our protocol to our research…

Measurement of affective states in everyday life is of fundamental importance in many types of quality of life, health, and psychological research. Ecological momentary assessment (EMA) is the recognized method of choice, but the respondent burden can be high. The day reconstruction method (DRM) was developed by Kahneman and colleagues ("Science,"…

"Expanding Choice in Elementary and Secondary Education: A Report on Rethinking the Federal Role in Education" presents a seemingly egalitarian prescription for the federal government to expand school choice. An examination of the arguments and evidence for increasing choice, however, reveals at least three important shortcomings. First, the…

College students from the working class have interesting stories to tell about the meaning and operation of mobility through education. The author, herself a "working-class academic," explores some of the issues and dilemmas of uncovering and presenting these stories. Specifically, the author addresses: (1) the effects of interviewing those…

The Education Trust research report "Stuck Schools" suggests a framework for identifying chronically low-performing schools in need of turnaround. The study uses Maryland and Indiana to show that some low-performing schools make progress while others remain stagnant. The report has four serious problems of reliability and validity,…

Some concerns about the use and reporting of coefficient alpha are addressed. It is also shown that alpha is not a measure of homogeneity or unidimensionality. Four ways in which researchers use of alpha can convey inaccurate information or a lack of understanding are reviewed. (SLD)

Tests the hypotheses that overconfidence occurs when strength is high and weight is low and that underconfidence occurs when weight is high and strength is low in 4 experiments with 153 students. Discusses weighing evidence, overconfidence/underconfidence in intuitive judgments, and item difficulty's effect on overconfidence. Relates the…

Treatment integrity (also referred to as "treatment fidelity," "intervention integrity," and "procedural reliability") is an important methodological concerning both research and practice because treatment integrity data are essential to making valid conclusions regarding treatment outcomes. Despite its relationship to validity, treatment…

Cross-cultural adaptation of study instruments is a difficult, time-consuming, but arguably cost-effective process. If conducted properly, it has the advantage that the translated study instruments are accurate, easy to understand, accessible, and culturally appropriate to the target audience and produce reliable and valid data. This article…