This guide offers a comprehensive handbook to scientific research methodology and experimental design, specifically for novice MA and PhD researchers in Education and Language Learning (TESOL/TEFL). It establishes scientific research as a systematic, objective inquiry focused on identifying cause-and-effect relationships through empirical data.…

Education systems are increasingly creating rich, longitudinal data sets with frequent, and even real-time, data updates of many student measures, including daily attendance, homework submissions, and exam scores. These data sets provide an opportunity for district and school staff members to move beyond an indicators-based approach and instead…

Background: Youth programs often rely on self-reported data without clear evidence as to the accuracy of these reports. Although the validity of self-reporting has been confirmed among some high school and college age students, one area that is absent from extant literature is a serious investigation among younger children. Moreover, there is…

As lines between research paradigms continue to blur with the ever-increasing popularity of mixed methods research, there are useful, and occasionally oxymoronic, opportunities for educational researchers to juxtapose tools from opposing methods. The gold standard is just not possible in so much of what we do with small-scale research, nor is it…

In this article, the author challenges the validity and usefulness of the concept of "paradigm," as this term has been used in the social sciences generally, and specifically in the debates over research methods. He emphasizes that in criticizing what he sees as the misuse of the paradigm concept, he is not arguing for dismissing or ignoring…

In this paper we described how we mixed research approaches in a Randomized Control Trial (RCT) of a school principal professional development program. Using examples from our study we illustrate how combining qualitative and quantitative data can address some key challenges from validating instruments and measures of mediator variables to…

In Shadish (2010) and West and Thoemmes (2010), the authors contrasted 2 approaches to causality. The first originated in the psychology literature and is associated with work by Campbell (e.g., Shadish, Cook, & Campbell, 2002), and the second has its roots in the statistics literature and is associated with work by Rubin (e.g., Rubin, 2006). In…

This report summarizes the complex research literature on quantitative methods for assessing how impacts of educational interventions on instructional practices and student learning differ across students, educators, and schools. It also provides technical guidance about the use and interpretation of these methods. The research topics addressed…

Districts now have access to a wealth of new information that can help target students with appropriate supports and bring focus and coherence to college readiness efforts. However, the abundance of data has brought its own challenges. Schools and school systems are often overwhelmed with the amount of data available. The capacity of districts to…

Some researchers have argued that discrepant broad index scores invalidate IQs, but others have questioned the fundamental logic of that argument. To resolve this debate, the present study used a nationally representative sample of children (N = 1,200) who were matched individually for IQ. Children with significantly uneven broad index score…

A sequential design utilizing identical samples was used to classify mixed methods studies via a two-dimensional model, wherein sampling designs were grouped according to the time orientation of each study's components and the relationship of the qualitative and quantitative samples. A quantitative analysis of 121 studies representing nine fields…

This article describes the strengths and weaknesses of different models of classification (primarily monothetic and polythetic) and then focuses on cluster analysis as a method of pursuing a polythetic model. Design features that need to be incorporated in a cluster analytic study are discussed in relation to theory formulation, internal validity,…

Challenges the typical treatment of causal effects in longitudinal data, arguing that models should be conceptualized and tested in ways that directly reflect prior assumptions as to the trait- or state-like nature of the variables. Examples demonstrate that meaningful longitudinal studies of state variables can be conducted without assuming their…

This report provides preliminary technical information on a method for analyzing the factors involved in the closing of elementary schools. Included is a presentation of data and a brief discussion bearing on descriptive statistics, reliability, and validity. An intercorrelation matrix is also examined. The method employs 9 factors that have a…

A review of 158 applied behavior analysis studies with children as subjects, published in the "Journal of Applied Behavior Analysis" between 1980 and 1990, found that (1) 16% measured the accuracy of independent variable implementation, and (2) two-thirds did not operationally define components of the independent variable. Specific recommendations…