Survey research in education is a popular quantitative research design and is used to determine opinions, attitudes, behavior, habits, and perceptions on a wide variety of topics. While many people are familiar with survey methods, it is often used in a 'quick and dirty' way to get information. This case aims to describe some of the challenges of…

Elliot et al. (2010) reported multiple experimental findings that the color red modified women's ratings of attractiveness, sexual desirability, and status of a photographed man. An analysis of the reported statistics of these studies indicates that the experiments lack sufficient power to support these claims. Given the power of the experiments,…

Regression discontinuity design (RD) has been widely used to produce reliable causal estimates. Researchers have validated the accuracy of RD design using within study comparisons (Cook, Shadish & Wong, 2008; Cook & Steiner, 2010; Shadish et al, 2011). Within study comparisons examines the validity of a quasi-experiment by comparing its…

We examine the performance of planned missing (PM) designs for correlated latent growth curve models. Using simulated data from a model where latent growth curves are fitted to two constructs over five time points, we apply three kinds of planned missingness. The first is item-level planned missingness using a three-form design at each wave such…

Critically examining common statistical approaches and their strengths and weaknesses is an important step in advancing recreation and leisure sciences. To continue this critical examination and to inform methodological decision making, this study compared three approaches to determine how alternative approaches may result in contradictory…

Research that elucidates the complexities of gender balance is itself a controversial field. This article examines challenges and limitations in quantitative as well as qualitative research on gender balance in early childhood education and care (ECEC). Cross country comparisons of male and female participation in the ECEC workforce are often…

Overlap-based measures are increasingly applied in the synthesis of single-subject research. This article considers two criticisms of overlap-based metrics, specifically that they do not measure magnitude of effect and do not adequately correspond with visual analysis. It is argued that these criticisms are based on fundamental misconceptions…

Despite a growing number of studies on the effectiveness of teaching and quality in higher education, reports indicate that more work is needed regarding meeting expectations in teaching quality, in particular, in investigating lecturers' application of pedagogical knowledge when transferring discipline-specific knowledge. This study aimed to…

Applied linguists have turned increasingly in recent years to meta-analysis as the preferred means of synthesizing quantitative research. The first step in the meta-analytic process involves defining a domain of interest. Despite its apparent simplicity, this step involves a great deal of subjectivity on the part of the meta-analyst. This article…

Online surveys are increasingly used in educational research, yet little attention has focused on ethical issues associated with their use in educational settings. Here, we draw on the broader literature to discuss 5 key ethical issues in the context of educational survey research: dual teacher/researcher roles; informed consent; use of…

Randomized experiments are commonly used to evaluate if particular interventions improve student achievement. While these experiments can establish that a treatment actually "causes" changes, typically the participants are not randomly selected from a well-defined population and therefore the results do not readily generalize. Three…

Researchers suggest that certain supportive behaviors of mentors could increase the benefits of school-based mentoring for youth. However, the literature contains few validated instruments to measure these behaviors. In our present study, we aimed to construct and validate a tool to measure the supportive behaviors of mentors participating in…

This paper reviews literature on the rationale, challenges, and recommendations for choosing a nonequivalent comparison (NEC) group design when evaluating intervention effects. After reviewing frequently addressed threats to validity, the paper describes recommendations for strengthening the research design and how the recommendations were…

Despite the value of the comparative perspective for the study of higher education is widely recognised, there is little consensus about the specific methodological approaches. Quantitative tools outlined their relevance for addressing comparative analyses since they are supposed to reducing the complexity, finding out and graduating similarities…

This research project case study describes the research practicalities and design of a mixed methods dissertation study about the language-learning motivation of students who had attended a language immersion school. Mixed methods research, an approach to research in which elements of both quantitative (numbers-focused) and qualitative…