Purpose: The primary objective of this research paper was to examine the objectivity of the preselected items evaluation (PIE) method, a prevalent translation scoring method deployed by international institutions such as UAntwerpen, UGent and the University of Granada. Design/methodology/approach: This research critically analyzed the scientific…

The integration of artificial intelligence (AI), and particularly generative AI, into research methods is rapidly transforming both academic and industry marketing research, including both methods practices and education regarding these practices. AI application within methods offers new opportunities for enhancing efficiency, automating…

Introduction: The What Works Clearinghouse (WWC) reviews rigorous research on educational interventions with a goal of identifying "what works" and making that information accessible to educators and policymakers. In rating the quality of causal research, the WWC has historically prioritized internal validity over external validity. One…

When conducting a postal survey, a traditional recommendation is to use paper postage stamps instead of an automated postage system, to make sure that invitations have a more personal and attractive appearance. In this research note, we investigate whether this traditional recommendation is still valid. In the autumn of 2020, a postal survey was…

Outside of education, many academic disciplines developed and promulgated checklists to evaluate the quality of qualitative studies. This article used an embedded mixed methods approach in three stages: development of a conceptual framework reviewing 62 guidelines; a content analysis of key areas of the guidelines; and review of dissertations to…

Alongside the rise in online qualitative research, fraudulent representation of research participants is increasing, and current strategies to address this issue within studies of non-neurodiverse samples may exclude autistic voices. Seeking to reduce fraudulent or "scammer" participation in our focus group study, and increase data…

Transactional validity, a common approach in participatory research, is attained when preliminary analyses of research results are discussed with research participants and their feedback is incorporated in the analysis. Member checking is one way of achieving transactional validity, which has been heralded as a stronger version of validity reached…

Multilevel modeling (MLM) is a statistical technique for analyzing clustered data. Despite its long history, the technique and accompanying computer programs are rapidly evolving. Given the complexity of multilevel models, it is crucial for researchers to provide complete and transparent descriptions of the data, statistical analyses, and results.…

Data analysis is a significant methodological component when conducting quantitative education studies. Guidelines for conducting data analyses in quantitative education studies are common but often underemphasize four important methodological components impacting the validity of inferences: quality of constructed measures, proper handling of…

This report presents a review of the statistical practices of 30 journals representative of the second language field. A review of 150 articles showed a number of prevalent statistical violations including incomplete reporting of reliability, validity, non-significant results, effect sizes, and assumption checks as well as making inferences from…

This research study aims to evaluate the graduate theses and articles conducted on the concept of instructional leadership over the period between 2002 and 2017 in Turkey by means of methodological and statistical analysis techniques. In the study, which is configured by using the case study design as one of the qualitative research methods, the…

Based on findings from a large meta-data-based literature survey, this article is intended to provide a comprehensive synthesis of key features of China's music education system as seen through the lens of n = 116 major research studies, drawn from a total of N = 3,257 high-impact Chinese journal articles published during 2007 to 2019. The…

The use of video surveillance footage presents a new possibility in educational research as a reliable and valid source of learning and teaching activities in classrooms. However, the unique nature of surveillance footage requires different approaches and poses distinctive challenges in utilizing it in research, yet no methodological guides are…

To draw informed conclusions from research studies, research consumers need full and accurate descriptions of study methods and procedures. Preregistration has been proposed as a means to clarify reporting of research methods and procedures, with the goal of reducing bias in research. However, preregistration has been applied primarily to research…

The use of controlled comparisons pervades comparative historical analysis. Heated debates have surrounded the methodological purchase of such comparisons. However, the quality and validity of the conceptual building blocks on which the comparisons are based have largely been ignored. This article discusses a particular problem pertaining to these…