Purpose: The primary objective of this research paper was to examine the objectivity of the preselected items evaluation (PIE) method, a prevalent translation scoring method deployed by international institutions such as UAntwerpen, UGent and the University of Granada. Design/methodology/approach: This research critically analyzed the scientific…

Rapid approaches are essential when resources are limited and when findings are required in real-time to inform decisions. Limitations exist in their design and implementation, which can lead to a reduced level of trust in findings. This review sought to map the methods used across rapid evaluations and research to facilitate timeliness and…

Narratives that dominant the discourse of the experiences of people of color, specifically in education settings, are incomplete. Therefore, identifying methodological approaches that emphasize the perspectives of minoritized groups is essential. Counternarratives have been applied to help (re)tell stories of the oppressed to challenge dominant…

The peer review process is important for substantiating the quality of research. In this study, we examine the experiences of peer reviewers in agricultural education. The research presented in this article is part of a larger study that also examined the perspectives of researchers about peer review. We used a survey to collect data from a random…

School leadership research literature has a large number of widely used constructs. Could fewer constructs bring more clarity? This study evaluates construct content validity, defined as the extent to which a measure's items reflect a theoretical content domain, in school leadership literature. To do so, we reviewed 29 articles that used Teaching…

Evidence synthesis findings depend on the assumption that the included studies follow good clinical practice and results are not fabricated or false. Studies which are problematic due to scientific misconduct, poor research practice, or honest error may distort evidence synthesis findings. Authors of evidence synthesis need transparent mechanisms…

Agent-based models (ABMs) often rely on psychometric constructs such as 'opinions', 'stubbornness', 'happiness', etc. The measurement process for these constructs is quite different from the one used in physics as there is no standardized unit of measurement for opinion or happiness. Consequently, measurements are usually affected by 'psychometric…

Asymmetric Likert-type items in research studies can present several challenges in data analysis, particularly concerning missing data. These items are often characterized by a skewed scaling, where either there is no neutral response option or an unequal number of possible positive and negative responses. The use of conventional techniques, such…

Qualitative research sample size determination has always been a contentious and confusing issue. Studies are often vague when explaining the processes and justifications that have been used to determine sample size and strategy. Some provide no mention of sampling at all, whilst others rely too heavily on the concept of saturation for determining…

RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two…

This paper presents a systematic literature review characterizing the methodological properties of usability studies conducted on educational and learning technologies in the past 20 years. PRISMA guidelines were followed to identify, select, and review relevant research and report results. Our rigorous review focused on (1) categories of…

Social research methodologists have postulated that the transparency of survey procedures and data processing is mandatory for assessing the Total Survey Error. Recent analyses of data from cross-national surveys have demonstrated an increase in the quality of documentation reports over time and significant differences in documentation quality…

This paper tries to probe the common quality issues in academic master theses of education majors and propose some possible countermeasures. The study sampled and evaluated 275 academic master theses of education majors in China to examine common quality issues in them, and used a questionnaire to survey 210 academic master's degree candidates of…

As a research field geared toward understanding and improving learning, Learning Analytics (LA) must be able to provide empirical support for causal claims. However, as a highly applied field, tightly controlled randomized experiments are not always feasible nor desirable. Instead, researchers often rely on observational data, based on which they…

Ethics review of research involving humans has become something of an institution in recent years. It is intended to protect participants from harm and, to that end, follows rigorous standards. Given recent changes in research methodologies utilized in medical research, it may be that ethics review for some kinds of studies needs to be reexamined.…