Despite the rising popularity of big data, there is speculation that evaluators have been slow adopters of these new statistical approaches. Several possible reasons have been offered for why this is the case: ethical concerns, institutional capacity, and evaluator capacity and values. In this method note, we address one of these barriers and aim…

Systematic reviews are a time-consuming yet effective approach to understanding research trends. While researchers have investigated how to speed up the process of screening studies for potential inclusion, few have focused on to what extent we can use algorithms to extract data instead of human coders. In this study, we explore to what extent…

Psycholinguistics is the research of language in the mind: The architecture of storing the different components of language in our memory, and the ways we access that knowledge when processing and producing language. Psycholinguists usually conduct quantitative experimental research in which certain standards of experimental design, data…

This paper describes, and lays out an argument for, the use of a procedure to help groups of reviewers to judge the quality of prior research reports. It argues why such a procedure is needed, and how other existing approaches are only relevant to some kinds of research, meaning that a review or synthesis cannot successfully combine quality…

The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to…

This paper argues for three aspects of tolerance with respect to QCA research: tolerance with respect to different approaches to QCA; producing QCA research with tolerance (work that is resistant to criticism); and for QCA researchers to be clear about the tolerance of the solutions they present -- especially in terms of calibration and truth…

Systematic reviews (SRs) have an important role in the healthcare decision-making practice. Assessing the overall confidence in the results of SRs using quality assessment tools, such as "A MeaSurement Tool to Assess Systematic Reviews 2" (AMSTAR 2), is crucial since not all SRs are conducted using the most rigorous methods. In this…

Assessment and institutional research offices have too much data and too little time. Standard reporting often crowds out opportunities for innovative research. Fortunately, advancements in data science now offer a clear solution. It is equal parts technique and philosophy. The first and easiest step is to modernize data work. This column…

The rapid development of artificial intelligence has driven the transformation of educational evaluation into big data-driven. This study used a systematic literature review method to analyzed 44 empirical research articles on the evaluation of big data education. Firstly, it has shown an increasing trend year by year, and is mainly published in…

This study analyzed studies done in Turkey in the context of curriculum evaluation (CE) by asking, "How is it made? The study was carried out in two stages. In the first stage, the document analysis method used 215 theses written between 1991 and 2020 on CE were analyzed according to the "thesis review form." In the second stage,…

The process of quantification is a powerful development shaping many domains of life today. In the area of education, for example, performance measurement, testing and ranking have become common tools of governance. Quantification is not a neutral way of describing society, but a process of valorisation. It has three sociologically relevant…

Many articles have been published in gifted education in recent years. This study aims to provide a comprehensive review of the evolution of academic studies in gifted education. In this context, the structural topic modeling (STM) method was used to analyze the topics and trends in the field. STM is a machine learning technique that utilizes…

Purpose: The purpose of this dissertation is to investigate the feasibility of using tests of robustness in peer review. This study involved selecting three high-impact papers which featured open data and utilized bioinformatic analyses but provided no source code and refactoring these to allow external survey participants to swap tools,…

The reason for conducting the present case study was the problems the researchers encountered during data collection for another research project (Primary Study) entitled "The effects of virtual versus traditional flipped classes on EFL learners' grammar knowledge, self-regulation, and autonomy." Two online questionnaires were…

The purpose of this paper was to review best-practice methods of collecting and analyzing speech production data from minimally verbal autistic speakers. Data on speech production data in minimally verbal individuals are valuable for a variety of purposes, including phenotyping, clinical assessment, and treatment monitoring. Both perceptual…