RobotReviewer is a tool for automatically assessing the risk of bias in randomized controlled trials, but there is limited evidence of its reliability. We evaluated the agreement between RobotReviewer and humans regarding the risk of bias assessment based on 1955 randomized controlled trials. The risk of bias in these trials was assessed via two…

Background: Cultivating students' problem-solving knowledge and skills in the workspace has been regarded as a primary goal in technical and vocational education. To this end, students are usually guided to complete the learning of workspace knowledge and skills by following the question, prompt, interaction and evaluation (QPIE) procedure.…

In massive open online course (MOOC) environments, computer-based analysis of course reviews enables instructors and course designers to develop intervention strategies and improve instruction to support learners' learning. This study aimed to automatically and effectively identify learners' concerned topics within their written reviews. First, we…

Comparative studies on paper and pencil--and computer-based tests principally focus on statistical analysis of students' performances. In educational assessment, comparing students' performance (in terms of right or wrong results) does not imply a comparison of problem-solving processes followed by students. In this paper, we present a theoretical…

As a research field geared toward understanding and improving learning, Learning Analytics (LA) must be able to provide empirical support for causal claims. However, as a highly applied field, tightly controlled randomized experiments are not always feasible nor desirable. Instead, researchers often rely on observational data, based on which they…

We develop a structural after measurement (SAM) method for structural equation models (SEMs) that accommodates missing data. The results show that the proposed SAM missing data estimator outperforms conventional full information (FI) estimators in terms of convergence, bias, and root-mean-square-error in small-to-moderate samples or large samples…

To be able to support students' competence development in solving physics problems over the course of a lesson series effectively, teachers need a proper appreciation of students' deficiencies. As teachers commonly assess students' competence by means of written tests, teachers are challenged to interpret students' work on these tests and to…

University rankings have gradually become an issue for concern in the academic community worldwide. Several mechanisms with different methodologies have been developed to rank the universities appropriately. However, some ranking tools have notable issues, especially with the indicators adopted. Some are based merely on research performance,…

Purpose: The aim of this paper is to study the use of FIS Response Analysis for Critical Thinking Assessment (FRACTA) method to assess critical thinking in STEM problem solving. The use of FIS (facts, ideas and solutions) chart as a tool to elicit student critical thinking responses and the method of scoring the responses are investigated. Method:…

Background: Invention and mastery learning approaches differ in their foundational educational paradigms, proposed mechanisms of learning, and potential impacts on learning outcomes. They also differ in their resource requirements. We explored the relative effects of 'invent and problem-solve, followed by instruction' (PS-I) learning compared to…

Classroom management practices are critical to the success of teachers and students, and a growing number of programs have been developed to improve these practices. However, there has been less investigation into observational tools to assess classroom management and exploration of whether it can be measured consistently by observers across…

The advent of problem-based assessment has opened the question of whether or not conventional approaches to assessment are truly a representative of students' knowledge (Lampert, 2001). In an attempt to find out about the suitability of problem-based tests in evaluating student-teachers' knowledge of teaching English, a problem-based test was…

Purpose: This paper aims to compare two types of prompts, encouraging participants to think about real-world examples or engineering principles to show how these two approaches can result in vastly different design practices. Design/methodology/approach: Two studies (N = 20, N = 40) examine the impact of two different prompts. Non-expert students,…

Classroom management practices are critical to the success of teachers and students, and a growing number of programs have been developed to improve these practices. However, there has been less investigation into observational tools to assess classroom management and explore whether it can be measured consistently by observers across elementary,…

A mathematical client-driven task known as Model-Eliciting Activities was implemented with students of different levels of achievement (i.e., low, average, and high) at the high-school level. The study strived to prove that Model-Eliciting Activities can be solved by students at any achievement level and be used as an assessment tool. Students…