Background: Meta-analyses of educational interventions have consistently documented the importance of methodological factors related to the choice of outcome measures. In particular, when interventions are evaluated using measures developed by researchers involved with the intervention or its evaluation, the effect sizes tend to be larger than…

The purpose of this study is to describe and evaluate a multilingual automated essay scoring (AES) system for grading essays in three languages. Two different sentence embedding models were evaluated within the AES system, multilingual BERT (mBERT) and language-agnostic BERT sentence embedding (LaBSE). German, Italian, and Czech essays were…

Students are assessed by coursework and/or exams, all of which are marked by assessors (markers). Student and marker performances are then subject to end-of-session board of examiner handling and analysis. This occurs annually and is the basis for evaluating students but also the wider learning and teaching efficiency of an academic institution.…

Students' expectations for their laboratory coursework are theorized to have an impact on their learning experiences and behaviors, such as engagement. Before students' expectations and engagement can be explored in different types of undergraduate chemistry laboratory courses, appropriate measures of these constructs must be identified, and…

Introduction: The Situational Judgment Test of Socioemotional Competence Development in Young People (DCSE-J) is a copyleft psychoeducational instrument for evaluating socioemotional competencies in adolescents. Different studies have analysed its psychometric validity, reliability and criterion validity. The present study aims to analyse…

Background: The current study aimed to validate the utility of previously established validity indicators derived from MOXO-d-CPT's continuous performance test. Method: Healthy simulators feigned impairment after searching online for relevant information, an ecologically valid coaching condition (n = 39). They were compared to ADHD patients (n =…

This dissertation describes an experiment to investigate how learners with different levels of background in computer science learn core concepts of computer science, in particular, algorithms. We designed a study to focus on cognitive task analysis for eliciting the empirical mental elements of learning two graph algorithms. Cognitive workload…

Considerable debate continues regarding the efficacy of the website RateMyProfessors.com (RMP). To date, however, virtually no direct, experimental research has been reported which directly bears on questions relating to sampling adequacy or item adequacy in producing what favorable correlations have been reported. The authors compare the data…

Traditionally, an anatomy practical examination is conducted using a free response format (FRF). However, this format is resource-intensive, as it requires a relatively large time investment from anatomy course faculty in preparation and grading. Thus, several interventions have been reported where the response format was changed to a selected…

Constructed-response (CR) questions are a mainstay of introductory physics textbooks and exams. However, because of the time, cost, and scoring reliability constraints associated with this format, CR questions are being increasingly replaced by multiple-choice (MC) questions in formal exams. The integrated testlet (IT) is a recently developed…

The original SOCO Scale was reduced to 10 items by Thomas, Soutar, and Ryan (2001). The author conducted a pretest and a posttest in his Personal Selling class during the Fall 2009 semester. Significant differences by gender, student sales experience and family member in the sales field were identified. The author once again pretested the…

Peer assessment typically requires students to judge peers' work against assessment criteria. We tested an alternative approach in which students judged pairs of scripts against one another in the absence of assessment criteria. First year mathematics undergraduates (N?=?194) sat a written test on conceptual understanding of multivariable…

Recent investigations challenge the construct validity of sustained silent reading tests. Performance of two groups of post-secondary students (e.g. struggling and non-struggling) on a sustained silent reading test and two types of cloze test (i.e. maze and open-ended) was compared in order to identify the test format that contributes greater…

Secondary education in Fiji ends with the Form 7 examination. Predictive validity for academic success of Form 7 scores which form the basis for admission into the Bachelor of Medicine Bachelor of Surgery programme of the Fiji School of Medicine was examined via a cohort of 129 students. Success rates for year 1 in 2008, 2009, and 2010 were 90.7…

Typical validation studies on standard setting models, most notably the Angoff and modified Angoff models, have ignored construct development, a critical aspect associated with all conceptualizations of measurement processes. Stone compared the Angoff and objective standard setting (OSS) models and found that Angoff failed to define a legitimate…