Running Records are thought to be an excellent formative assessment tool because they generate results that educators can use to make their teaching more responsive. Despite the technical nature of scoring Running Records and the kinds of important decisions that are attached to their analysis, few studies have investigated assessor accuracy. We…

As computer-based testing becomes more prevalent, the attention paid to response time (RT) in assessment practice and psychometric research correspondingly increases. This study explores the rate of Type I error in detecting preknowledge cheating behaviors, the power of the Kullback-Leibler (KL) divergence measure, and the L person fit statistic…

Background/purpose: This study aimed to reveal the accuracy of estimation of multiple-choice test items parameters following the models of the item-response theory in measurement. Materials/methods: The researchers depended on the measurement accuracy indicators, which express the absolute difference between the estimated and actual values of the…

Although it is commonly believed that automation aids human decision-making, conflicting evidence raises questions about whether individuals would gain greater advantages from automation in difficult tasks. Our study examines the combined influence of task difficulty and automation reliability on aided decision-making. We assessed decision…

Equating is a statistical procedure used to adjust for the difference in form difficulty such that scores on those forms can be used and interpreted comparably. In practice, however, equating methods are often implemented without considering the extent to which two forms differ in difficulty. The study aims to examine the effect of the magnitude…

When cross-examining witnesses, lawyers will sometimes use complex questions to "trip-up" witnesses and thus cast doubt on their testimony. Extending prior research, the present study showed that multifaceted test questions (which contain both a true and false proposition) reduced accuracy for both misled and control participants but had…

Accurate metacognitive monitoring improves performance in a variety of naturalistic contexts. However, the laboratory contexts used to study metacognition differ from naturalistic environments in important ways. Specifically, laboratory experiments require learners to make repeated, overt judgments that are thought to reflect underlying…

Foreign language (L2) learners' speaking proficiency is often quantified using two dimensions: intuitive human ratings and analytical, linguistic complexity, accuracy, and fluency (CAF) indices. While previous research and assessment practices have predominantly focused on either the subjective approach to L2 speaking or the objective one, it is…

Background and Context: Feedback on one's progress is essential to new programming language learners, particularly in out-of-classroom settings. Though many study materials offer assessment mechanisms, most do not examine the accuracy of the feedback they deliver, nor give evidence on its validity. Objective: We investigate the potential use of a…

In three experiments, we sought to understand when and why people use an algorithm decision aid. Distinct from recent approaches, we explicitly enumerate the algorithm's accuracy while also providing summary feedback and training that allowed participants to assess their own skills. Our results highlight that such direct performance comparisons…

Background: Flying accuracy is influenced by pilots' affective reactions to task demands. A better understanding of task-related emotions and flying performance is needed to enhance pilot training. Objective: Understand pilot trainees' performance and emotional dynamics (intensity, frequency and variability) based on training phase and difficulty…

This study examined the relationship between complexity, accuracy and fluency (CAF), using longitudinal data. Conducted from the perspective of complex dynamic systems theory (CDST), this research adopted four CAF general measures to track the development of five Chinese undergraduates' English speech over a 15-week period. LOWESS graphs supported…

This study explored the impact of oxidation-reduction reaction problem difficulty on university students' cognitive load using event-related potentials (ERPs). Forty-eight balanced low and high difficulty problems were designed. Fifteen undergraduate students majoring in chemistry (8 females and 7 males) participated in the study. Results…

Unfolding models are frequently used to develop scales for measuring attitudes. Recently, unfolding models have been applied to examine rater severity and accuracy within the context of rater-mediated assessments. One of the problems in applying unfolding models to rater-mediated assessments is that the substantive interpretations of the latent…

Rubrics, which are designed to inform learners about assessment criteria and quality levels concerning a task assignment, are a widely used means to support learners in self-assessing their task performance. Usually, rubrics enhance task performance. Surprisingly, however, relatively little is known about the underlying mechanisms via which…