Although Machine Learning (ML) is used already in our daily lives, few are familiar with the technology. This poses new challenges for students to understand ML, its potential, and limitations as well as to empower them to become creators of intelligent solutions. To effectively guide the learning of ML, this article proposes a scoring rubric for…

This collaborative action research highlights the need for developing students' evaluative competence and self-reflection by embedding self-and-peer assessment into online instruction. Over the course of a semester in an online master program in mathematics and computer sciences, students conducted research on assigned topics, held presentations,…

In language performance tests, raters are important as their scoring decisions determine which aspects of performance the scores represent; however, raters are considered as one of the potential sources contributing to unwanted variability in scores (Davis, 2012). Although a great number of studies have been conducted to unpack how rater…

Purpose: This study describes the development and pilot testing of the Observational System for Recording Physical Activity-Elementary School (OSRAC-E) Version. Method: This system was developed to observe and document the levels and types of physical activity and physical and social contexts of physical activity in elementary school students…

Measuring the quality of classroom-based interactive shared book reading within the early childhood classroom represents a specific dimension of teacher-child interactions that is of great interest to researchers. This interest reflects decades of research demonstrating the benefit of reading to young children in both the home and the classroom.…

Autistic adults who have a history of committing crimes present a major problem for providers of services in terms of legal disposal options and possible interventions, and greater understanding of this group and their associated needs is required. For this reason, we aimed to investigate the face validity of a proposed sub-typology of autistic…

Purpose: The purpose of this study is to share quality process experience from a US comprehensive university to use both direct (participation rate and assessment quality) and indirect assessment measures (assessment survey) to evaluate the quality process. Design/methodology/approach: A mixed method design was used to evaluate the quality…

In this study, we examined the relationship of special education teachers' performance on the RESET Explicit Instruction observation protocol with student growth on academic measures. Special education teachers provided video recorded observations of three instructional lessons along with data from standardized, curriculum-based academic measures…

Online courses often include discussion forums, which provide a rich source of data to better understand and improve students' learning experiences. However, forum messages frequently contain private information that prevents researchers from analyzing these data. We present a method for discovering and redacting private information including…

Automated essay scoring engines (AESEs) are becoming increasingly popular as an efficient method for performance assessments in writing, including many language assessments that are used worldwide. Before they can be used operationally, AESEs must be "trained" using machine-learning techniques that incorporate human ratings. However, the…

Taxonomies of cognitive demand are frequently used to ensure that assessment tasks include questions ranging from low to high cognitive demand. This paper investigates inter-rater agreement among four evaluators on the cognitive demand of the South African National Senior Certificate Life Sciences examinations after training, practice and…

Rater variability has always been identified as an important source of measurement error in performance assessment, especially for oral proficiency tests. Rater training is commonly used as a means for compensating various sources of rater variability and adjusting their assessment quality. However, there is little research regarding the nature of…

One common modification to the Angoff standard-setting method is to have panelists round their ratings to the nearest 0.05 or 0.10 instead of 0.01. Several reasons have been offered as to why it may make sense to have panelists round their ratings to the nearest 0.05 or 0.10. In this article, we examine one reason that has been suggested, which is…

Classroom observations remain the predominant data source used in teacher evaluations, but little is known about how rater characteristics may affect teachers' scores. For special educators, whose instructional practice requires specialized knowledge and skills, school administrators (i.e., the raters) without experience in special education…

Implementation of research-based, Tier 1 behavior management strategies can be monitored to provide data-driven feedback and in support of integrity. The "Measure of Active Supervision and Interaction" (MASI) was developed to measure four behavior management practices (i.e., Praise, Correction, References to Behavior Expectations, Active…