Observations seem particularly susceptible to rater error due to the level of subjectivity involved in assessment. Thus, the present paper aims to investigate: (1) inter-rater agreement (IRA) using the Classroom Assessment Scoring System -- Secondary version (CLASS-S) and (2) the CLASS-S factor structure in a Norwegian context. Inter-rater…

Background: Clinical swallowing assessment is largely limited to qualitative assessment of behavioural observations. There are limited quantitative data that can be compared with a healthy population for identification of impairment. The Test of Masticating and Swallowing Solids (TOMASS) was developed as a quantitative assessment of solid bolus…

This paper investigated the intuitive development of the concept of integers among primary school students. In order to reveal if primary school students had an intuitive sense of integers, an assessment consisting of five questions was prepared and applied to a total 100 4th grade students. A variety of integer concepts were utilized in the…

The UTeach Observation Protocol (UTOP) was designed to inform STEM teacher education. The instrument has been used in prior studies examining inter-rater reliability and relationships to teacher value-added scores. However, prior work has not shown examples of how rating with the UTOP works in practice nor has it discussed the instrument's…

The literature on Automated Essay Scoring (AES) systems has provided useful validation frameworks for any assessment that includes AES scoring. Furthermore, evidence for the scoring fidelity of AES systems is accumulating. Yet questions remain when appraising the scoring performance of AES systems. These questions include: (a) which essays are…

In the current study, two pools of 250 essays, all written as a response to the same prompt, were rated by two groups of raters (14 or 15 raters per group), thereby providing an approximation to the essay's true score. An automated essay scoring (AES) system was trained on the datasets and then scored the essays using a cross-validation scheme. By…

The student evaluation of teaching process is generally thought to produce reliable results. The consistency is found within class and instructor averages, while a considerable amount of inconsistency exists with individual student responses. This paper reviews these issues along with a detailed examination of common measures of reliability that…

The current study was designed to evaluate a rotating momentary time sampling (MTS) data collection system. A rotating MTS system has been used to measure activity preferences of preschoolers but not to collect data on responses that vary in duration and frequency (e.g., talking). We collected data on talking for 10 preschoolers using a 5-s MTS…

The goal of this case study was to introduce students to ways to conduct research on parent child cognitive learning interactions. To this end, the case study begins with an overview of the theoretical and empirical work supporting the development of my research program on parent child cognitive learning interaction research and continues with a…

Few assessment tools have been designed to assess motor creativity, and the existing tools have limitations. To bridge this gap, the current study aimed at designing a new movement creativity assessment tool that considers the unique features underlying the expression of creativity through movement. A modified Delphi technique was used to collect…

Moderation is put forward as they key strategy for improving the reliability of teacher assessment. However, for many teachers the word "moderation" conjures up ideas of uncomfortable situations in which marking is being checked by others and there are prolonged arguments about tiny features of individual work. In this article, the…

This study examined the exchangeability of total scores (i.e., intelligent quotients [IQs]) from three brief intelligence tests. Tests were administered to 36 children with intellectual giftedness, scored live by one set of primary examiners and later scored by a secondary examiner. For each student, six IQs were calculated, and all 216 values…

Objective: The objective was to examine reference errors in research articles published in Research on Social Work Practice. High rates of reference errors in other top social work journals have been noted in previous studies. Methods: Via a sampling frame of 22,177 total references among 464 research articles published in the previous decade, a…

The validity and reliability of the Test of Gross Motor Development-3 (TGMD-3) were measured, taking into consideration the preference for visual learning of children with autism spectrum disorder (ASD). The TGMD-3 was administered to 14 children with ASD (4-10 years) and 21 age-matched typically developing children under two conditions: TGMD-3…

In this research report, we describe the design and empirical findings for a large-scale study of essay writing ability with approximately 2,500 high school students in Germany and Switzerland on the basis of 2 tasks with 2 associated prompts, each from a standardized writing assessment whose scoring involved both human and automated components.…