Single-case research design is a useful methodology for evaluating the presence of a functional relation between an intervention and the mathematical performance of students with a learning disability. However, a functional relation cannot be established with threats to internal validity of the design. External validity is impacted if researchers…

Assessment and evaluation are at the heart of all functioning systems, as progress can be determined with the information collected by these assessments. The problem addressed in this study is the lack of actionable data to evaluate whether assessments aligned with Montessori benchmarks support the achievement of student outcomes. To address the…

Classroom management (CM) practices have a well-established, intuitive, and empirical connection with student academic, social, emotional, and behavioral outcomes. CM, defined as educator practices used to create supportive classroom environments, may be the implementation factor that is most impactful of the universal Tier I supports. Recognizing…

The study aimed to validate the factors that affect the inter-rater reliability of secondary school certificate (SSC) papers in high-stake testing. For this purpose, papers of Urdu and English of Board of Intermediate and Secondary Education (BISE) were selected. A survey method was used to collect marking on the same set of papers from each rater…

In this study, we analyze the two-phase bottom-up procedure applied by the Future and Emerging Technologies Program (FET-Open) at the Research Executive Agency (REA) of the European Commission (EC), for the evaluation of highly interdisciplinary, multi-beneficiary research proposals which request funding. In the first phase, remote experts assess…

The purpose of this study was to investigate a new way of evaluating interrater reliability that can allow one to determine if two raters differ with respect to their rating on a polytomous rating scale or constructed response item. Specifically, differential item functioning (DIF) analyses were used to assess interrater reliability and compared…

Many high-stakes examinations in the United Kingdom (UK) use both constructed-response items and selected-response items. We need to evaluate the inter-rater reliability for constructed-response items that are scored by humans. While there are a variety of methods for evaluating rater consistency across ratings in the psychometric literature, we…

Purpose: The purpose of this study was to evaluate the reliability and validity of the Kannada version of the Consensus Auditory-Perceptual Evaluation of Voice (CAPE-V). Method: The Kannada version of CAPE-V comprises six phrases that are phonetically designed as per the CAPE-V requirements. Sixty-five (21 individuals with dysphonia and 44…

To examine reliability and validity of the new Social Motor Function Classification System for Children with Autism Spectrum Disorders (SMFCS-ASD). The SMFCS-ASD reliability was examined on 25 children (62.4 months SD 7.8) with ASD among six physical therapists. The validity study involved 1001 children (57.0 months, SD 9.9) with ASD using the…

Classroom observation is a common approach to teacher evaluation. Yet, concerns about differences in rater judgment are widespread. Despite this concern, few researchers have examined the practical impact of such differences in rater judgments on teachers' judged effectiveness. This study fills that gap. Using data from a large-scale teacher…

Observations are widely used in research and evaluation to characterize teaching and learning activities. Because conducting observations is typically resource intensive, it is important that inferences from observation data are made confidently. While attention focuses on interrater reliability, the reliability of a single-class measure over the…

Thin slices ratings (i.e., ratings based on first impressions) have yielded intriguingly accurate results in various domains. Among other, researcher have applied the thin slices technique to assess instructional quality, showing that teacher-student interactions can be reliably inferred by just very short snippets of classroom instruction. The…

Our study aims to provide empirical evidence for and against the valid use of primary school students' ratings of three generic dimensions of teaching quality (classroom management, supportive climate, cognitive activation). We examine whether students discriminate between corresponding dimensions in different subjects, taking into account whether…

Purpose: Meaningful changes in picture naming responses may be obscured when measuring accuracy instead of quality. A statistic that incorporates information about the severity and nature of impairments may be more sensitive to the effects of treatment. Method: We analyzed data from repeated administrations of a naming test to 72 participants with…

Executive function challenges are commonly reported in the home setting for children with an autism spectrum disorder diagnosis (hereafter, autism), but little is known about these challenges in the school setting. A total of 337 youth (autism, N = 241 and typically developing, N = 96) were assessed using Behavior Rating Inventory of Executive…