Rater variation in performance assessment can impact test-takers' scores and compromise assessments' fairness and validity (Crooks, Kane, & Cohen, 1996). Rater variation can also undermine a test's validity and fairness; therefore, it is important to investigate raters' scoring patterns in order to inform rater training. Substantial work has…

A framework for evaluation and use of automated scoring of constructed-response tasks is provided that entails both evaluation of automated scoring as well as guidelines for implementation and maintenance in the context of constantly evolving technologies. Consideration of validity issues and challenges associated with automated scoring are…

Consumers are increasingly turning to both the internet and apps for mental health assistance. Mobile technologies such as smart phones and tablets offer swift and anonymous access for students to internet sites and app environments. Availability, however, does not guarantee quality or credibility. This web-based pilot study was undertaken to…

This study intends to develop a standardized instrument for measuring classroom teaching and learning in secondary chemistry lessons. Based on previous studies and interviews with expert teachers, the progression of five quality levels was constructed hypothetically to represent the quality of chemistry lessons in Chinese secondary schools. The…

This article synthesized evidence for the validity and reliability of the Strengths and Difficulties Questionnaire in children aged 3-5 years. A systematic review using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement guidelines was carried out. Study quality was rated using the Consensus-based Standards for the…

This article reviews critical discourse analysis scholarship in education research from 2004 to 2012. Our methodology was carried out in three stages. First, we searched educational databases. Second, we completed an analytic review template for each article and encoded these data into a digital spreadsheet to assess macro-trends in the field.…

The aim of this study was to describe the activities of Estonian preschool teachers in the context of different teaching approaches and to compare the assessments of teachers of their own teaching with the assessments of observers of their teaching. For the data collection, the Early Childhood Classroom Observation Measure (ECCOM) and a structured…

A corpus of 313 freshman college essays was analyzed in order to better understand the forms and functions of humor in academic writing. Human ratings of humor and wordplay were statistically aggregated using Factor Analysis to provide an overall "Humor" component score for each essay in the corpus. In addition, the essays were also…

This study used generalizability theory to measure reliability on the Recognizing Effective Special Education Teachers (RESET) observation tool designed to evaluate special education teacher effectiveness. At the time of this study, the RESET tool included three evidence-based instructional practices (direct, explicit instruction; whole-group…

This paper reports on a mixed-methods approach to evaluate rater performance on a local oral English proficiency test. Three types of reliability estimates were reported to examine rater performance from different perspectives. Quantitative results were also triangulated with qualitative rater comments to arrive at a more representative picture of…

Educational professionals need assessments that yield psychometrically sound scores to assess students' behavioral and emotional functioning in order to guide data-driven decision-making processes. Rating scales have been found to be effective and economical, and often multiple informant perspectives can be obtained. The agreement between multiple…

Scholastic tests regard cognitive abilities to be domain-specific competences. However, high correlations between competences indicate either high task similarity or a dependence on common factors. The present rating study examined the validity of 12 Programme for International Student Assessment (PISA) and Third or Trends in International…

An online Moodle Workshop was evaluated for peer assessment effectiveness. A quasiexperiment was designed using a Seminar in Professionalism course taught in face-to-face mode to undergraduate students across two campuses. The first goal was to determine if Moodle Workshop awarded a fair peer grader grade. The second objective was to estimate if…

Successful collaboration among speech and language therapists (SLTs) and teachers fosters the creation of communication friendly classrooms that maximize children's spoken and written language learning. However, these groups of professionals may have insufficient opportunity in their professional study to develop the shared knowledge, perceptions…

This study investigated informant agreement on emotional and behavior problems and social skills in youth with autism spectrum disorder or intellectual disability using meta-analytic methods. Forty-nine studies were included, consisting of 107 effect sizes. The mean weighted effect size across all raters and all behaviors was 0.36, reflecting…