Although reliability is essential to validity, most research on violence risk assessment tools has paid little attention to strategies for improving rater agreement. The authors evaluated the degree to which perceived subjectivity in scoring guidelines for items from two measures--the Psychopathy Checklist-Revised (PCL-R) and the Historical,…

The quality of measurement systems used in almost all natural sciences other than behavior analysis is usually evaluated through calibration study rather than relying on interobserver agreement. We demonstrated some of the basic features of calibration using observer-measured rates of free-operant responding from 10 scripted 10-min calibration…

This study investigated factors that could affect inter-examiner reliability in the pronunciation assessment component of speaking tests. We hypothesized that the rating of pronunciation is susceptible to variation in assessment due to the amount of exposure examiners have to nonnative English accents. An inter-rater variability analysis was…

The current study examines how using a multidimensional battery of predictors (high-school grade point average (GPA), SAT/ACT, and biodata), and weighting the predictors based on the different values institutions place on various student performance dimensions (college GPA, organizational citizenship behaviors (OCBs), and behaviorally anchored…

As the ability to read proficiently is essential for success both in and out of the school setting, literacy has become an area of particular focus in today's classrooms. While recent assessments indicate that students are making progress in the area of reading (e.g., National Assessment of Educational Progress [NAEP], 2011), there continues to be…

The Applied Language Centre at University College Dublin offers foreign language modules to students in ten languages at CEFR [Common European Framework of Reference for Languages] levels ranging from A1 to B2. Efforts have been underway in the Centre to standardise the assessment components across languages to ensure parity between module credits…

This paper reports the results of interrater agreement analyses on a pilot special education teacher evaluation instrument, the Recognizing Effective Special Education Teachers (RESET) Observation Tool (OT). Using evidence-based instructional practices as the basis for the evaluation, the RESET OT is designed for the spectrum of different…

With over 800 million users worldwide, the global importance of Facebook as a social-networking platform is beyond doubt. This popularity, particularly among university-students, has encouraged research to explore ways in which social networking can be adapted into virtual learning environments. In particular, this study uses the think-aloud…

Children's social and emotional competence abilities have been linked to successful social interactions and academic performance. This study examined the teacher and observer ratings of social and emotional competence for 89 young (3- to 5-year-old), African American children from economically stressed urban environments. There was a specific…

Background: Sentence construction and syntactic organization are known to be poor in children with specific language impairments (SLI), but little is known about the way in which children with SLI approach language tasks, and static standardized tests contribute little to the differentiation of skills within the population of children with…

Objective: The American Board of Psychiatry and Neurology (ABPN) announced in 2007 that general psychiatry training programs must conduct Clinical Skills Verification (CSV), consisting of observed clinical interviews and case presentations during residency, as one requirement to establish graduates' eligibility to sit for the written certification…

Background: LaVigna, Christian, and Willis (2005) reported on a project where Institute for Applied Behaviour Analysis (IABA) staff trained a professional team in New Zealand (NZ) to provide behavioural services that met defined criteria. The NZ team was then trained to train other practitioners to meet the same professional standards. However, no…

The concept of voice has long attracted the attention of teachers, but more recently has also been the focus of a growing body of research aiming to understand voice as self-representation in writing. Adopting a socio-cultural orientation to voice, studies have revealed much about how textual choices are used by readers to build images of…

Students' writing can provide better insight into their thinking than can multiple-choice questions. However, resource constraints often prevent faculty from using writing assessments in large undergraduate science courses. We investigated the use of computer software to analyze student writing and to uncover student ideas about chemistry in an…

Background: Having an objective means of evaluating the quality of behaviour support plans (BSPs) could assist service providers and statutory authorities to monitor and improve the quality of support provided to people with intellectual disability (ID) who exhibit challenging behaviour. The Behaviour Support Plan Quality Evaluation Guide II…