This review of current writing assessment practices focuses upon the adult population, an area significantly underrepresented within psychoeducational literature. As compared to other populations, such as K-12 students, there are few options for the practitioner wishing to evaluate adult writers by means of standardized assessment instruments.…

Effectively designed science learning environments revolve around students' sensemaking through the use of evidence to ground explanations about natural phenomena. However, little research has been conducted to investigate elementary teachers' learning to promote students' sensemaking in elementary (K-5) classrooms. The purpose of this…

The assessment of vocational workplace-based qualifications in England relies on human assessors (raters). These assessors observe naturally occurring, non-standardised evidence, unique to each learner and evaluate the learner as competent/not yet competent against content standards. Whilst these are considered difficult to measure, this study…

This paper presents the findings of a study designed to explore qualification users' perceptions and experiences of reliability in the context of national assessment outcomes in England. The study consisted of 17 focus groups conducted across six sectors of qualification users: students, teachers, trainee teachers, job-seekers, employers and…

This study is an attempt to investigate the effect of direct and indirect feedback on the writing performance of Iranian learners of Japanese as a foreign language. During one academic semester, three indirect feedback types including underlining, coding and translation were used as well as direct type of feedback in order to see which one makes a…

This paper investigates engagement (E), alignment (A), and rigor (R) as vital signs of high-quality teacher instruction as measured by the EAR Classroom Visit Protocol, designed by the Institute for Research and Reform in Education (IRRE). Findings indicated that both school leaders and outside raters could learn to score the protocol with…

Oral fluency and foreign accent distinguish L2 from L1 speech production. In language testing practices, both fluency and accent are usually assessed by raters. This study investigates what exactly native raters of fluency and accent take into account when judging L2. Our aim is to explore the relationship between objectively measured temporal,…

For accreditation and programmatic decision making, education school administrators use inter-rater reliability analyses to judge credibility of student-teacher assessments. Although weak levels of agreement between university-appointed supervisors and cooperating teachers are usually interpreted to indicate that the process is not being…

In this study, using mixed methods, we investigated the longer term effects of eCoaching through advanced online bug-in-ear (BIE) technology. Quantitative data on five dependent variables were extracted from 14 participants' electronically archived video files at three points in time--Spring 1 (i.e., baseline, which was the first semester of…

This study investigates how university instructors from various disciplines at a large, comprehensive university in the United States evaluate different varieties of English from countries considered "outer circle" (OC) countries, formerly colonized countries where English has been transplanted and is now used unofficially and officially…

Systematic reviews are used to inform practice, and develop guidelines and protocols. A questionnaire to quantify the risk of bias in systematic reviews, the review paper assessment (RPA) tool, was developed and tested. A search of electronic databases provided a data set of review articles that were then independently reviewed by two assessors…

Notwithstanding broad utility of COPs (classroom observation protocols), there has been limited documentation of the psychometric properties of even the most popular COPs. This study attempted to fill this void by closely examining the item and domain-level IRR (inter-rater reliability) of a COP that was used in a federally funded striving readers…

The empirical literature investigating general and domain-specific self-concepts of adults with learning disabilities was examined using meta-analytic techniques. Eight inclusion criteria were developed to evaluate this literature and led to the inclusion of 22 studies. Results indicated that adults with learning disabilities reported lower…

The Short-Term Assessment of Risk and Treatability (START; C. D. Webster, M. L. Martin, J. Brink, T. L. Nicholls, & S. L. Desmarais, 2009; C. D. Webster, M. L. Martin, J. Brink, T. L. Nicholls, & C. Middleton, 2004) is a relatively new structured professional judgment guide for the assessment and management of short-term risks associated…

Although differential ratings by multiple informants are an important issue in survey design, few studies test the degree of difference between informants. This study examined differences in caregiver and teacher ratings of school readiness of children from a disadvantaged urban community in Ireland. School readiness was assessed using the Short…