The current study examined the initial psychometrics of the "Early Childhood Behavior Problem Screening Scale" (ECBPSS; Epstein & Nelson, 2006), namely the factor structure and associated internal consistency of factor items of parent and teacher versions as well as interrater reliability. Data came from samples of preschool- and…

A process for judging the alignment between curriculum standards and assessments developed by the author is presented. This process produces information on the relationship of standards and assessments on four alignment criteria: Categorical Concurrence, Depth of Knowledge Consistency, Range of Knowledge Correspondence, and Balance of…

The authors of this article examine the analytic and reporting features of research articles cited in "Studying Teacher Education: The Report of the AERA Panel on Research and Teacher Education" (Cochran-Smith & Zeichner, 2005b) that used quantitative reporting practices. Their purpose was to help to identify reporting practices that can be…

A study into how academics learn to assess student performance affirms the significance of context in understanding learning in the academic workplace. The study involved three case studies in academic departments with significant differences in the teaching, research and professional dimensions of academic life. This article reports on the…

Seniors submitted and revised essays within ePortfolios for evaluation by high school faculty and university first-year writing instructors. The project also examined differences between high school and college teachers' responses and effects on students' assumptions about the writing expected and valued in college. Results suggest techniques…

Preservice special education teacher candidates in higher education training programs are given little or no specific training in the use of basic native language or cultural academic instructional techniques to support the learning needs of students who have not reached a level of English proficiency. Preservice special education students are…

Performance assessments have become the norm for evaluating language learners' writing abilities in international examinations of English proficiency. Two aspects of these assessments are usually systematically varied: test takers respond to different prompts, and their responses are read by different raters. This raises the possibility of undue…

This study focused on the impact of oral dialogue journals on Iranian EFL learners' pronunciation. Three classes of intermediate learners, after being reassured of their homogeneity, were randomly assigned to treatment (14 students), control (9 students), and placebo (10 students) groups. Learners in the treatment group had to respond to the…

The psychometric characteristics of the Test of Written English (TWE) rating scale were explored. Rasch model scalar analysis methodology was employed with more than 4,000 scored essays across 2 elicitation prompts to gather information about the rating scale and rating process. Results suggested that the intervals between TWE scale steps were…

A desirable goal would be to develop a methodology for scoring essays so that the final grades are less affected by when or by whom each essay was read. It seems sensible to derive such grades by somehow adjusting the ratings originally given by each reader. This essay describes a solution that relies on statistical adjustment, using the context…

An accepted criterion for gauging the fairness of examinees' scores, derived from judge-awarded ratings, has been the size of the correlation between the judges and the inter-rater reliability. Various means of achieving inter-rater reliability were reviewed, and a model to measure inter-rater reliability is forwarded. Both theoretical and…

Investigated the effects of knowledge of current pay levels and perceived job gender on subsequent job evaluations. Confirmed the hypothesis that jobs with high pay levels would receive higher evaluations than jobs with low pay levels. Found evidence of a pro-female bias in the job evaluation ratings of 53 job evaluators. (Author/ABB)

Comparisons of temperament ratings of preschool and kindergarten children to determine level of agreement among observers showed that teacher and parent ratings were in no greater agreement than student and parent ratings. Teachers and students displayed highest number of significant correlations. Agreement did not increase with age of children.…

A Monte Carlo study, involving 6,000 "computer subjects" and three raters, explored the reliability of the inverse screen test for cluster analysis. Results indicate that the inverse screen may be a useful and reliable cluster analytic technique for determining the number of true groups. (TJH)

A method for statistically removing interrater grading inconsistency as a source of bias is presented and a study using this technique is described. Assumptions, techniques, and practical considerations are also discussed. (Author/MSE)