In this study, we describe a framework for monitoring rater performance over time. We present several statistical indices to identify raters whose standards drift and explain how to use those indices operationally. To illustrate the use of the framework, we analyzed rating data from the 2002 Advanced Placement English Literature and Composition…

Traditional methods for examining differential item functioning (DIF) in polytomously scored test items yield a single item-level index of DIF and thus provide no information concerning which score levels are implicated in the DIF effect. To address this limitation of DIF methodology, the framework of differential step functioning (DSF) has…

The pressing need for English oral communication skills in multifarious contexts today is compelling impetus behind the large number of studies done on oral proficiency interviewing. Moreover, given the recently articulated concerns with the fairness and social dimension of such interviews, parallel concerns have been raised as to how most fairly…

Although the Angoff procedure is among the most widely used standard setting procedures for tests comprising multiple-choice items, research has shown that subject matter experts have considerable difficulty accurately making the required judgments in the absence of examinee performance data. Some authors have viewed the need to provide…

In this article, we introduce a person-fit statistic called the hierarchy consistency index (HCI) to help detect misfitting item response vectors for tests developed and analyzed based on a cognitive model. The HCI ranges from -1.0 to 1.0, with values close to -1.0 indicating that students respond unexpectedly or differently from the responses…

Standard setting (the process of establishing minimum passing scores on high-stakes exams) is a highly evaluative and policy-driven process. It is a common belief that standard setting panels should be diverse and representative. There is concern, however, that panelists with varying characteristics may differentially influence the results of the…

Since the break up of the USSR, its former republics have seen the emergence and rapid expansion of an examinations industry that was, to all intents and purposes, unknown in Soviet times. New national assessment agencies have been established and been charged with, amongst other things, developing high-stakes exams to replace the diverse and…

Large scale tests have been considered by many scholars in the field of language testing and teaching to influence teaching and learning considerably. The present study looks at the effect of a large scale test (Konkoor) on the attitudes of teachers in high schools. Konkoor is the university entrance examination in Iran which is taken by at least…

Distance Education has globally become one of the important solutions for increasing admission into the universities, decongesting campuses and efficient utilization of time and space. To ensure the sustainability of the programmes' noble objectives calls for periodic re-evaluation of its modus operandi including the assessment of the perception…

This handbook will assist principals and school testing coordinators in implementing the spring 2007 administration of the Developmental Reading Assessment (DRA). Information regarding administration timeline, reporting, process, online tools and contact personnel is discussed. Contents include: (1) Scheduling; (2) Identify Primary Test…

Comprehensive academic achievement tests are routinely used by school psychologists in psycho-educational assessment batteries to identify learning disabled students. A variety of assessment measures are used across age groups to determine if a discrepancy exists between academic achievement and intellectual functioning; however, among the most…

In this inquiry, I examine the cross educational experiences of ten Arab undergraduate students in two English-language universities in Montreal. Participants were from Libya, Tunisia, Algeria, and Morocco and have been in Canada for three to seven years. Classic qualitative methodological tools of in-depth interviews, participant observation and…

State test scores administered for accountability purposes are regularly used to adjust instruction in nuanced ways. This is no accident--No Child Left Behind demanded that students' scores be returned quickly to teachers in order that this might be the case, and the idea of data-driven decision making continues as one way the promise of education…

Examination malpractice or cheating has become a global phenomenon. In different countries of the world today, developed and developing, academic dishonesty especially cheating in examinations has heightened and taken frightening dimension. In many countries of the world this phenomenon has become a serious matter of concern that has left many…

The purpose of this classroom-based study was to discover the kinds of skill integration tasks that were employed by English teachers in Kuwait and to measure their attitudes toward implementing the skill integration technique in their classrooms. Data collection involved recording 25 hours of classroom-based observations, conducting interviews…