There is little doubt as to the importance of creativity for both economic and social progress and of the need for educational systems to enhance and nurture it. However, are educational systems promoting creativity? The general feeling is that it is, in fact, being "killed" in schools. The educational systems were designed for a…

Variation in policies virtual schools use to calculate course completion and retention rates impacts the comparability of these quality metrics. This study surveyed 159 U.S. virtual schools examining the variability in trial period and course completion policies--two policies that affect course completion rates. Of the 86 respondents, almost 70%…

Local, state, and national economies are facing unprecedented levels of international competition. The current fiscal crisis has hampered the ability of many governments in the developed world to directly facilitate economic growth. At the same time, many governments in the developing world are investing significant new resources into local…

Learning outcomes have become widely used in higher education, but also misused to the point of being controversial and a bureaucratic burden. This paper distinguishes three kinds of learning outcome found in current literature: (1) those used in individual teaching events; (2) those specified for modules or short courses; and (3) those specified…

For better or for worse, the assessment of research quality is one of the primary drivers of the behaviour of the academic community with all sorts of potential for distorting that behaviour. So, if you are going to assess research quality, how do you do it? This article explores some of the problems and possibilities, with particular reference to…

In large-scale assessments, such as state-wide testing programs, national sample-based assessments, and international comparative studies, there are many steps involved in the measurement and reporting of student achievement. There are always sources of inaccuracies in each of the steps. It is of interest to identify the source and magnitude of…

Nonrandom assignment of students to teachers can bias value-added estimates of teachers' causal effects. Rothstein (2008, 2010) shows that typical value-added models indicate large counterfactual effects of fifth-grade teachers on students' fourth-grade learning, indicating that classroom assignments are far from random. This article quantifies…

In this study, we describe a framework for monitoring rater performance over time. We present several statistical indices to identify raters whose standards drift and explain how to use those indices operationally. To illustrate the use of the framework, we analyzed rating data from the 2002 Advanced Placement English Literature and Composition…

"Voice" is widely considered to be a feature of effective writing. It's no surprise, then, that voice criteria frequently appear on rubrics used to score student essays in large-scale writing assessments. However, composition theorists hold vastly different views regarding voice and how it should be applied in the evaluation of student writing, if…

Although the Angoff procedure is among the most widely used standard setting procedures for tests comprising multiple-choice items, research has shown that subject matter experts have considerable difficulty accurately making the required judgments in the absence of examinee performance data. Some authors have viewed the need to provide…

A central commitment for professional development schools (PDSs) is to link preservice teacher preparation and in-service teacher professional development with improved learning outcomes for pupils. PDSs are expected to improve student achievement in two primary ways: (1) by enriching and intensifying the learning environment through professional…

In this article, we introduce a person-fit statistic called the hierarchy consistency index (HCI) to help detect misfitting item response vectors for tests developed and analyzed based on a cognitive model. The HCI ranges from -1.0 to 1.0, with values close to -1.0 indicating that students respond unexpectedly or differently from the responses…

In this commentary, the author addresses what is referred to as the deterministic input, noisy "and" gate (DINA) model. The author mentions concerns with how this model has been formulated and presented. In particular, the author points out that there is a lack of recognition of the confounding of profiles that generally arises and then discusses…

Rupp and Templin (2008) do a good job at describing the ever expanding landscape of Diagnostic Classification Models (DCM). In many ways, their review article clearly points to some of the questions that need to be answered before DCMs can become part of the psychometric practitioners toolkit. Apart from the issues mentioned in this article that…

In this commentary, the authors discuss some of the issues regarding the use of diagnostic classification models that practitioners should keep in mind. In the authors experience, these issues are not as well known as they should be. The authors then provide recommendations on diagnostic scoring.