In the summer of 2011, the Tennessee Department of Education contracted with the National Institute for Excellence in Teaching (NIET) to provide a four-day training for all evaluators across the state. NIET trained more than 5,000 evaluators intensively in the state model (districts using alternative instruments delivered their own training).…

In this article, we introduce a person-fit statistic called the hierarchy consistency index (HCI) to help detect misfitting item response vectors for tests developed and analyzed based on a cognitive model. The HCI ranges from -1.0 to 1.0, with values close to -1.0 indicating that students respond unexpectedly or differently from the responses…

In this commentary, the author addresses what is referred to as the deterministic input, noisy "and" gate (DINA) model. The author mentions concerns with how this model has been formulated and presented. In particular, the author points out that there is a lack of recognition of the confounding of profiles that generally arises and then discusses…

Rupp and Templin (2008) do a good job at describing the ever expanding landscape of Diagnostic Classification Models (DCM). In many ways, their review article clearly points to some of the questions that need to be answered before DCMs can become part of the psychometric practitioners toolkit. Apart from the issues mentioned in this article that…

In this commentary, the authors discuss some of the issues regarding the use of diagnostic classification models that practitioners should keep in mind. In the authors experience, these issues are not as well known as they should be. The authors then provide recommendations on diagnostic scoring.

The state of teacher evaluation is a significantly and frequently neglected problem in public education, an enterprise that spends $400 billion annually on salaries and benefits. Because teacher evaluation is at the center of the quality of teaching in the nation's classrooms, it has the potential to be a powerful lever of teacher and school…

The Individuals with Disabilities Education Improvement Act of 2004 allows schools to use a child's response to research-based intervention (RTI) as a part of procedures to identify students with learning disabilities. This paper considers whether RTI-based assessment models meet ethical and legal standards for acceptable assessment practices.…

Purpose: This paper aims to explore how teaching and assessing reflective learning skills can support postgraduate practitioners studying organisational change and explores the challenges for tutors in assessing these journals. Design/methodology/approach: Assessment criteria were developed from the literature on reflective practice and…

This paper forms part of an exploration of assessment on one part-time higher education (HE) course: an in-service, professional qualification for teachers and trainers in the learning and skills sector which is delivered on a franchise basis across a network of further education colleges in the north of England. This paper proposes that the…

In this collection, evaluation theorists and practitioners consider evaluator role and identity in evaluation theory, practice, and society. The chapters are: (1) "Three Frameworks for Considering Evaluator Role" (Jean A. King and Laurie Stevahn); (2) "Toward Better Understanding of Alternative Evaluator Roles" (Melvin M. Mark); (3) "Making (More)…

Discusses ethical challenges faced by evaluators, especially those who have gotten into problems while conducting the evaluation. Introduces the other articles in this theme section. (SLD)

Notes that, very early in program planning, budgets are estimated for major program components, including evaluation. Proposes heuristic for establishing reasonable evaluation estimates in initial planning. Describes logical sources of information from which to construct such a heuristic. Recommends that, for typical programs and projects, initial…

Reflects on an evaluation that aspired to be inclusive but generally failed to provide a backdrop for a discussion of inclusive evaluation. Identifies issues of absence of significant stakeholders, making of values by method, and the limited authority of the evaluation. Shows how easily deliberative intentions are distorted. (SLD)

Challenges the assumption that evaluation should aim for fostering deliberative democracy. A modest effort by an evaluator to contribute to deliberative democracy is defensible, but vigorous political advocacy violates social expectation. (Author/SLD)

Whether there can be professional reasons for turning down an evaluation contract is explored. Considering how evaluations can promote either guild maintenance interests or societal improvement interests illuminates several dilemmas in the proper conduct of evaluation and suggests the need for greater explicit attention to what it means for the…