The Treatment Integrity Planning Protocol (TIPP) provides a structured process for collaboratively creating a treatment integrity assessment within a consultation framework. The authors evaluated the effect of the TIPP on the implementation of an intervention designed to improve the consistency of students' mathematics performance. Treatment…

In large-scale assessments, such as state-wide testing programs, national sample-based assessments, and international comparative studies, there are many steps involved in the measurement and reporting of student achievement. There are always sources of inaccuracies in each of the steps. It is of interest to identify the source and magnitude of…

Scientifically-based child assessment tools, observation, screening, portfolios, individualized research-based curriculum are now requirements for accreditation by the NAEYC. Head Start has set them as expectations in their performance standards. And increasingly, entities that fund early childhood programs are looking to the use of child…

Although the Delphi is widely used, research on certain methodological issues is somewhat limited. After a brief introduction to the strengths, limitations, and methodological challenges of the technique, we share our experiences (as well as problems encountered) with an electronic Delphi of educational program evaluation (EPE) in the Asia-Pacific…

This article offers an account of a doctoral insider research project that became problematic. The project was investigating mathematics teaching in a university in the UK, and by contrasting the research account with research diary entries pertaining to two interviewees, different interpretations of the interview data are evident. These…

In this study, we describe a framework for monitoring rater performance over time. We present several statistical indices to identify raters whose standards drift and explain how to use those indices operationally. To illustrate the use of the framework, we analyzed rating data from the 2002 Advanced Placement English Literature and Composition…

This paper discusses an instructional approach designed to help preservice teachers understand how assessments can be influenced by personal biases. In order to achieve this objective, we developed an analogy-based activity called "The Dog Show Analogy." After participating in the activity, we have observed that the participating preservice…

One critical issue arising in the educational use of collaborative learning concerns the teacher's difficulty in evaluating the contribution and participation of each student in group-work. This article aims to illustrate and discuss a methodology that enables evaluation of the collaborative learning process based on co-writing in a wiki…

There is already a growing literature on assessment and an emergent scholarship that people in the scholarship of teaching and learning (SoTL) community need both to promote and to build on if assessment is to be a scholarly activity. Awareness alone, however, is unlikely to be enough. The poor practice highlighted in this article is not simply…

Student evaluations should "be ethical, fair, useful, feasible, and accurate" [JCSEE (2003). "The student evaluation standards." Arlen Gullickson, Chair. Thousand Oaks, CA: Corwin]. This study focuses on defining ethical behavior and examining educators' ethical judgments in relation to assessment. It describes the results from…

In April 2007, more than 50 practitioners, advocates, funders, and evaluators met for 2 days in Pittsburgh, Pennsylvania, to discuss the issues that arise in evaluating systems change or systems-building initiatives and to suggest approaches to resolving those issues. The symposium was designed to be as realistic and practical as possible by…

"Voice" is widely considered to be a feature of effective writing. It's no surprise, then, that voice criteria frequently appear on rubrics used to score student essays in large-scale writing assessments. However, composition theorists hold vastly different views regarding voice and how it should be applied in the evaluation of student writing, if…

"Assessment on college campuses has a sordid history, and it is fairly simple to find someone with a traumatic tale to tell. It is wise to respect that that reputation is deserved." "How do you modify the inner workings and culture of a massive institution with minimal resources and even less authority (other than GE course…

What is "fairness" in the context of educational assessment? I apply this question to a number of contemporary educational assessment practices and policies. My approach to philosophy of education owes much to Wittgenstein. A commentary set apart from the main body of the paper focuses on my style of philosophising. Wittgenstein teaches us to…

In 2002, voters in Florida approved a constitutional amendment limiting class sizes in public schools to 18 students in the elementary grades, 22 students in middle grades, and 25 in high school grades. Analyzing statewide achievement data for school districts from 2004-2006 and for schools in 2007, this study purports to find that "mandated…