This study evaluated large-scale state writing assessments for the inclusion of motivational characteristics in the writing task and written prompt. We identified 6 motivational variables from the authentic activity literature: time allocation, audience specification, audience intimacy, definition of task, allowance for multiple perspectives, and…

Systematic evaluation is an essential tool for understanding program effectiveness. This article describes the pilot test of a statewide evaluation tool for the Supplemental Nutrition Assistance Program-Education (SNAP-Ed). A computer algorithm helped Community Nutrition Educators (CNEs) build surveys specific to their varied educational settings…

State-level testing programs continue to grow, and the challenge of validation does not wane. Although more than a decade has passed since the 1999 Joint Standards for Educational and Psychological Testing set out a call for the organization of validity evidence into validity arguments, practical examples of such arguments are not readily…

This study is an evaluation of the psychometric issues associated with estimating objective level scores, often referred to as "subscores." The article begins by introducing the concepts of reliability and validity for subscores from statewide achievement tests. These issues are discussed with reference to popular scaling techniques, classical…

In 1979-80, the Arkansas minimum competency tests were administered to a sample of 5,000 students in grades 3, 6, and 8. To determine how well test objectives matched the curriculum, their teachers estimated how many of the four items per objective a randomly selected student would answer correctly. Because chi square test comparisons of teacher…

A method to investigate consequential evidence of validity for a state assessment developed to change teacher instructional practices is presented. Survey responses from over 1,000 Kansas teachers were used to construct a path model that allowed effects of the state assessment to be studied at building and teacher levels. (SLD)

Security procedures for the New Jersey High School Proficiency Test (HSPT) are discussed and evaluated. All New Jersey high school students are required to pass the HSPT, which was administered for the first time in 1984. Generally, security plans are designed to limit access to test questions prior to test administration and to prevent…

Using data from state and academic reports, an independent committee of researchers has evaluated the Massachusetts Teacher Tests. Scores are found to be highly unreliable, and the tests are found to contain questionable content. Suspending use of the tests is recommended. (SLD)

Reports on an ongoing evaluation of the Vermont portfolio assessment program. Indicates that the positive news about the instructional effects of the assessment program are in contrast with the empirical findings about the quality of the data the program has yielded. (SLD)

The relationships between ratings on the Idaho Alternate Assessment (IAA) for 116 students with significant disabilities and corresponding ratings for the same students on two norm-referenced teacher rating scales were examined to gain evidence about the validity of resulting IAA scores. To contextualize these findings, another group of 54…

Focusing on the problems of assessing the speaking skills of secondary school students, this paper provides one example of how those problems were addressed in the Massachusetts speaking assessment. The paper identifies four requirements for measures of speaking skills: (1) feasibility, (2) reliability, (3) validity, and (4) freedom from bias. The…

The Connecticut Teacher Assessment Center Project has, since 1986, been developing a semistructured interview in the area of mathematics to evaluate beginning teacher competence. The strategy for validation of the project's performance tests, Connecticut's reform initiatives, and implications of systematic validity for traditional psychometric…

Key findings of a pilot study of the Alaska Instructional Diagnostic System (AIDS) are summarized. The AIDS pilot test served to verify the appropriateness of the skills survey as well as the validity and reliability of the items. The AIDS testing system includes three components: (1) upper level skills surveys (grades 3-8); (2) lower level skill…

High-stakes state-mandated testing programs are discussed, illustrating that proposed educational innovations are adopted because of their power as symbols of value orientations in the wider culture. In such programs, tests represent order and control, focus on important outcomes, and symbolize basic moral values. (SLD)

The use of confirmatory factor analytic procedures to examine the dimensionality of writing skills as measured by a large-scale direct writing test was illustrated. Internal construct validity evidence about the nature of writing skills measured by the test was provided. Data used were scores assigned by about 100 trained professional raters on a…