The purpose of this study is to further enhance the existing reliability and validity evidence related to the School Implementation Scale (SIS) and its use for evaluating the implementation of different integrated academic and behavior tiered Response-To-Intervention (RTI) models. Previous methodology conducted for validation of the instrument has…

Community-based education programs must demonstrate effectiveness to various funding sources. The pilot study reported here (funded by CYFAR, NIFA, USDA award #2008-41520-04810) had the goal of determining if state level programs with varied curriculum could use a common evaluation tool to demonstrate efficacy. Results in parenting and youth…

The long-term goal of the State Systemic Improvement Plan (SSIP) and other federal and state early intervention and early childhood education initiatives is improved child and family outcomes. States play a critical role in supporting practitioners in the use of evidence-based practices to improve child and family outcomes. When practitioners…

The ABC Quality program, South Carolina's Quality Rating and Improvement System (QRIS), is a statewide system designed to improve the quality of early care and education services. Historically, two observation instruments are used for centers that elect to participate in the state's voluntary QRIS depending on the level of quality for which they…

Systematic evaluation is an essential tool for understanding program effectiveness. This article describes the pilot test of a statewide evaluation tool for the Supplemental Nutrition Assistance Program-Education (SNAP-Ed). A computer algorithm helped Community Nutrition Educators (CNEs) build surveys specific to their varied educational settings…

This article proposes that sampling design effects have potentially huge unrecognized impacts on the results reported by large-scale district and state assessments in the United States. When design effects are unrecognized and unaccounted for they lead to underestimating the sampling error in item and test statistics. Underestimating the sampling…

State-level testing programs continue to grow, and the challenge of validation does not wane. Although more than a decade has passed since the 1999 Joint Standards for Educational and Psychological Testing set out a call for the organization of validity evidence into validity arguments, practical examples of such arguments are not readily…

This study is an evaluation of the psychometric issues associated with estimating objective level scores, often referred to as "subscores." The article begins by introducing the concepts of reliability and validity for subscores from statewide achievement tests. These issues are discussed with reference to popular scaling techniques, classical…

This study evaluated large-scale state writing assessments for the inclusion of motivational characteristics in the writing task and written prompt. We identified 6 motivational variables from the authentic activity literature: time allocation, audience specification, audience intimacy, definition of task, allowance for multiple perspectives, and…

As part of a Response to Intervention (RTI) model, curriculum-based measurement (CBM) is an evidence-based assessment system that uses screening and formative assessments to help guide instruction. The two primary purposes of this study are to examine the relations between the easyCBM benchmark measures and a statewide large-scale assessment and…

To assist administrators, teachers, and other school personnel in understanding and implementing the various components of the Indiana Statewide Testing for Educational Progress (ISTEP) program, this manual details the ISTEP program, a broad-based achievement testing plan providing Indiana schools with a means of assessing their overall…

The purpose of this report is to provide summary information regarding the evaluation of the Early Prevention of School Failure Project (EPSFP). Initially funded by the state of Illinois, the EPSFP encourages school districts to identify and remediate deficiencies in students which appear to be related to early school failure. Immediately…

State directors of teacher certification and testing provided updated data (reports, press clippings, manuals, legislative hearings, and validity studies) in April 1987 describing their testing programs and the latest passing rates. Twenty-seven states have admissions testing programs, and 44 have certification testing programs. It is unclear…

Reports on an ongoing evaluation of the Vermont portfolio assessment program. Indicates that the positive news about the instructional effects of the assessment program are in contrast with the empirical findings about the quality of the data the program has yielded. (SLD)

The 1992-93 school year saw the second statewide implementation of the Vermont portfolio-assessment program, and RAND continued its ongoing evaluation of the program's implementation, effects, and data quality. While the first year's study found evidence of the impact of the assessment program and low reliability of portfolio scoring, this year's…