The National Assessment of Educational Progress (NAEP) Validity Studies Panel undertook a systematic analysis to consider the domain of validity threats to NAEP and to identify the most urgent research priorities. A framework of six broad categories was developed: (1) the constructs measured within each of NAEP's subject domains; (2) the manner in…

Describes and assesses missing data methods currently used to analyze data from matrix sampling designs implemented by the National Assessment of Educational Progress. Several improved methods are developed, and these models are evaluated using an EM algorithm to obtain maximum likelihood estimates followed by multiple imputation of complete data…

Model-based methods for estimating the population mean in stratified clustered sampling are described. The importance of adjusting the weights is assessed by an approach considering the sampling variation of the adjusted weights and its (variance) components. The resulting estimators are more efficient than the jackknife estimators for a variety…

States participating in the National Assessment of Educational Progress State Assessment program (state NAEP) are required to sample at least 2,500 students from at least 100 schools per subject assessed. In this ideal situation, 25 students are assessed for a subject in each school selected for that subject. Two problems have arisen: some states…

The use of sampling weights in deriving population estimates for the National Assessment of Educational Progress (NAEP) and the effects of nonresponse and undercoverage on those estimates are described. The estimation of sampling variability from complex sample surveys is also reviewed, concentrating on the jackknife repeated replication…

The effects of certain characteristics (e.g., sample design) of National Assessment of Educational Progress (NAEP) data on statistical analysis techniques are considered. Ignoring special features of NAEP data and proceeding with a standard analysis can produce inferences that underestimate the true variability and overestimate the true degrees of…

Several issues related to the design and reporting of results from the National Assessment of Educational Progress (NAEP) are discussed in the context of current expectations for the NAEP and its origins. These issues include: (1) content coverage and format; (2) estimation procedures; and (3) reporting problems. (SLD)

Political and legislative pressures have posed a number of measurement issues and challenges to the development of sound, valid voluntary national tests (VNTs). This paper focuses on what appear to be the most difficult technical issues related to the VNT proposed by President Clinton in 1997. Technical issues refer to psychometric issues, as…

In 1988, whether the National Assessment of Educational Progress (NAEP)--the "Nation's Report Card"--could become a report card for the states was studied. A trial assessment program was authorized to determine whether state assessments following the NAEP format could produce reliable and useful estimates of educational progress. This…

This Monte Carlo study examined the effect of complex sampling of items on the measurement of differential item functioning (DIF) using the Mantel-Haenszel procedure. Data were generated using a three-parameter logistic item response theory model according to the balanced incomplete block (BIB) design used in the National Assessment of Educational…

Concepts behind plausible values in estimating population characteristics from sparse matrix samples of item responses are discussed. The use of marginal analyses is described in the context of the National Assessment of Educational Progress, and the approach is illustrated with Scholastic Aptitude Test data for 9,075 high school seniors. (SLD)

Procedures for obtaining student samples for the National Assessment of Educational Progress (NAEP) and deriving survey weights for analysis of survey data are described. Sample designs are economically and operationally feasible, and weighting procedures result in increased precision of estimates as they account for the probabilities of student…

A review of the National Assessment of Educational Progress (NAEP), which originated 20 years ago, is presented; and the original political, ideological and social science assumptions for the design of NAEP are examined. The report criticizes the founding principle as hampering NAEP's maximum utility and exploratory power. In examining the form of…

This report presents an investigation by the American College Testing Program (ACT) of an alternative design for the National Assessment of Educational Progress (NAEP). The proposed design greatly simplifies the data collection and analysis procedures needed to produce assessment results and has the potential to produce results that are more…

This report documents the design, administration, and data analysis procedure of the National Assessment of Education Progress (NAEP) for 1996. It indicates the technical decisions that were made and the rationale behind them. Detailed substantive findings are not presented in this report. These chapters provide technical information about the…