Nonresponse bias is a widely prevalent problem for data on education. We develop a ten-step exemplar to guide nonresponse bias analysis (NRBA) in cross-sectional studies and apply these steps to the Early Childhood Longitudinal Study, Kindergarten Class of 2010-2011. A key step is the construction of indices of nonresponse bias based on proxy…

This case study provides an account of my experiences and lessons learned in designing and implementing a survey research. The research was part of my doctoral thesis in educational measurement and quantitative methods. The purpose of the study was to examine the effectiveness of a Completely Structured Training compared with Partially Structured…

Including all children in large-scale educational studies is a pressing concern. Omitting certain types of children from studies can lead to skewed findings that promote inaccuracies about learning levels or educational quality. Increasingly, assessments are a method for investigating the quality of education systems, but national assessments are…

An explanation of the misuse of statistical significance testing and the true meaning of "significance" is offered. Literature about the criticism of current practices of researchers and publications is reviewed in the context of tests of significance. The problem under consideration occurs when researchers attempt to do more than just establish…

A research psychologist and education consultant offers school board members some principles for appraising statistics: (1) distinguish between rates and numbers; (2) distinguish between ranks and scores; (3) try to "look through" the pictorial representation of data; (4) double check the data for accuracy; and (5) consider the source of the data.…

Demonstrates the extent to which school-level performance in the British General Certificate of Secondary Education (GCSE) program is underpinned by variations in the socioeconomic backgrounds of students. Concludes that uncontextualized performance statistics are flawed, and that policies for school improvement must address underlying constraints…

The purpose of this research study was to assess item response model-test data fit using residuals. First, a comparison of raw and standardized residuals for describing model-test data fit was carried out. Second, hypotheses concerning the relationship between residual sizes and several item characteristics were studied. The analyses with…

The major finding of the Law-Related Education Evaluation Project report for Year 1 (1981), that law-related education courses can reduce juvenile delinquency, is of limited use to educational decision makers and could be misleading. The research design leaves much to be desired; however, that fact must be considered in light of the difficulty of…

In a review of the challenges to progress in providing social research evidence that might usefully inform policy, Oakley (2004) argues strongly that the "paradigm divide" between qualitative and quantitative research communities continues to constitute a major problem. Oakley refers to a number of recent critiques of what is seen as "misplaced…

This paper reports 20 unsolved problems in educational measurement and points toward what seem to be promising avenues of solution. The first group of concerns involves validity, posing the problems of obtaining suitable validity criteria, determining and measuring the predictor variables that best characterize the traits and proficiencies of…

Efficient methods of using the Statistical Package for the Social Sciences (SPSS) to analyze National Assessment of Educational Progress (NAEP) data files are discussed. One error in the NAEP SPSS file is discussed, and another error (which may be system-dependent) is mentioned. In addition, purely mathematical methods are used to address the…

Illustrates problems facing researchers trying to demonstrate causal relationships between types of nonparental care and differences between groups of Swedish children. Argues that efforts must be made to validate and interpret differences that are found. Indicates ways to avoid misinterpretation of differences that are attributable to…

One of the overriding problems of any psychometric study is that aspects of behavior that are nonquantifiable or difficult to quantify are not considered. The other side of this coin is that those aspects of behavior that are easiest to quantify tend to be what are examined. In evaluating bilingual education programs, problems are found in the…

The norm-referenced evaluation model (RMC Model A) for Title I project evaluation, consists of procedures whereby the expected posttest standing of a treatment group under the null condition is generated from their pretest standing. It is assumed that the treatment group is not selected on the basis of their pretest scores and can be considered…

The methodological parameters of evaluative research studies help to identify a conceptualization of the phenomenon under investigation. Holistic critiques that identify the conceptual models under which the research was conducted shed light on the validity of the disparate inferences made from a number of studies of schools and school…