A latent trait method is presented to investigate the possibility that Angoff or Nedelsky judges specify inconsistent probabilities in standard setting techniques for objectives-based instructional programs. It is suggested that judges frequently specify a low probability of success for an easy item but a large probability for a hard item. The…

The RMC Research Corporation evaluation model C1--the special regression model (SRM)--was evaluated through a series of computer simulations and compared with an alternative model, the norm referenced model (NRM). Using local data and national norm data to determine reasonable values for sample size and pretest posttest correlation parameters, the…

Accurate estimation of variance components used in generalizability theory is essential for the theory to be viewed as an efficacious mechanism for studying the reliability and validity of a measurement procedure. This paper explores two alternatives for dealing with the apparent instability of small sample size used in determining the accuracy of…

The implications of a mathematical model of test scores are explored where the data are limited to a random sample of items without replacement from an indefinitely large population or item domain in which items are scored either zero or one. The purpose is to obtain an unbiased estimate of a student's proportion of items correct in the item…

A general method for estimating the unknown coefficients in a set of linear structural equations is described. In its most general form the method allows for both errors in equations (residuals, disturbances) and errors in variables (errors of measurement, observational errors) and yields estimates of the residual variance-covariance matrix and…

The 1987-88 Schools and Staffing Survey (SASS) looked at a national sample of elementary and secondary public and private schools. Components of the SASS, a data collection effort of the National Center for Education Statistics, were surveys of teacher demand and shortage, schools, administrators, and teachers. Approximately 13,000 schools and…

The percentage of students at/above a cut point (PAAC) is one of the most common measures used for reporting school-level performance relative to a proficiency standard (L. Cronbach, N. Bradburn, and D. Horvitz, 1994). The two purposes of this study were to introduce procedures for estimating standard errors for school PAACs under a…

The purpose of this research was to study the effect of the three different ways of increasing the number of points contributed by constructed response (CR) items on the reliability of test scores from mixed-item-format tests. The assumption of unidimensionality that underlies the accuracy of item response theory model-based standard error…

In this study we test the performance of some nonexperimental estimators of impacts applied to an educational intervention--reduction in class size--where achievement test scores were the outcome. We compare the nonexperimental estimates of the impacts to "true impact" estimates provided by a random-assignment design used to assess the…

Contends that the nature of physics has been misrepresented by blurring or ignoring important distinctions between "errors" and "discrepancies" and that dealing with these and related problems can improve students' enjoyment of labs and understanding of physics. Nature of physics, role of experiments, experimental errors, and error analysis are…

Willmott's comparison of General Certificate of Education (GCE) scores in 1968 and 1973 is reexamined. The trend toward an increasing percentage of students who pass is confirmed, but estimates of standard errors indicate that subtest differences may be attributed to the sampling plan. (CP)

The stability of scores on 2 types of performance assessments, an observed hands-on investigation and a notebook surrogate, was investigated for 29 sixth graders on 2 occasions. Results indicate that student performance and procedures changed and that generalizability across occasions was moderate. Implications for assessment are discussed. (SLD)

Addresses three issues concerning the assessment of the overrating and underrating of academic competence: (1) the impossibility of separating effects of overrating and underrating from effects of perceived and actual competence; (2) the questionable validity of Connell and Ilardi's method; and (3) the proposal of a new method and its implications…

Type I error rates of four multivariate tests (Pilai-Bartlett trace, Johansen's test, James' first-order test, and James' second-order test) were compared for heterogeneous covariance matrices in 360 simulated experiments. The superior performance of Johansen's test and James' second-order test is discussed. (SLD)

The ability of 3-way INDSCAL and ALSCAL models to recover true structure in proximity data based on 2-dimensional configurations varying in number of subjects (15 and 20) and stimuli, amount of error, and monotonic transformation is examined. INDSCAL outperformed metric and nonmetric ALSCAL in all conditions. (SLD)