The consistency of ratings between two sections of a course taught by an instructor within a given semester was studied by examining sources of variation in rating instructor/course obtained through a college teacher evaluation system. Data were analyzed for five business school instructors at a Sogang University in Seoul (South Korea). Each…

The effectiveness of the freshman admissions prediction equation (FAPE) at the University of Georgia (Athens) was studied, using data for 3,378 freshman (1,490 males and 1,888 females) in 1987-88. For the 1987-88 data, a previous (1982) prediction equation functioned poorly in terms of predictive accuracy. New prediction models were constructed…

A loglinear item response theory (IRT) model is proposed that relates polytomously scored item responses to a multidimensional latent space. Each item may have a different response function where each item response may be explained by one or more latent traits. Item response functions may follow a partial credit model (D. Andrich, 1978; and G. N.…

Several models for optimizing incomplete sample designs with respect to information on the item parameters are presented. The following cases are considered: (1) known ability parameters; (2) unknown ability parameters; (3) item sets with multiple ability scales; and (4) response models with multiple item parameters. The models are able to cope…

Use of the bootstrap resampling technique (BRT) is assessed in its application to resampling analysis associated with measurement of payment allocation errors by federally funded Family Assistance Programs. The BRT is applied to a food stamp quality control database in Oregon. This analysis highlights the outlier-sensitivity of the…

This article examines students' ability to use the balance model to solve for unknowns. A teaching experiment was conducted in four Year 3 classrooms. This experiment focused on exploring the application of the balance model as an analogue for representing equations and solving for unknowns. The teaching experiment promoted a shift by students…

A popular method of analyzing test items for differential item functioning (DIF) is to compute a statistic that conditions samples of examinees from different populations on an estimate of ability. This conditioning or matching by ability is intended to produce an appropriate statistic that is sensitive to true differences in item functioning,…

A two-dimensional, compensatory item response model and a unidimensional model were fitted to the reading and mathematics items in the Woodcock-Johnson Psycho-Educational Battery-Revised for a sample of 1,000 adults aged 20-39 years. Multidimensional information theory predicts that if the unidimensional abilities can be represented as vectors in…

Analyses for results of a series of studies examining intercorrelations among a set of as many as p+1 variables are presented. Several estimators of a pooled or average correlation vector and its variances are derived for cases in which some studies do not report complete correlation matrices. A test of the homogeneity (consistency) of the…

The phenomenon of simultaneous differential item functioning (DIF) amplification and cancellation and the role of the SIBTEST computer program in detecting it were studied. A variety of simulated test data was generated for this purpose. In addition, the following real test data were used: (1) American College Testing program data for 2,115 males…

One of the most frequently used research methods in education and psychology involves repeated observations on the same individuals. When sample sizes are relatively small and a multivariate analysis lacks power, there are currently two analytical options in testing time effects. One is to assume a time series structure to these observations, and…

The rule space model permits measurement of cognitive skill acquisition, diagnosis of cognitive errors, and detection of the strengths and weaknesses of knowledge possessed by individuals. Two ways to classify an individual into his or her most plausible latent state of knowledge include: (1) hypothesis testing--Bayes' decision rules for minimum…

The validity of the assumptions under which the ordinal true score test theory was derived was examined using (1) simulation based on classical test theory; (2) a long empirical test with data from 321 sixth graders; and (3) an extensive simulation with 480 datasets based on the 3-parameter model. (SLD)

Some unresolved methodological and philosophical issues in the assessment of performance trends are discussed as they relate to the National Assessment of Educational Progress (NAEP). How to measure performance change while remaining responsive to advances in curriculum and the technology of assessment is a central dilemma. (SLD)

The difference between item bias and item impact and the way they relate to item validity are discussed from a multidimensional item response theory perspective. The Mantel-Haenszel procedure and the Simultaneous Item Bias strategy are used in a Monte Carlo study to illustrate detection of item bias. (SLD)