An instrument for to assessing the extent to which postsecondary vocational education programs are competency-based consisted of 159 items grouped into 8 subscales corresponding to 4 standards: (1) identification and validation of competencies; (2) procedures for assessing student competencies; (3) instructional procedures; and (4) administrative…

The purpose of this paper is to link empirical Bayes methods with two specific topics in item response theory--item/subtest regression, and testing the goodness of fit of the Rasch model--under the assumptions of local independence and sufficiency. It is shown that item/subtest regression results in empirical Bayes estimates only if the Rasch…

A procedure used to average several Mantel-Haenszel delta difference values for an item is described and evaluated. The differential item functioning (DIF) procedure used by the Educational Testing Service (ETS) is based on the Mantel-Haenszel statistical technique for studying matched groups. It is standard procedure at ETS to analyze test items…

A nominal response model-based computerized adaptive testing procedure (nominal CAT) was implemented using simulated data. Ability estimates from the nominal CAT were compared to those from a CAT based upon the three-parameter logistic model (3PL CAT). Furthermore, estimates from both CAT procedures were compared with the known true abilities used…

The Florida Statewide Assessment Program tests students' achievement of basic skills in reading, writing, and mathematics at grades 3, 5, 8, and 10 on the State Student Assessment Test-Part I (SSAT-I). In addition, high school students are tested on practical math and reading tasks on the State Student Assessment Test-Part II (SSAT-II). This…

Several investigators, using the Bem Sex Role Inventory (BSRI), have concluded that masculine is the desired sex role of males and females. Other investigators have suggested that some of the feminine BSRI items are undesirable. In order to determine if this masculine ideal might be due to the nature of the BSRI feminine items (more undesirable…

Factor analysis is the traditional method for studying the dimensionality of test data. However, under common conditions, the factor analysis of tetrachoric correlations does not recover the underlying structure of dichotomous data. The purpose of this paper is to demonstrate that the factor analyses of tetrachoric correlations is unlikely to…

Information on statistical detection of two different kinds of unethical activity associated with standardized testing is presented. The first is advance presentation of specific test items or excessive coaching using items very similar to those on a test. Cluster analysis is recommended for detection of aberrant vectors of percentages correct on…

The most appropriate statistical model for the small-scale (n<100) studies common in language testing research is the Rasch one-parameter logistic model. The Rasch model provides a wide range of options for conducting research, refining existing examinations, and developing tailored (computerized adaptive) language tests. Three investigations…

The present monte carlo simulation study was designed to examine the effects of multidimensionality during the administration of computerized adaptive testing (CAT). It was assumed that multidimensionality existed in the individuals to whom test items were being administered, i.e., that the correct or incorrect responses given by an individual…

How accurately deltas (statistics measuring the difficulty of items) established by pre-test populations reflect deltas obtained from final form populations, and the consequent utility of pre-test deltas for constructing final (operational test) forms to meet developed statistical specifications were studied. Data were examined from five subject…

The agreement between item response theory-based and Mantel Haenszel (MH) methods in identifying biased items on tests was studied. Data came from item responses of four spaced samples of 1,000 examinees each--two samples of 1,000 Anglo-American and two samples of 1,000 Native American students taking the New Mexico High School Proficiency…

Diagnostic testing can provide specific information about student skills as a decision-making aid to teachers in prescribing instruction, identifying needs for remediation, determining effective instructional materials and methods, and ultimately, improving student learning. Diagnostic testing, as viewed here, includes individual and group…

The successful Wisconsin Title 1 project item bank offers a valid, flexible, and efficient means of providing migrant student tests in reading and mathematics tailored to instructor curricula. The item bank system consists of nine PASCAL computer programs which maintain, search, and select from approximately 1,000 test items stored on floppy disks…

Teacher questions have been the subject of a substantial amount of research and analysis in recent years. Debate has centered around the appropriate use of this instructional and evaluative tool. A study examined the impact on student achievement of manipulating abstractness-concreteness, independent of cognitive level, in achievement test items.…