Item compromise has long posed challenges in educational measurement, jeopardizing both test validity and test security of continuous tests. Detecting compromised items is therefore crucial to address this concern. The present literature on compromised item detection reveals two notable gaps: First, the majority of existing methods are based upon…

A dozen different approaches that yield 13 reliability indices for criterion-referenced tests were identified and grouped into three categories: threshold loss function, squared-error loss function, and domain score estimation. Indices were evaluated within each category. (Author/RL)

Presents a job aid for determining test-item format called TIFAID (Test Item Format Job Aid), based on adequately constructed instructional objectives. The four sections of the job aid are described: (1) a task classification system; (2) task-related questions; (3) a flowchart; and (4) a tips and techniques guide. (Contains four references.) (LRW)

Three studies explored the effects of grouping versus randomized items in questionnaires on internal consistency and test-retest reliability with samples of 80, 80, and 100, respectively, university students and undergraduates. The 2 correlational and 1 experimental studies were reasonably consistent in demonstrating that neither format was…

Provides a framework for understanding the concept of validity and how it applies to simulation research. Extant tests of validity are discussed in terms of this framework, and a general schema for testing validity is proposed. (MBR)

This study examined performance of a simulated computerized adaptive test that was designed to help direct the development of a medical recertification examination. The item pool consisted of 229 single-best-answer items from a random sample of 3,000 examinees, calibrated using the two-parameter logistic model. Examinees' responses were known. For…