While score comparability across test forms typically relies on common (or randomly equivalent) examinees or items, innovations in item formats, test delivery, and efforts to extend the range of score interpretation may require a special data collection before examinees or items can be used in this way--or may be incompatible with common examinee…

This paper presents the concepts and observations in the author's keynote address at the May 2019 "Opportunity versus Challenge: Exploring Usage of Log-File and Process Data in International Large-Scale Assessments" conference in Dublin, Ireland. This paper recaps briefly some key points that emerged at the December 2018 ETS symposium on…

Much psychological research depends on participants' diligence in filling out materials such as surveys. However, not all participants are motivated to respond attentively, which leads to unintended issues with data quality, known as careless responding. Our question is: how do different modes of data collection--paper/pencil, computer/web-based,…

This study investigated the performance of four widely used data-collection designs in detecting test-mode effects (i.e., computer-based versus paper-based testing). The experimental conditions included four data-collection designs, two test-administration modes, and the availability of an anchor assessment. The test-level and item-level results…

There has been increasing interest in using of web-based surveys--rather than paper based surveys--for collecting data on alcohol and other drug use in middle and high schools in the US. However, prior research has indicated that respondent confidentiality is an underlying concern with online data collection especially when computer-assisted…

This article details the construction of a computer program to test cognitive processing differences in adolescents engaged in a standard presentation of tests versus a fantasy-based game presentation. The article will discuss the challenges of creating a replication of traditional psychological tests into a new medium which holds comparable…

A single group (SG) equating design with nearly equivalent test forms (SiGNET) design was developed by Grant (2006) to equate small volume tests. The basis of this design is that examinees take two largely overlapping test forms within a single administration. The scored items for the operational form are divided into mini-tests called testlets.…

The maximum incorporation of computer coding into an instrument is recommended to reduce errors in coding information from questionnaires. Specific suggestions for guiding the precoding process for response options, numeric identifiers, and assignment of card columns are proposed for mainframe computer data entry. (BS)

A quadratic curve test equating method for equating different test forms under a random-groups data collection design is proposed that equates the first three central moments of the test forms. When applied to real test data, the method performs as well as other equating methods. Procedures from implementing the test are described. (SLD)

In a study of questioning formats, children and adults answered spelling questions in an open-ended condition or one of two close-ended conditions where options were likely or unlikely. Participants presented with unlikely-response options generated their own responses more often than participants presented with likely-response options. Children…

The study investigated the structural corroboration of instructional evaluation information collected from one source (students) by three different methods: responses to objective questionnaire items, written comments to open-ended questions, and group interview results. The three types of information presented a similar general impression of…

A specific application of test equating is described, namely that of credentialing examination programs in the health professions. Considered are: (1) the role of test equating in the credentialing process; and (2) the issues that must be considered when implementing test equating in a credentialing examination program. (Author/LMO)

This report builds on and extends existent research on population invariance to new tests and issues. The authors lay the foundation for a deeper understanding of the use of population invariance measures in a wide variety of practical contexts. The invariance of linear, equipercentile and IRT equating methods are examined using data from five…

The purposes of this paper are to analyze some problems in using student test scores to evaluate teachers and to propose an achievement-based model for teacher evaluation that is effective, affordable, fair, legally defensible, and politically acceptable. The system is designed for detecting and documenting poor teacher performance. (Auth/JAZ)