A psychological framework for different types of items commonly used with mixed-format exams is proposed. A choice model based on signal detection theory (SDT) is used for multiple-choice (MC) items, whereas an item response theory (IRT) model is used for open-ended (OE) items. The SDT and IRT models are shown to share a common conceptualization…

Foundation of quality examination is based on the key features of validity and reliability of question papers. Question papers of examination boards in Pakistan--usually these features as contents of a question paper--are written by a single paper setter. To address these issues, Federal Board of Intermediate and Secondary Education (FBISE)…

In this article, we discuss the development and the administration of a multiple-choice test, which we named "Test of Calculus and Vectors in Mathematics and Physics" (TCV-MP), aimed at comparing students' ability to answer questions on derivatives, integrals, and vectors in a purely mathematical context and in the context of physics.…

Speeded tests set time limits so that few examinees can reach all items, and power tests allow most test-takers sufficient time to attempt all items. Educational achievement tests are sometimes described as "timed power tests" because the amount of time provided is intended to allow nearly all students to complete the test, yet this…

This classroom experiment investigates the effects of adding representational pictures to multiple-choice and constructed-response test items to understand the role of the response format for the multimedia effect in testing. Participants were 575 fifth- and sixth-graders who answered 28 science test items--seven items in each of four experimental…

Three mode comparability studies were conducted on the following Saturday national ACT test dates: October 26, 2019, December 14, 2019, and February 8, 2020. The primary goal of these studies was to evaluate whether ACT scores exhibited mode effects between paper and online testing that would necessitate statistical adjustments to the online…

Writing multiple-choice test items to measure student learning in higher education is a challenge. Based on extensive scholarly research and experience, the author describes various item formats, offers guidelines for creating these items, and provides many examples of both good and bad test items. He also suggests some shortcuts for developing…

This study examined indicators of cheating on a statewide summative assessment for grades 3--8 over a 4-year period. Between year 2 and year 3 of this study, the state launched an aggressive, highly publicized investigation into cheating by educators. The basis of the investigation was an erasure analysis. The current study found that the number…

The purpose of this study is to evaluate two methodological perspectives of test fairness using a national Secondary School Certificate (SSC) examinations. SSC is a suit of multi-subject national qualification tests at Grade 10 level in South Asian countries, such as Bangladesh, India, and Pakistan. Because it is a high-stakes test, the fairness…

The study investigated the impact of item position in multiple-choice test on student performance at the Basic Education Certificate Examination (BECE) level in Ghana. The sample consisted of 810 Junior Secondary School (JSS) Form 3 students selected from 12 different schools. A quasi-experimental design was used. The instrument for the project…

The purpose of this research was to determine the effect of formative testing and self-directed learning on mathematics learning outcomes. The research was conducted at an elementary school in central Jakarta during the 2014/2015 school year. Seventy-two fourth-grade students who were selected using random sampling participated in this study. Data…

Detecting unusual similarity in the item responses of a pair of examinees usually conditions on the pair's overall test performance (e.g., raw scores). Doing this, however, often requires assumptions about the invariance of other examinee pair characteristics. In this study, we examined the appropriateness of such assumptions about selected…

This study examines the use of subpopulation invariance indices to evaluate the appropriateness of using a multiple-choice (MC) item anchor in mixed-format tests, which include both MC and constructed-response (CR) items. Linking functions were derived in the nonequivalent groups with anchor test (NEAT) design using an MC-only anchor set for 4…

Significant momentum exists for next-generation assessments to increasingly utilize technology to develop and deliver performance-based assessments. Many traditional challenges with this assessment approach still apply, including psychometric concerns related to performance-based tasks (PBTs), which include low reliability, efficiency of…

In 2008-09, results from End-of-Grade (EOG) reading and mathematics tests in WCPSS continued to demonstrate an upward trend across grade levels and student subgroups. Disaggregation of results by ethnicity, income level, disability status, and English proficiency status showed that achievement gaps between historically underperforming subgroups…