Certification organizations aim to assess candidates on their breadth and depth of knowledge to determine eligibility for certification in their field of specialty. Assessments used for certification, when appropriately constructed, should use questions (or items) that assess the entirety of the field. However, comparing the plethora of the…

Test writing is a fundamental component of teaching. With increasing pressure to teach larger groups of students, conduct formal assessment of learning outcomes, and offer online and hybrid classes, there is a need for alternatives to constructed response problem-solving test questions. We believe that appropriate use of multiple-choice (MC)…

Multiple-choice quizzes (MCQs) are a popular form of assessment. A rapid shift to online assessment during the COVID-19 pandemic in 2020, drove the uptake of MCQs, yet limited invigilation and wide access to material on the internet allow students to solve the questions via internet search. ChatGPT, an artificial intelligence (AI) agent trained on…

The recent integration of visual capabilities into Large Language Models (LLMs) has the potential to play a pivotal role in science and technology education, where visual elements such as diagrams, charts, and tables are commonly used to improve the learning experience. This study investigates the performance of ChatGPT-4 Vision, OpenAI's most…

This study explored the impact of homogeneity of answer choices on item difficulty and discrimination. Twenty-two matched pairs of elementary and secondary mathematics items were administered to randomly equivalent samples of students. Each item pair comparison was treated as a separate study with the set of effect sizes analyzed using…

In this quasi-experimental study, the effects of lexical coverage through pseudo word manipulation in dialogue comprehension are investigated. Forty-four first-year students in a Japanese university listened to five dialogues at different lexical coverage levels: 98%, 95%, 90%, 85%, and 83%. The results of the comprehension tests confirm the…

Measurement instruments of early childhood teachers' mathematical pedagogical content knowledge (MPCK) have to consider the special characteristics of early childhood teaching. Early childhood teaching includes some planned activities but in contrast to learning in school, it is often motivated and generated by situations which unfold…

This three-paper dissertation explores item cluster-based assessments, first in general as it relates to modeling, and then, specific issues surrounding a particular item cluster-based assessment designed. There should be a reasonable analogy between the structure of a psychometric model and the cognitive theory that the assessment is based upon.…

The purpose of this study is to describe a methodology to recover the item model used to generate multiple-choice test items with a novel graph theory approach. Beginning with the generated test items and working backward to recover the original item model provides a model-based method for validating the content used to automatically generate test…

Writing multiple-choice test items to measure student learning in higher education is a challenge. Based on extensive scholarly research and experience, the author describes various item formats, offers guidelines for creating these items, and provides many examples of both good and bad test items. He also suggests some shortcuts for developing…

According to the model of constructive alignment, learners adjust their learning strategies to the announced assessment (backwash effect). Hence, when teaching for understanding, the assessment method should be aligned with this teaching goal to ensure that learners engage in corresponding learning strategies. A quasi-experimental field study with…

Providing two or more versions of multiple-choice exams has long been a popular strategy for reducing the opportunity for students to engage in academic dishonesty. While the results of studies comparing exam scores under different question-order conditions have been inconclusive, the potential importance of contextual cues to aid student recall…

Grading can be accelerated to make time for more effective instruction. This article presents specific time management strategies selected to decrease administrative time required of faculty and teaching assistants, including a multiple answer multiple choice interface for exams, a three-tier grading system for open ended problem solving, and a…

The National Assessment of Educational Progress (NAEP) is the only continuing and nationally representative measure of trends in academic achievement of U.S. elementary and secondary school students in various subjects. For more than four decades, NAEP assessments have been conducted periodically in reading, mathematics, science, writing, U.S.…

This paper reports the development of a fixed-choice test for the assessment of basic knowledge in psychology, for use with undergraduate as well as graduate students. Test content is selected based on a core concepts approach and includes a sample of concepts which are indexed most frequently in common introductory psychology textbooks. In a…