This paper presents the concepts and observations in the author's keynote address at the May 2019 "Opportunity versus Challenge: Exploring Usage of Log-File and Process Data in International Large-Scale Assessments" conference in Dublin, Ireland. This paper recaps briefly some key points that emerged at the December 2018 ETS symposium on…

The history and evolution of science assessment remains poorly known, especially in the context of the exam question contents. Here we analyze the Finnish matriculation examination in biology from the 1920s to 1960s to understand how the exam has evolved in both its knowledge content and educational form. Each question was classified according to…

School systems across the country are transitioning from paper-based testing (PBT) to computer-based testing (CBT). As this technological shift occurs, more research is necessary to understand the practical and performance implications of administering CBTs. Currently, there is a paucity of research using CBTs to examine working memory (WM)…

Two experiments investigated the processes underlying the picture superiority effect on recognition memory. Studied pictures were associated with higher accuracy than studied words, regardless of whether test stimuli were words (Experiment 1) or pictures (Experiment 2). Event-related brain potentials (ERPs) recorded during test suggested that the…

The Wisconsin Card Sorting Test (WCST) assesses executive and frontal lobe function and can be administered manually or by computer. Despite the widespread application of the 2 versions, the psychometric equivalence of their scores has rarely been evaluated and only a limited set of criteria has been considered. The present experimental study (N =…

This article examines the influence of keyboarding versus handwriting in a high-stakes writing assessment. Conclusions are based on data collected from a pilot project to move Old Dominion University's Exit Exam of Writing Proficiency from a handwritten format into a dual-option format (i.e., the students may choose to handwrite or keyboard the…

Constructed-response formats are desired for measuring complex and dynamic response processes that require the examinee to understand the structures of problems and micro-level cognitive tasks. These micro-level tasks and their organized structures are usually unobservable. This study shows that elementary graph theory is useful for organizing…

A study was conducted to investigate: (1) the relationships between the results from various forms of assessment and the patterns of correlation across content areas; (2) how cognitive components correlate with the test results from different classroom assessments; and (3) how content areas affected the relationships. Data were collected from a…

Contrasts between constructed response items and stem-equivalent multiple-choice counterparts typically have involved averaging item characteristics, and this aggregation has masked differences in statistical properties at the item level. Moreover, even aggregated format differences have not been explained in terms of differential cognitive…

This paper discusses the design and pedagogy underpinning the use of online quiz items in which students are presented with a range of questions designed to enhance understanding of complex linguistic constructions. It explores the design of different types of quiz questions from the perspective of pedagogy and cognitive demand. The particular…

The history of the Educational Testing Service (ETS) Factor Kits is summarized. The original ETS Factor Kit was developed in 1954 and contained 51 items, three each for each of 15 factors and six for a 16th factor. The next edition was developed in 1963 and included adaptations (clones) of the defining tests instead of the exact copies. These…