Since Gisela Granena's influential work, LLAMA D v2, a sound recognition subtest of LLAMA aptitude tests, has been used as a measure of implicit learning aptitude in second language acquisition research. The validity of this test, however, is little known and the results of studies with this instrument have been somewhat inconsistent. In this…

The Hearts and Flowers (H&F) task is a computerized executive functioning (EF) assessment that has been used to measure EF from early childhood to adulthood. It provides data on accuracy and reaction time (RT) across three different task blocks (hearts, flowers, and mixed). However, there is a lack of consensus in the field on how to score the…

Understanding the cognitive processes, skills and strategies that examinees use in testing is important for construct validity and score interpretability. Although response processes evidence has long been included as an important aspect of validity (i.e., "Standards for Educational and Psychological Tests," 1999), relevant studies are…

As the use of process data in large-scale educational assessments is becoming more common, it is clear that data on examinees' test-taking behaviors can illuminate their performance, and can have crucial ramifications concerning assessments' validity. A thorough review of the literature in the field may inform researchers and practitioners of…

Research on methods for measuring examinee engagement with constructed-response items is limited. The present study used data from the PISA 2018 Reading domain to construct and compare indicators of test-taking effort on constructed-response items: response time, number of actions, the union (combining effortless responses detected by either…

The reaction time-based concealed information test (RT-CIT) has been used to judge the veracity of an examinees claim to be naïve by using RTs to test for recognition of relevant details. Here, we explore the validity of the RT-CIT to generate new knowledge about the incident--the searching CIT. In a mock terrorism study (n = 60) the RT-CIT not…

Challenges to measuring students' attitudes toward statistics remain despite decades of focused research. Measuring the expectancy-value theory (EVT) Cost construct has been especially challenging owing in part to the historical lack of research about it. To measure the EVT Cost construct better, this study asked university students to respond to…

The Assessment of Language in Adults using Self-reported Skills (ALASS) is an online tool aimed at providing a rapid indication of language-related skills at two levels: basic performance and social use. In Study 1, we have developed and validated a new tool with two objective measures: a lexical decision test and a grammaticality judgement test,…

When examinees perceive a test as low stakes, it is logical to assume that some of them will not put out their maximum effort. This condition makes the validity of the test results more complicated. Although many studies have investigated motivational fluctuation across tests during a testing session, only a small number of studies have…

Purpose: This study aimed to develop and test a measure of real-time continuous speech understanding to be used with natural dialogues. Method: The measure was based on a category monitoring paradigm and employed five existing recordings of natural dialogues from which the different test categories and associated target words were derived. For…

Touch screen tablets are being increasingly used in schools for learning and assessment. However, the validity and reliability of assessments delivered via tablets are largely unknown. The present study tested the psychometric properties of a tablet-based app designed to measure early literacy skills. Tablet-based tests were also compared with…

We developed and tested a behavioral version of the Defining Issues Test-1 revised (DIT-1r), which is a measure of the development of moral judgment. We conducted a behavioral experiment using the behavioral Defining Issues Test (bDIT) to examine the relationship between participants' moral developmental status, moral competence, and reaction time…

International large-scale assessments (ILSAs) transitioned from paper-based assessments to computer-based assessments (CBAs) facilitating the use of new item types and more effective data collection tools. This allows implementation of more complex test designs and to collect process and response time (RT) data. These new data types can be used to…

Background: The idea of using low-stakes assessment results is often mentioned when designing educational system reforms. However, when tests have no consequences for the students, test takers may not make enough effort when completing the test, and their lack of engagement may negatively affect the validity of the conclusions of the studies that…

The aim of this study is to propose advantages provided by computerized tools when assessing reading ability. A new computer-based reading assessment evaluating both word reading and reading comprehension processes was administered to 687 children in primary (N=400) and secondary (N=287) schools. Accuracy (weighted scores) and speed of access…