Multilingual summative assessments face significant challenges due to tensions that exist between multiple language provision and comparability. Yet, conventional approaches for investigating comparability in multilingual assessments fail to accommodate assessments that comprise extended responses that target complex constructs. This article…

Digital technologies play an important role in our daily lives. Smartphones and tablet computers are very common worldwide and are available for everybody from a very early age. This trend offers the opportunity to track digital usage data for psychological and educational research purposes. The current paper introduces two research projects, the…

This article demonstrates how recent advances in technology allow fine-grained analyses of candidate-produced essays, thus providing a deeper insight on writing performance. We examined how essay features, automatically extracted using natural language processing and keystroke logging techniques, can predict various performance measures using data…

Consideration of socioemotional skills in admissions potentially can increase representation of racial and ethnic minorities and women in graduate and professional education as well as identify candidates more likely to succeed in graduate and professional school. Research on one such assessment, the ETS Personal Potential Index (PPI), showed that…

The construct of reading comprehension has changed significantly in the twenty-first century; however, some test designs have not evolved sufficiently to capture these changes. Specifically, the nature of literacy sources and skills required has changed (wrought primarily by widespread use of digital technologies). Modern theories of comprehension…

The amount and severity of security threats have increased considerably over the past two decades, calling into question the validity of assessments administered around the world. These threats have increased for a number of reasons, including the popular use of computerized and online technologies for test administration and the use of almost…

In this article, the characteristics of five test review models are described. The five models are the US review system at the Buros Center for Testing, the German Test Review System of the Committee on Tests, the Brazilian System for the Evaluation of Psychological Tests, the European EFPA Review Model, and the Dutch COTAN Evaluation System for…

The present research compares the operation of situational judgement and biodata measures between Chinese and U.S. respondents. We describe the development and past research on both measures, followed by hypothesized differences across the two groups of respondents. We base hypotheses on the nature of the Chinese and U.S. educational systems and…

The test review process used by the Buros Center for Testing is described as a series of 11 steps: (1) identifying tests to be reviewed, (2) obtaining tests and preparing test descriptions, (3) determining whether tests meet review criteria, (4) identifying appropriate reviewers, (5) selecting reviewers, (6) sending instructions and materials to…

Using an empirically-based simulation study, we show that typically used methods of choosing an item calibration sample have significant impacts on achievement bias and system rankings. We examine whether recent PISA accommodations, especially for lower performing participants, can mitigate some of this bias. Our findings indicate that standard…

The Reduced Reparameterized Unified Model (Reduced RUM) is a diagnostic classification model for educational assessment that has received considerable attention among psychometricians. However, the computational options for researchers and practitioners who wish to use the Reduced RUM in their work, but do not feel comfortable writing their own…

A critical component of the standard setting process is collecting evidence to evaluate the recommended cut scores and their use for making decisions and classifying students based on test performance. Kane (1994, 2001) proposed a framework by which practitioners can identify and evaluate evidence of the results of the standard setting from (1)…

Out-of-level testing refers to the practice of assessing a student with a test that is intended for students at a higher or lower grade level. Although the appropriateness of out-of-level testing for accountability purposes has been questioned by educators and policymakers, incorporating out-of-level items in formative assessments for accurate…

This article sets the stage for the description of a variety of approaches to test reviewing worldwide. It describes the importance of test reviewing as a protection of the public and of society and also the benefits of this activity for test users, who must choose measures to use in particular situations with particular clients at a particular…

In 2006, a (new) German standard for test reviewing was passed (Testkuratorium, 2006). There was already a European standard in place (European Federation of Psychologists' Associations, 2008). This article presents the German standard for test reviewing and explains how the German test review system was derived from demands in the German standard…