This paper presents the concepts and observations in the author's keynote address at the May 2019 "Opportunity versus Challenge: Exploring Usage of Log-File and Process Data in International Large-Scale Assessments" conference in Dublin, Ireland. This paper recaps briefly some key points that emerged at the December 2018 ETS symposium on…

In 2005 the Hungarian school-leaving examination system underwent a significant transformation. In case of history the aim was to give a greater role to the development of students' knowledge acquisition and source analysis skills by more focusing on students' work with historical sources in classes. However, it was clear that the achievement of…

In the last twenty-five years, the discussion surrounding validity evidence has shifted both in language and scope, from the work of Messick and Kane to the updated Standards for Educational and Psychological Testing. However, these discussions haven't necessarily focused on best practices for different types of instruments or assessments, taking…

Drawing on Glenn Fulcher's extensive work in performance-based language assessment of speaking, this paper explores the assessment of L2 speaking ability in local language testing contexts. For that purpose, I review Fulcher's influential work that highlights the relationship between the speaking construct, the task, the performance, and the…

Successful exam composition can be a difficult task. Exams should not only assess student comprehension, but be learning tools in and of themselves. In a biotechnology course delivered to nonmajors at a business college, objective multiple-choice test questions often require students to choose the exception or "not true" choice. Anecdotal student…

Due to the high-stakes nature of medical exams it is prudent for test agencies to critically evaluate test data and control for potential threats to validity. For the typical multiple station performance assessments used in medicine, it may take time for examinees to become comfortable with the test format and administrative protocol. Since each…

This article is based on an experiment designed to conduct an interpreting test for multiple candidates online, using web-based synchronous cyber classrooms. The test model was based on the accreditation test for Professional Interpreters produced by the National Accreditation Authority of Translators and Interpreters (NAATI) in Australia.…

Business schools often attain accreditation to demonstrate program efficacy. J. A. Marts, J. D. Baker, and J. M. Garris (1988) hypothesized that candidates from Association to Advance Collegiate Schools of Business International (AACSB)-accredited accounting programs perform better on the CPA exam than do candidates from non-AACSB-accredited…

Some of the measurement issues encountered in the equating of performance assessments designed for use in teacher certification decisions are described. Analytic strategies based on examinee data that involve modification of existing procedures and judgmental strategies involving the judgments of experts to determine score equivalence are also…

Conceptual, practical, and logistical issues in the development of science performance assessments (SPAs) are discussed. The conceptual framework identifies task, response format, and scoring system as components, and conceives of SPAs as tasks that attempt to recreate conditions in which scientists work. Developing SPAs is a sophisticated effort…

Consistent with its mission of synthesizing available information on performance assessment in a variety of skill areas, the Clearinghouse for Applied Performance Assessment has prepared this summary of the role of performance assessment in selected published tests of problem-solving skill. Performance assessment is defined in terms of assessment…

If annual performance assessments are to yield results that can be compared from year to year, many technical problems must be addressed. It is essential that tests to be equated measure the same construct. Methods of equating performance assessment scores, ways of equating system assessments, and standard setting are discussed. (SLD)

A study was conducted to investigate: (1) the relationships between the results from various forms of assessment and the patterns of correlation across content areas; (2) how cognitive components correlate with the test results from different classroom assessments; and (3) how content areas affected the relationships. Data were collected from a…

The study described in this paper is part of an effort to improve understanding of the science assessment of the National Assessment of Educational Progress (NAEP). It involved the coding of all the items in the 1996 NAEP science assessments, which included 45 blocks (15 each for grades 4, 8, and 12) and over 500 items. Each of the approximately…

The idea that validity should be considered a property of inferences, rather than of assessments, has developed slowly over the past century. In early writings about the validity of educational assessments, validity was defined as a property of an assessment. The most common definition was that an assessment was valid to the extent that it…