I consider recent attempts to establish standards, principles, and goals for artificial intelligence (AI) through the lens of educational measurement. Distinctions are made between generative AI and AI-adjacent methods and applications of AI in formative versus summative assessment contexts. While expressing optimism about its possibilities, I…

The past decade has seen an emerging interest in mapping (aligning or linking) test scores to language proficiency levels of external performance scales or frameworks, such as the Common European Framework of Reference (CEFR), as well as locally developed frameworks, such as China's Standards of English Language Ability (CSE). Such alignment is…

The 2018 Nebraska Student-Centered Assessment System (NSCAS) Summative technical report documents the processes and procedures implemented to support the Spring 2018 NSCAS Summative English Language Arts (ELA), Mathematics, and Science assessments by NWEA under the supervision of the Nebraska Department of Education (NDE). The technical report…

The Arizona English Language Learners Assessment (AZELLA) is used by the Arizona Department of Education to determine which children should receive English support services. AZELLA results are used to determine if children are either proficient in English or have English language skills in one of four pre-proficient categories (pre-emergent,…

Recently, there has been a significant expansion in the use of performance assessment in large scale testing programs. Although there has been significant support from curriculum and policy stakeholders, the technical feasibility of large scale performance assessments has remained a question. This report is intended to contribute to the debate by…

Influences on the educational validity of examinations are reviewed. Changes occurring in approaches to standard setting are traced. A view of reliability is presented, with emphasis on assessment of project work, which often involves individual investigation and design by students. A consistency index formula for grading standards is presented.…

The implicit definition of competence and the inferential chain that links the standard-setting process to the decision outcomes of the method are considered for two classes of standard-setting procedures: those involving data-free judgments of items and those involving data-based judgment of items. The major underlying assumptions of competence…

Two new indices of stability of content-referenced standard-setting results are presented, relating variability of judges' decisions to the variability of candidate scores and to the reliability of the test. These indices are used to indicate whether scores resulting from a standard-setting study are of sufficient precision. (Author/LMO)

A brief history of developments in the assessment of the reliability of credentialing examinations is presented, and some new results are outlined that highlight the interactions among scoring, standard setting, and the reliability and validity of pass-fail decisions. Decision consistency is an important concept in evaluating credentialing…

This study examined the contribution of five categories of variables to the standard-setting recommendations of a representative sample of 241 judges recommending standards for the General Knowledge and Communication Skills Tests of the National Teacher Examinations (NTE) for the state of North Carolina. It was based on the assumption that…

The increasing interest in oral proficiency during the past decade prompted Educational Testing Service (ETS) to undertake the development of the Test of Spoken English (TSE), a standardized test of speaking proficiency of non-native speakers of English. The test has been validated for the selection of non-native teaching assistants applying to…

Technical advances of the last 15 years in measurement theory and practice are described, notably criterion-referenced testing, item response theory, and computers and testing. Several remaining problems concerning the development and validation of credentialing examinations are also considered. (Author/LMO)

Because of concerns about the validity of tests used in education and employment, a consultation on June 16, 1989, focused on tests of ability, achievement, and other skills. Invited experts were asked to address a set of issues common to both education and employment testing, primarily related to test construction procedures and how to establish…

This is the sixth and final report of the National Research Council's Committee on the Performance of Military Personnel on the Joint-Service Job Performance Measurement/Enlistment Standards (JPM) Project, a project designed to develop measures of performance for entry-level military jobs so that enlistment standards could be linked to performance…

Comparative data and results of validation and standard-setting studies are presented for five subject-matter tests for teacher certification in two states. The five tests evaluated were available from the Educational Testing Service (ETS): one National Teacher Examination specialty area test (special education); and four tests from a consortium…