I consider recent attempts to establish standards, principles, and goals for artificial intelligence (AI) through the lens of educational measurement. Distinctions are made between generative AI and AI-adjacent methods and applications of AI in formative versus summative assessment contexts. While expressing optimism about its possibilities, I…

This paper analyses the psychometric paradigm for understanding educational measurement and standard setting by considering the extent to which that understanding is based on metaphors. I argue that through the metaphors 'more is up' and 'good is up' we spatialise concepts that are not intrinsically spatial. Human abilities form a distinct…

This report provides a resource for state and district assessment staff as they develop requests for proposals (RFPs) for interim assessments for all of their students, including those who require accessibility features, to demonstrate their knowledge and skills as well as those who may need an alternate assessment because of their significant…

Seldom have comparative studies of educational assessment systems been undertaken, especially regarding their standard setting procedures. This study examines the effects of governance structures on the power relations in standard setting in the dominant school-leaving or university-entrance examinations. We present acritical analysis of the…

Changes to the American Board of Internal Medicine's Maintenance of Certification Program: Preserving Core Assessment Values As Maintenance of Certification (MoC) for practicing physicians has evolved into more continuous programs, controversy has arisen over how physicians should self-regulate their profession and serve the public trust in doing…

While 21st century skills are getting increasing traction in both academia and the professional world, a big challenge today is that there has been no consensus on the unified conceptual framework to define and measure 21st century skills. This consensus is indispensable for understanding, teaching and assessing these skills. Additionally, few…

This case study provides an account of my experiences and lessons learned in designing and implementing a survey research. The research was part of my doctoral thesis in educational measurement and quantitative methods. The purpose of the study was to examine the effectiveness of a Completely Structured Training compared with Partially Structured…

This article explores the challenge of setting performance standards in a non-Western context. The study is centered on standard-setting practice in the national learning assessments of Trinidad and Tobago. Quantitative and qualitative data from annual evaluations between 2005 and 2009 were compiled, analyzed, and deconstructed. In the mixed…

Standard-setting procedures are a key component within many large-scale educational assessment systems. They are consensual approaches in which committees of experts set cut-scores on continuous proficiency scales, which facilitate communication of proficiency distributions of students to a wide variety of stakeholders. This communicative function…

The purpose of this study is to describe a new approach for evaluating the judgments of standard-setting panelists within the context of the bookmark procedure. The bookmark procedure is widely used for setting performance standards on high-stakes assessments. A many-faceted Rasch (MFR) model is proposed for evaluating the bookmark judgments of…

Much ado has been made about setting high standards over the past year. Current conversations about creating a common national standard largely focus on the substantive curriculum to be taught at various grade levels. Even more important is each state's expectations for student performance with respect to the curriculum, as expressed through its…

Accountability efforts at California's community colleges gained prominence in the 1990s under an agreement known as Partnership for Excellence (PFE) that involved the chancellor's office, the legislature, and the governor's office. This agreement was the first wide-ranging accountability effort undertaken by the state's community college system.…

The previous issue of "Blueprint" reported significant variability among state standards for education. Such variability means that depending where a student attends school, he might not cover the same material in a course called Algebra I, or Algebra II, or Grade 4, or Grade 6. Many citizens and leaders understand that having a single set of…

In this study, we investigated the changes in physical education programs from 2001 to 2006 based on an analysis of data reported in The Shape of the Nation Report: Status of Physical Education in the USA (National Association for Sport and Physical Education, 2006). Means and standard deviations for numeric variables in the reports were computed,…

This paper examines the Programme for International Student Assessment (PISA), which has become a major and influential component of the Organisation for Economic Cooperation and Development's (OECD) educational work. This measure of comparative performance of educational systems of member and other nations is based on tests commissioned by the…