Impact evaluation and measurement are highly complex and can pose challenges for both social impact providers and funders. Measuring the impact of social interventions requires the continuous exploration and improvement of evaluation approaches and tools. This article explores the available evidence on meta-evaluation--the "evaluation of…

Purpose: Making policy makers, researcher, education leaders, and assessment developers aware that what matters in education assessment is a wicked problem that cannot be easily solved following traditional approaches. Design/Approach/Methods: Starting from the questions that what matters in education assessment, this article presented such…

Psychometrics has recently undergone extensive criticism within the medical education literature. The use of quantitative measurement using psychometric instruments such as response scales is thought to emphasize a narrow range of relevant learner skills and competencies. Recent reviews and commentaries suggest that a paradigm shift might be…

Value-added "Progress" measures are to be introduced for all English schools in 2016 as "headline" measures of school performance. This move comes despite research highlighting high levels of instability in value-added measures and concerns about the omission of contextual variables in the planned measure. This article studies…

The use of multi-category scales is increasing for the monitoring of IEP goals, classroom and school rules, and Behavior Improvement Plans (BIPs). Although they require greater inference than traditional data counting, little is known about the inter-rater reliability of these scales. This simulation study examined the performance of nine…

Calls for employability skill development in undergraduates now extend across many culturally similar developed economies. Government initiatives, industry professional accreditation criteria, and the development of academic teaching and learning standards increasingly drive the employability agenda, further cementing the need for skill…

This paper examines the construct measured by automated essay scoring (AES) systems. AES systems measure features of the text structure, linguistic structure, and conventional print form of essays; as such, the systems primarily measure text production skills. In the current state-of-the-art, AES provide little direct evidence about such matters…

This article explores a number of key issues with regard to the measurement of creativity in the course of conducting psychological research or when applying various evaluation measures. It is argued that, although creativity is a fuzzy concept, it is no more difficult to investigate than other fuzzy concepts people tend to take for granted. At…

This paper presents myths as well as facts about value-added analysis. These myths include: (1) "Value-added isn't fair to teachers who work in high-need schools, where students tend to lag far behind academically"; (2) "Value-added scores are too volatile from year-to-year to be trusted"; (3) "There's no research behind value-added"; (4) "Using…

This chapter addresses the psychometric challenges in assessing English language learners (ELLs) and students with disabilities (SWDs). The first section addresses some general considerations in the assessment of ELLs and SWDs, including the prevalence of ELLs and SWDs in the student population, federal and state legislation that requires the…

The literature on bibliometric indices for assessing scholarly impact, in particular the "h" index (Hirsch, 2005) and its many variants, is extensive, but nevertheless Ruscio and colleagues (this issue) succeed in making a valuable contribution. They have made the effort of collecting publication and citation data for no less than 1,750…

This study critically investigates the suitability of United Nations' composite indices and other related measures--among which the Gender Inequality Index just released in 2010--to capture gender inequalities in the context of "highly developed" countries, focusing on the case of Europe. Our results indicate that many of the gender gaps…

We examined articles in the "Journal of Applied Behavior Analysis" in which results of functional analyses indicated that problem behavior was maintained by multiple sources of reinforcement. Data for 88 (16.9%) of 521 subjects reported in 168 studies met the criteria for multiple control. Data for 11 subjects (2.1%) involved a single response…

This commentary examines the role of the unit from the perspective of the definition of measurement in physics as the ratio of two magnitudes, one of which is defined as the unit; it is an important and timely contribution to measurement in the social sciences. There are many different points that could be commented upon, but the author will…

Ruscio and colleagues (Ruscio, Seaman, D'Oriano, Stremlo, & Mahalchik, this issue) have done a great service by systematically comparing indices of scholarly impact. Three aspects of their work are particularly valuable: (1) Their assessment of the proliferating collection of metrics, whose development has become something of a cottage industry,…