It is meaningful to create a comprehensive score to extract information from mass continuous data when they measure the same latent concept. Therefore, this study adopts the logic of psychometrics to conduct scales on continuous data under the Rasch models. This study also explores the effect of different data discretization methods on scale…

It is common to find mixed-format data results from the use of both multiple-choice (MC) and constructed-response (CR) questions on assessments. Dealing with these mixed response types involves understanding what the assessment is measuring, and the use of suitable measurement models to estimate latent abilities. Past research in educational…

Human ratings are ubiquitous in creativity research. Yet, the process of rating responses to creativity tasks -- typically several hundred or thousands of responses, per rater -- is often time-consuming and expensive. Planned missing data designs, where raters only rate a subset of the total number of responses, have been recently proposed as one…

Item Response Theory (IRT) models are a popular analytic method for self report data. We show how traditional IRT models can be vulnerable to specific kinds of asymmetric measurement error (AME) in self-report data, because the models spread the error to all estimates -- even those of items that do not contribute error. We quantify the impact of…

For decades, schools nationwide have collected household income data to identify those eligible for free and reduced-price lunch programs. State leaders have long used this information as a proxy for student economic disadvantage--yet this data is insufficient to understand and address students' needs. Measures based on FRL data lack critical…

Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

The Next Generation Science Standards (NGSS) delineate three interrelated dimensions that describe what students should know and how they should engage in science learning. These present significant challenges for assessment because traditional assessments may not be able to capture the ways in which students engage with content. Science…

Recent work on measuring growth with categorical outcome variables has combined the item response theory (IRT) measurement model with the latent growth curve model and extended the assessment of growth to multidimensional IRT models and higher order IRT models. However, there is a lack of synthetic studies that clearly evaluate the strength and…

An important aspect of qualitative research is reaching saturation--loosely, a point at which observing more data will not lead to discovery of more information related to the research questions. However, there has been no validated means of objectively establishing saturation. This article proposes a novel quantitative approach to measuring…

Recent work on measuring growth with categorical outcome variables has combined the item response theory (IRT) measurement model with the latent growth curve (LGC) model (e.g., McArdle, 1988) and extended the assessment of growth to multidimensional IRT models (e.g., Hsieh, von Eye, & Maier, 2010; Huang, 2013) and higher-order IRT models…

Researchers use many different metrics for evaluation of performance of student models. The aim of this paper is to provide an overview of commonly used metrics, to discuss properties, advantages, and disadvantages of different metrics, to summarize current practice in educational data mining, and to provide guidance for evaluation of student…

The role and impact of diversity in higher education has permeated conversations about student access and achievement for many years. Language articulated by various courts suggests that higher education policies should reflect a broad conceptualisation of diversity beyond that of the magnitude and proportion of race and ethnicity, yet…

Being able to characterize the size of raindrops is useful in a number of fields including meteorology, hydrology, agriculture and telecommunications. Associated with this article are data sets containing surface (i.e. ground-level) measurements of raindrop size from two different instruments and two different geographical locations. Students may…

At their core, Intelligent Tutoring Systems consist of a student model and a policy. The student model captures the state of the student and the policy uses the student model to individualize instruction. Policies require different properties from the student model. For example, a mastery threshold policy requires the student model to have a way…

Central to the mission of most educational institutions is the task of preparing the next generation of citizens to contribute to society. Schools, colleges, and universities value a range of outcomes--e.g., problem solving, creativity, collaboration, citizenship, service to community--as well as academic outcomes in traditional subjects. Often…