Classical item analysis (CIA) entails summarizing items based on two key attributes: item difficulty and item discrimination, defined as the proportion of examinees answering correctly and the difference in correctness between high and low scorers. Recent insights reveal a direct link between these measures and aspects of signal detection theory…

It is imperative to collect validity evidence prior to interpreting and using test scores. During the process of collecting validity evidence, test developers should consider whether test scores are contaminated by sources of extraneous information. This is referred to as construct irrelevant variance, or the "degree to which test scores are…

This convergent, parallel, mixed-methods study with qualitative and quantitative content analysis methods was conducted to identify what type of thinking is required by the College and Career Readiness Assessment (CCRA+) by (a) determining the frequency and percentage of questions categorized as higher-level thinking within each cell of Hess'…

Multiple-select multiple-choice items, or multiple-choice items with more than one correct answer, are used to quickly assess content on standardized assessments. Because there are multiple keys to these item types, there are also multiple ways to score student responses to these items. The purpose of this study was to investigate how changing the…

Despite the increasing demand for diagnostic information, observed subscores have been often reported to lack adequate psychometric qualities such as reliability, distinctiveness, and validity. Therefore, several statistical techniques based on CTT and IRT frameworks have been proposed to improve the quality of subscores. More recently, DCM has…

The following is a five-chapter dissertation surrounding the use of text mining techniques for better understanding the language of mathematics items from standardized tests to improve linguistic equity of these items to support assessment of English Language Learners. Introduction: The dissertation begins with an overview of the problem that…

This dissertation explored whether unnecessary linguistic complexity (LC) in mathematics and biology assessment items changes the direction and significance of differential item functioning (DIF) between subgroups emergent bilinguals (EBs) and English proficient students (EPs). Due to inconsistencies in measuring LC in items, Study One adapted a…

Pseudowords (words that are not real but resemble real words in a language) have been used increasingly as a technique to reduce contamination due to construct-irrelevant variance in assessments of verbal fluid reasoning (Gf). However, despite pseudowords being researched heavily in other psychology sub-disciplines, they have received little…

Despite the growing recognition that second language (L2) listening is a skill incorporating the ability to process visual information along with the auditory stimulus, standardized L2 listening assessments have been predominantly operationalizing this language skill as visual-free (Buck, 2001; Kang, Gutierrez Arvizu, Chaipuapae, & Lesnov,…

Investigating the psychometric properties of a screening instrument for young children is necessary to ascertain its quality and accuracy. In light of the important role culture plays on human beliefs and parenting styles, a newly translated and adapted test needs to be studied. Evaluating outcomes on a translated version of a test may reveal…

The purpose of this study was to examine the overall level of student achievement on the 2012 Kansas History-Government Assessment in Grades 6, 8, and high school, with major emphasis on the subject area of economics. It explored four specific research questions in order to: (1) determine the level of student knowledge of assessed economic…

This study investigates the effect of fitting a unidimensional IRT model to multidimensional data in content-balanced computerized adaptive testing (CAT). Unconstrained CAT with the maximum information item selection method is chosen as the baseline, and the performances of three content balancing procedures, the constrained CAT (CCAT), the…

Multiple factors have influenced testing agencies to more carefully consider the manner and frequency in which pretest item data are collected and analyzed. One potentially promising development is judges' estimates of item difficulty. Accurate estimates of item difficulty may be used to reduce pretest samples sizes, supplement insufficient…

The Taiwanese early intervention/early childhood special education (EI/ECSE) system is modeled after the federal legislation and practices of the U.S., incorporating specific cultural beliefs in Taiwan. Nonetheless, in EI/ECSE assessments, several challenges are presented, including: (a) limited resources and funding, (b) lack of reliable and…