Researchers design diagnostic assessments to measure students' knowledge structures and processing skills to provide information about their cognitive attribute. The purpose of this study is to determine the instrument's validity and score reliability, as well as to investigate the use of classical test theory to identify item characteristics. The…

Validity of a measurement refers to appropriate test score meanings, uses, and interpretations (Messick, 1989; Kane, 1992). There are different approaches to validity: an evidentiary aspect of validity is one requiring gathering statistical evidence to evaluate test score meaning. A common approach to validation is comparisons of test score equity…

This study aims to design mathematical literacy instruments that have evidence of content and construct validity and are reliable for use as an assessment for learning. The research involved eight experts as instrument validators and 273 eighth-grade students of junior high school in Yogyakarta Province. The results showed that the ten…

In this article, we highlight the need for a precisely defined construct in score-based validation and discuss the contribution of cognitive theories to accurately and comprehensively defining the construct. We propose a framework for integrating cognitively based theoretical and empirical evidence to specify and evaluate the construct. We apply…

The present study examined the technical adequacy of curriculum-based measures (CBMs) of early numeracy. Six 1-min early mathematics tasks were administered to 137 kindergarten and first-grade students, along with an omnibus test of early mathematics. The CBM measures included Count Out Loud, Quantity Discrimination, Number Identification, Missing…

In the following technical report, we present evidence of the technical adequacy of the easyCBM[R] math measures in grades K-2. In addition to reliability information, we present criterion-related validity evidence, both concurrent and predictive, and construct validity evidence. The results represent data gathered throughout the 2009/2010 school…

Many teachers and curriculum specialists claim that the reading demand of many mathematics items is so great that students do not perform well on mathematics tests, even though they have a good understanding of mathematics. The purpose of this research was to test this claim empirically. This analysis was accomplished by considering examinees that…

In this technical report, data are presented on the practical utility, reliability, and validity of the easyCBM[R] mathematics (2009-2010 version) measures for students in grades 3-8 within four districts in two states. Analyses include: minimum acceptable within-year growth; minimum acceptable year-end benchmark performance; internal and…

This study examined the construct validity of the KeyMath-Revised by testing the factorial model proposed by the test author. Results failed to confirm the proposed factorial model and suggested that the KeyMath-Revised assesses two domains that are difficult to interpret, rather than the three proposed by the test author. (Author/JDD)

The validation of key contextual features of algebra word problems was studied in two phases. In the first phase, five experts were asked to assess the appropriateness of the concepts in the problems and the adequacy of the assignment of the contextual features to the problems. In the second phase, construct validity was established by having 6…

Basic skills tests in which measurement of growth is defined in the context of national norms are discussed. An integral part of the construction of norms is the production of a trait or ability score which is normed. Test publishers define a multidimensional trait by including items which measure it. The trait as a construct is a distinct…

The present study, using a sample of 1,491 grade 10 students from a mid-western province in Canada, examined the validity evidence on the Canadian Tests of Basic Skills. Factor analysis of the 34 micro-skills (content by skill/process classification of items) of the four tests indicated that the Reading Comprehension, Written Expression, and Using…

Presented at the Annual National Council on Measurement in Education (NCME) in New York in March 2008. This presentation explores how different item weighting can affect the effective weights, validity coefficents and test reliability of composite scores among test takers.