This study explores six item-reduction methodologies used to shorten an existing complex problem-solving non-objective test by evaluating how each shortened form performs across three sources of validity evidence (i.e., test content, internal structure, and relationships with other variables). Two concerns prompted the development of the present study. First, to address practical concerns in education while administering psychometrically sound assessments, shorter, yet rigorous assessments, measuring complex student processes including problem solving are desirable. Second, methods to shorten assessments within psychology and medicine have been explored to some extent, best practices for item-reduction methods within educational testing has largely been absent. The "Problem Solving Measure 6" (PSM6) was designed to measure sixth-grade students' mathematical problem-solving ability aligned with the Common Core State Standards for Mathematics (CCSSM) (Bostic & Sondergeld, 2015). The original 15-item assessment was reduced using three qualitative models (content only, content plus difficulty, and content, difficulty, and point-biserial) and three quantitative item-reduction methods (Rasch measurement, factor analysis, and point-biserial). Each item-reduction method was used in an attempt to identify five-item and ten-item shortened versions of the "PSM6." The shortened tests were then evaluated based on test content, internal structure, and relationships to other variables. Test content was evaluated through an expert panel rating each item on its essentiality to measure its associated mathematical domain. Internal structure was evaluated through a Rasch (1960, 1980) measurement analysis. Relationships to other variable was evaluated through gender and ability differences, measure score correlations between the original and each short form, and the consistency of student quartile placement across the original and short forms. Some limitations were encountered during the reductions. For example, due to strict data requirements for factor analyses, only a five-item shortened test was created using factor analysis. Similarly, because of decision "ties" amongst panel experts, a six-item and two five-item tests were constructed. While analyses of test content found that the quantitatively-derived tests did not preserve the five mathematical subdomains, most other indicators of validity were stronger for quantitatively defined short forms. Internal structure was strongest for the ten-item Rasch and point-biserial tests, while the qualitative and factor analysis tests were weak. Relationships to other variables was reasonable for the five-item and ten-item Rasch and point-biserial shortened tests, but was largely unacceptable for the qualitative and factor analysis shortened tests. Holistically, the validity evidence suggests that Rasch item-reduction methods may constitute best practice for shortening existing instruments by maintaining acceptable psychometric performance and excellent decision consistency. Additionally, concerns regarding test content within the Rasch tests are overshadowed by more clearly focusing on the holistic latent trait (mathematical problem solving) rather than on the micro-sub-domains making up the construct within CCSSM. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com.bibliotheek.ehb.be/en-US/products/dissertations/individuals.shtml.]