Surveys with rating scales are often used in higher education research to measure student learning and development, yet testing and reporting on the longitudinal psychometric properties of these instruments is rare. Rasch techniques allow scholars to map item difficulty and individual aptitude on the same linear, continuous scale to compare respondents' development against scale challenge. This paper uses an example from the Interfaith Diversity Experiences and Attitudes Longitudinal Survey (IDEALS) to demonstrate how Rasch modeling can help researchers improve their instruments over time. Results indicate that adding more difficult items, particularly related to behavior, improved scale performance from one time point to another.