Many teacher education institutions share a concern about assessing their preservice teachers' knowledge, skills, and dispositions related to teaching diverse P-12 students. This case study describes the iterative and sometimes difficult process involved in the development of a diversity awareness survey at one institution. The survey discussed here, administered each semester to approximately 400 preservice teachers in a college of education, produces a large amount of rich data that can be examined in relation to variables such as age, gender, intended certification level and discipline, and life experiences. Working with data from the survey the authors have articulated a hierarchy of six developmental levels that characterize participants' responses in relation to awareness/knowledge, skills, and dispositions. Initial findings indicate that most preservice teachers in the program are at a developmental level of three or four with regard to the open-ended questions about teaching diverse students. The focus of this article is on the challenge in designing a high-quality survey for this purpose. This survey and other similar institutional research studies are valuable tools for assessing program effectiveness as related to stated university goals, and in making programmatic and curricular decisions as a result. (Contains 4 tables.)