Rubrics are commonly used to assess individuals in the field of education. Although rubrics are frequently used as data sources for research studies, the creation and validation of rubrics have not been common experiences for most researchers. There are many practical issues, such as requiring ongoing training by the evaluation team, in order to ensure accurate data collection. In this case study, we share our personal experiences and impressions of literally "starting from scratch" on a 5-year grant funded by the U.S. government. "Starting from scratch" included identifying and/or creating assessment rubrics, training individuals to collect data, and rubric validation. We hope our perspectives identifying criteria, determining word choices, constructing valid and reliable rubrics, and scoring rubrics harmoniously will shed light on issues that may arise when working with rubrics in data collection for research projects. The data for this study were acquired amidst grant work funded by the U.S. Department of Education Investing in Innovation (i3) under grant number u411c110102. Any opinions, findings, and conclusions or recommendations expressed in this document are those of the authors and do not necessarily reflect the views of the U.S. Department of Education. [This content is provided in the format of an e-book.]