In the fall of 2009, the Bill and Melinda Gates Foundation launched the two-year Measures of Effective Teaching (MET) project to rigorously develop and test multiple measures of teacher effectiveness. As part of the project, partners from more than a dozen reputable academic, non-profit and for-profit organizations collected and analyzed data from the 2009-10 and 2010-11 school years from over 3,000 teacher volunteers and their classrooms across Charlotte-Mecklenburg Schools, Dallas Independent School District, Denver Public Schools, Hillsborough County Public Schools, Memphis City Schools, and the New York City Department of Education. Teachers and classrooms in Pittsburgh Public Schools also participated in the project by helping researchers with early-stage development and testing of the effectiveness measures before they were tested in the other MET project districts. This paper seeks to define and explain how classroom observations and teacher reflections were used to develop the "validation engine" for reviewing other observational protocols using four components: a secure video viewer; a score sheet template; an analytic engine; and an automated report generator. Results will provide information on individual indicators that can be used to revise the expectation for evidence, or possibly eliminate the indicator if it is not correlated with improvement in student achievement. Multiple observation protocols can be tested and retested regularly to demonstrate validity.