Over the previous two decades, the era of accountability has amplified efforts to measure educational effectiveness more than Edward Thorndike, the father of educational measurement, likely would have imagined. Expressly, the measurement structure for evaluating educational effectiveness continues to rely increasingly on one sole indicator--performance on state-endorsed, large-scaled tests--and the use of these test scores in isolation of other indicators that also capture what it means to be effective. This manuscript describes unresolved questions in educational measurement and provides recommendations to increase measurement validity along both quantitative and qualitative dimensions to move towards a more holistic and appropriate system of measuring educational effectiveness. As Thorndike would have put it, we shall be aided, not hindered, by these tests.