Despite much fanfare, new technologies have yet to fundamentally advance student outcomes in K-12 schools or other educational settings. We believe that the system that supports the development and dissemination of educational technology tools is falling short. The key missing ingredient is rigorous evaluation. No one knows what works and for whom. This policy memo articulates general principles that should guide the evaluation of educational technology; these evaluations have the promise to fill in critical information gaps and leverage the potential of new technologies to improve learning. We also present a case study of a new platform, EDUSTAR, conceived by the authors and implemented with a national nonprofit organization. The results from the platform pilot examples reveal several lessons for the future of educational technology.