This paper reports on the development and piloting of Classmoto, an online application designed to measure learner engagement. The application enables teachers to collect real-time analytics of student social, affective, and cognitive engagement. The results are immediately visible to the teacher. We investigated and reported on the engagement levels of 124 learners over a period of 15 weeks. Our analysis explores learners' perceptions of the usefulness of evaluating their engagement and the experiences of four teachers using the application in their courses. This paper demonstrates how engagement measures such as Classmoto can be used to innovate classroom teaching and learning. It also highlights limitations in the application, with recommendations for future development for the measurement of engagement.