Because the educational technology research literature is so broad, it is difficult for researchers to acquire an accurate sense of the issues and trends across the entire field. There has not been any recent effort to synthesize or critically analyse the systematic reviews in the area of educational technology. This study analysed 73 systematic literature reviews, including meta-analyses, focused on the evaluation of technology in educational contexts, in order to holistically understand the impact of learning technology use across different aspects of evaluation. Among these reviews, the most common theme examined was learning outcomes (89%), followed by affective elements (45%), behaviours (25%), technological elements (21%) and teaching/pedagogical aspects (19%). Most of the reviews found that the use of technology improved learning outcomes and affective perceptions. Approaches involving interaction, gamification, constructivism, student-centred learning and feedback were most effective. The analysis highlighted the need for more reviews focusing on multiple aspects of learning technology evaluation, on school level education and on the use of technology in naturalistic (non-interventional) settings. Critical reflections are also cast on the methods used to conduct systematic reviews in the educational technology field.