This paper explores methodological insights from a mixed methods study that aims to understand how school leaders promote literacy development in their schools. The study findings consider both the complementarities and the challenges of the qualitative and quantitative approaches to measuring leadership practices and their linkages with learning across schools. We begin by identifying a conundrum in school leadership and management (SLM) research -- strong effects found in qualitative studies and weaker effects in quantitative studies. From the literature we identify some of the central challenges that account for these differences. We then show how these challenges were and were not addressed in the mixed method research we conducted in an SLM study of South African primary schools in challenging contexts. We consider why the central aim of the study -- to develop a scalable instrument for measuring SLM -- remains elusive.