Educational leaders' effectiveness in solving problems is vital to school and system-level efforts to address macrosystem problems of educational inequity and social injustice. Leaders' problem-solving conversation attempts are typically influenced by three types of beliefs--beliefs about the nature of the problem, about what causes it, and about how to solve it. Effective problem solving demands testing the validity of these beliefs--the focus of our investigation. We analyzed 43 conversations between leaders and staff about equity related problems including teaching effectiveness. We first determined the types of beliefs held and the validity testing behaviors employed drawing on fine-grained coding frameworks. The quantification of these allowed us to use cross tabs and chi-square tests of independence to explore the relationship between leaders' use of validity testing behaviors (those identified as more routine or more robust, and those relating to both advocacy and inquiry) and belief type. Leaders tended to avoid discussion of problem causes, advocate more than inquire, bypass disagreements, and rarely explore logic between solutions and problem causes. There was a significant relationship between belief type and the likelihood that leaders will test the validity of those beliefs--beliefs about problem causes were the least likely to be tested. The patterns found here are likely to impact whether micro and mesosystem problems, and ultimately exo and macrosystem problems, are solved. Capability building in belief validity testing is vital for leadership professional learning to ensure curriculum, social justice and equity policy aspirations are realized in practice.