Any moves towards substantive equality in education must negotiate the contradictions between equality and efficiency. Equality of education comes about through both the widening of opportunity and the maintenance of educational quality, but in the context of limited resources, educational policy rarely serves both ends simultaneously. Regardless of imperatives involved in making particular policy choices, if the resulting outcomes are either too visible or the system is deemed to be too rigid, social inequality emerges as an intractable, highly salient issue. The critical questions for research thus become: How do various approaches to negotiating this central tension differ? How does the choice of strategies produce different results across different education systems? To explore these questions, this paper examines the function and outcomes of educational differentiation in Japan, both at the secondary and tertiary levels, in relation to social inequality. Given that Japan entered an era of "universal" access to higher education ahead of other high-income countries, it presents an ideal case to reflect on policy choices currently being considered or implemented elsewhere. Through cross-sectional analyses of three cohorts of Japanese graduates, it reveals that social equality in accessing elite secondary and higher education institutions deteriorates as privatisation of education advances. It concludes that hierarchical structure of secondary and higher education institutions, when coupled with policies advancing privatisation and universalisation, result in negative or inconsequential effects on social equality. (Contains 9 notes, 1 figure, and 6 tables.)