To address students' poor literacy outcomes, an intervention using a computer-based literacy tool, ABRACADABRA, was implemented in 6 Northern Australia primary schools. A pretest, posttest parallel group, single blind multisite randomized controlled trial was conducted with 308 students between the ages of 4 and 8 years old (M age = 5.8 years, SD = 0.8 years). Findings suggested that computer-based instruction under controlled conditions can improve student literacy, especially for Indigenous students at risk of reading difficulties. The authors examine the fidelity with which the computer-based literacy tool was implemented and the impact of implementation fidelity measures on student outcomes. Student exposure to and use of the literacy tool, and quality of instruction and lesson delivery, were analyzed for their influence on students' literacy outcomes. Implementation fidelity measures accounted for between 1.8% and 15% of the variance of intervention students' scores.