Visual attention develops rapidly across the first postnatal year, from reflexive eye movements driven by low-level stimulus properties to increasingly voluntary eye movements influenced by higher-order factors. To test the hypothesis that development reflects guidance by increasingly abstract features, we used representational similarity analysis to evaluate the representational link between gaze patterns (N = 47 5-7-month-old infants, N = 46 10-12-month old infants, N = 45 adults) and measurements of fMRI cortical activity patterns from adult participants as they viewed scenes from the Natural Scenes Dataset. We found that similarities across scenes for fixation patterns and neural activity patterns were significantly related only for low-level visual regions in younger infants but were related to mid-level regions in older infants and adults. These results support the hypothesis that, over development, visual attention shifts from being driven by the kinds of simple features represented in early visual cortex to being driven by the kinds of more abstract features found in mid-level areas of adult visual cortex. A video abstract of this article can be viewed at https://www.youtube.com/watch?v=WkIV9QB0Uq8.