Impairments in cognitive flexibility have been used to characterize the neuropsychological presentation of persons with autism spectrum disorders (ASDs). Previous studies have yielded mixed results. Our objective was to systematically review the sensitivity of cognitive flexibility measures in ASD using quantitative methods employed by meta-analytic statistical techniques. Seventy-two studies met inclusion criteria for analysis and included a total of 2,137 individuals with ASD and 2,185 healthy controls. Our findings demonstrate that while the shift sub-scale of the self-report version of the Behavioral Rating Inventory of Executive Function (BRIEF) showed approximate absolute discriminability, of all the performance measures that were systematically reviewed and evaluated, none could reliably differentiate between individuals with ASD and controls; this is not surprising given that cognitive flexibility is not a core deficit of ASD. Our findings suggest that while the shift sub-scale of the self-report version of the BRIEF is a promising clinical marker, clinical performance measures of cognitive flexibility may lack ecological validity and lastly, reinforces that impairments in cognitive flexibility do not uniformly characterize all persons with ASD.