In autistic adults, measurement tools may not adequately differentiate between autistic characteristics and features of anxiety. This may be particularly evident in the case of social anxiety disorder; however, few measures of social anxiety disorder have been validated for autistic adults. Instead, assessments are often made using measures designed and validated for non-autistic adults. To address this, we examined the suitability of the self-report Severity Measure for Social Anxiety Disorder in 284 autistic adults. We evaluated its psychometric properties, explored a rescored version that aimed to account for autistic differences in reporting and conducted content analysis on qualitative feedback from autistic participants. Confirmatory Factor Analysis did not confirm the unidimensional structure of the Severity Measure for Social Anxiety Disorder for autistic adults and over half attributed at least one response to a factor other than social anxiety (e.g. fatigue). Exploratory Factor Analysis of a rescored version of the Severity Measure for Social Anxiety Disorder based on autistic adults' responses to reflect those just due to social anxiety, revealed that a two-factor solution was most appropriate, again suggesting social anxiety was not a solitary latent variable. Our findings recommend caution be applied when using the Severity Measure for Social Anxiety Disorder with autistic adults and our content analysis may inform more precise future measurement.