Describing the relative severity and change in autism symptoms is crucial for the appropriate characterization of clinical and research populations. The calibrated severity score (CSS) of the Autism Diagnostic Observation Schedule-2 (ADOS-2; Lord et al., 2012) was created to better describe autism symptom severity consistently across different ages and language levels. The CSS has been widely used to quantify and compare symptom severity on a 10-point scale across Modules; however, its test re-test reliability has not been studied. With 608 ADOS observations, we showed strong test re-test reliability of the CSS across all ADOS Modules. The results support the use of the ADOS CSS as a reliable tool to quantify autism symptom severity across development.