At present, there are no brief, freely-available, informant-report measures that evaluate key challenging behaviors relevant to youth with autism spectrum disorder (ASD) or other developmental disabilities (DD). This paper describes the development, refinement, and initial psychometric evaluation of a new 18-item measure, the Open-Source Challenging Behavior Scale (OS-CBS). In a large sample (n = 2004, 169 with ASD, ages 2-17), results of psychometric analyses indicated a clear factor structure (property destruction, aggression, elopement, conduct problems, and self-injury and a general factor with high loadings from all items) based on exploratory structural equation modeling, good scale reliability ([alpha] = 0.66-0.83 for subscales, [alpha] = 0.91 total scale), measurement invariance across demographics, and good construct validity. The OS-CBS is a psychometrically-sound instrument for screening and monitoring intervention progress.