Self-report scales have become the most widely used instruments to capture people's self-perception of creativity. Previous studies, however, provided only a limited insight into the psychometric properties of such measures. This paper reports an extensive item response theory (IRT) analysis of the Short Scale of Creative Self (SSCS): one of the most frequently used scales of creative self-concept. Based on samples from 14 studies (overall N > 26,000), we report IRT parameters of creative self-efficacy and creative personal identity scales' items. We examined whether the scores obtained in the SSCS depend on the length of the response scale (5-versus-7-point Likert scales) and whether latent scores are comparable across different data collection methods (online, paper-and-pencil, phone), age, and gender. The results confirmed the two-factor structure of the SSCS, good psychometric properties of its items, as well as invariance regarding response scales, age, gender, and method of data collection. At the same time, the items (and consequently--scales) were easy in the psychometrical sense, thus providing much more reliable scores among individuals who scored low or medium in creative self-concept. Longer (7-point) and shorter (5-point) Likert scales performed similarly, with some psychometric arguments favoring fewer points on the scale. Gender differences were negligible (Cohen's d between 0.00 and 0.01). We discuss potential ways of further improvement and development of the SSCS.