In November 2022, ChatGPT, an Artificial Intelligence (AI) large language model (LLM) capable of generating human-like responses, was launched. ChatGPT has a variety of promising applications in education, such as using it as thought-partner in generating curricular resources. However, scholars also recognize that the use of ChatGPT raises concerns, such as outputs that are inaccurate, nonsensical, or vague. We, two mathematics teacher educators, engaged in a collaborative self-study using qualitative descriptive approaches to investigate the procedures, concepts, and cognitive demand of ChatGPT-generated mathematical tasks focused on fraction multiplication using the area model approach. We found that the ChatGPT-generated tasks were mostly procedural and not cognitively demanding. Moreover, despite ten variations of input prompts, ChatGPT did not produce any tasks that used the area model approach for fraction multiplication. Rather, it generated tasks focused on procedural approaches. Alarmingly, some tasks were conceptually and/or procedurally inaccurate and vague. We suggest that educators cannot fully rely on ChatGPT to generate cognitively demanding fraction multiplication tasks using the area model. We offer recommendations for educators' strategic use of ChatGPT to generate cognitively demanding mathematical tasks.