People with intellectual disabilities (IDs) are considered to be one of the most marginalized, isolated and disenfranchised groups in society. However, recent social prescription intervention programmes are being introduced to enhance the physical and mental well-being of these individuals through participation in leisure activities, thus increasing academic interest in research within the area. This article introduces the theoretical and methodological tensions in applying mainstream scholarly thinking of leisure and well-being to people with IDs and argues that by failing to acknowledge and address such tensions scholars are at risk of demonstrating intellectual ableism. Layering Kleiber's components of leisure over Seligman's Positive Emotion, Engagement, Relationships, Meaning, and Accomplishment concept of well-being, this article poses critical questions of how each contributing element of well-being could be reconceptualized for people with neurological diversities. The intersection of leisure, well-being and ID is ripe for research development; however, many studies in the domain are falling short of suitable theoretical discussion and methodological rigour. This article concludes with suggestions on how scholars can reduce intellectual ableism through inclusive design, methodological reporting, acknowledging bias and grappling theoretical dissonance.