Sport settings have long been documented as exclusionary environments for sexuality and gender diverse (SAGD) people and a key location of discrimination. Sport is associated with well-being, and increasing physical activity for disadvantaged groups is a key aim for many governments. This study reports on 13 semi-structured interviews with young SAGD people aged 18-24 in Australia that explore their attitudes to and experiences of sport and physical activity. We found that SAGD young people are 'game to play': they hold strong desires to participate in sport and physical activity, and articulated how sport could be more inclusive. Using Bourdieu's concept of 'capital', we outline how young SAGD people identify sport as a 'field' that requires and fosters various types of capital. We explore barriers they identified that prevent the accumulation of physical and social capital associated with sport participation and physical activity, and their insights for how this may be fostered. The 'field' of sport presents as an exclusionary environment, which allows for both explicit bullying and more subtle discriminatory practices. Passive assimilation approaches are not enough to provoke substantial change and active intervention is required to decentre the heteronormativity underpinning these spaces, which has implications for policy and practice to advance the inclusion of SAGD young people in sporting environments.