Background: This study compared levels of physical activity completed by adults with and without Down syndrome. Method: Fifteen adults with and 15 adults without Down syndrome matched for age and gender, took part. The intensity and duration of physical activity were measured using RT3 accelerometers worn for seven days. Results: Only, 12 participants with Down syndrome had complete physical activity data, and these participants and their matched controls (total: six females, 18 males; aged 25.8 ± 9.7) were included in the analyses. There were significantly lower levels of moderate and vigorous physical activity per day for people with Down syndrome (median = 27 min) compared to those without (median = 101 min) (p < 0.001). Participants without disability were twice more likely to achieve recommended levels of physical activity than people with Down syndrome. Conclusions: Adults with Down syndrome appear to participate in lower levels of physical activity than adults without Down syndrome. Further research should validate these estimates.