Feasible and accurate assessment tools developed for non-specialists are needed to scale community-based models of autism evaluation. The purpose of this study was to evaluate use of the Screening Tool for Autism in Toddlers and Young Children (STAT) when used by primary care practitioners (n = 10) across a statewide system of early diagnosis set within seven United States primary care practices. Specifically, for 130, 14- to 48-month-old children, we examined (1) agreement between STAT classification and Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) outcome and expert diagnosis of autism, (2) characteristics of children with inaccurate STAT classifications, (3) consistency between STAT classification and primary care practitioner diagnosis, and (4) the relationship between STAT/primary care practitioner classification and expert diagnosis. The STAT demonstrated good concurrent validity with the ADOS-2 (77% agreement). Concordance between specialty trained primary care practitioners with expert diagnosis was 80%-87% across three age-based subgroups of children. Children misclassified by the STAT were older, had higher developmental and adaptive skills, and lower autism symptoms. Primary care practitioner diagnosis aligned with STAT categorical risk classification in 86% of cases, and 73% of consistent classifications between STAT and primary care practitioner diagnosis matched expert diagnosis. Overall, the STAT demonstrates good accuracy when used by non-specialists as part of a diagnostic evaluation.