The present study used generalizability theory to investigate the dependability of systematic observations of students' problem behavior and on-task behavior in classrooms. The Direct Observation Form (McConaughy & Achenbach, 2009) was used with a sample of 24, 6- to-11-year-old children attending 18 different elementary schools. The participants were referred because of concerns about learning and behavioral problems. All students were observed on eight occasions in the morning and afternoon across four different days. Generalizability analyses revealed important differences in observed variance and the number of observations required to obtain acceptable dependability across different behavioral dimensions. Results are discussed in terms of their implications for future research and practice. (Contains 5 tables.)