A statewide evaluation in Idaho addressed whether teachers changed in teaching philosophy and technology skills following summer in-service workshops and whether these changes sustained themselves in the face of school year and classroom realities. The timing of data collection is an important factor to consider when interpreting program outcomes. Teachers' beliefs about teaching and their sense of competency with technologies varied at three different times: prior to summer workshops, after the summer workshops, and after 10 months of subsequent teaching. It is critical to distinguish changes that are temporary and due to enthusiasm immediately after a workshop from changes that sustain themselves or emerge over time as the initial impact of the workshop fades and the reality of the classroom comes to the fore. Guided by Katzer's (1981) "error model" of research, we address time of data collection and other sources of error to improve our knowledge of program impacts.