How various methodological decisions may influence studies of the effect of time-on-task on achievement are examined. Subjects were students in grades 2-5 in 18 classes taught by 12 teachers in a rural Maryland school district. All students were pre-tested in February 1978 in reading, language arts, math and social studies using the Comprehensive Test of Basic Skills. A post-test was given in May, 1978. It was found that altering definitions of time-on-task to include momentary off-task behaviors affected the conclusions for the importance of time-on-task. Clear evidence was presented that sampling segments of instruction would tend to obscure the positive results for time-on-task. It was also shown that reducing the number of days of observation weakened the effects of time-on-task. However, the timing of the observation was not very important for the noted effects. The effect of sampling fewer than six students was explored and, due to the effect on reliability, it was suggested that this approach would not be advisable. Results suggested that although there is an understandable urge to lessen the observation time in order to bolster the number of settings observed, such steps should only be taken cautiously. (Author/RL)