The current study was designed to evaluate a rotating momentary time sampling (MTS) data collection system. A rotating MTS system has been used to measure activity preferences of preschoolers but not to collect data on responses that vary in duration and frequency (e.g., talking). We collected data on talking for 10 preschoolers using a 5-s MTS…

Out-of-school time (OST) youth programs are inherently difficult to assess. They are often very dynamic: Many youth interact with one another and with staff members in various physical environments. Despite the challenge, measuring quality is critical to help program directors and policy makers identify where to improve and how to support those…

The Kindergarten Early Learning Scale (KELS) was developed as a concise observational assessment for young children. It examines three domains including (1) Math/Science, (2) Social Emotional/Social Studies, and (3) Language and Literacy, with a total of 10 items across the domains. Scores reported for each of the 10 items are based upon…

The purpose of this study is to contribute to efforts to improve methods for gathering and analyzing data from classroom observations in early literacy. The methodological approach addresses current problems of reliability and validity of classroom observations by taking into account differences in teachers' uses of instructional actions (e.g.,…

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes…

This paper reviews the prominent concepts of the stuttering event and concerns about the reliability of stuttering event measurements, specifically interjudge agreement. Recent attempts to resolve the stuttering measurement problem are reviewed, and the implications of developing an improved measurement system are discussed. (Author/JDD)

This paper contends that behavior observation data relating to speech-language pathology are reliable if they are not affected by differences among observers or other variations in the recording context. The theoretical bases of methods used to estimate reliability for observational data are reviewed, and suggestions are provided for improving the…

This study, involving five college students observing behavior of individuals with mental retardation, found that interobserver agreement was better in the presence of certain stimuli (such as when the subject's peers were present versus when they were not). Of 10 additional variables thought to potentially affect agreement scores, 1 (can't see,…

Background: Coaches and physical educators must obtain valid data relating to the contribution of each of their players in order to assess their level of performance in team sport competition. This information must also be collected and used in real game situations to be more valid. Developed initially for a physical education class context, the…

The Structured Observation System (SOS) is a data collection method developed to document changes in the communication behaviours of children identified with speech and language delays. The system employs a rating scale which reflects the occurrence of communication behaviours as well as the amount of assistance needed for behaviours to occur.…

Differences in agreement between observers and analysts of naturalistic narrative data cause problems in observation research. This paper discusses the advantages and disadvantages of several possible solutions. (Author/GDC)

Forty-seven young children, 15 with disabilities, were observed 4 times for types and levels of engagement. Results indicated that engagement is difficult to measure through molecular data collection techniques because of error in dependability measures. The number of observed sessions could be increased to achieve generalizability, but increases…

Continuous observational recording over 57 hours evaluated behavior states of three adults with profound and multiple disabilities. Two independent observers also recorded for 22 hours. Although overall percentage agreement was satisfactory (above 80%), agreement on occurrence was unsatisfactory (mean of 65%). Agreement data were superimposed on…

The investigation examined relationships among scales for observing and rating teacher performance. Beginning teachers with varying levels of professional experience (2, 9, and 16 months) were rated by pairs of observers on two occasions. Intercorrelations across occasions fell between .5 and .8. Interrater agreement ranged between .5 and .9.…

The purpose of this research is to determine whether we can use classroom artifacts as the basis for making valid judgments about the presence of reform-oriented teaching practices in middle-school mathematics classes. Our approach compares ratings based on collections of artifacts assembled by teachers according to our directions (the "Scoop…