The Play Cycle Observation Method (PCOM) is an observational tool developed to focus on the process of play and has shown good reliability when watching videos of children playing. This study piloted use of the PCOM in 'real time' in a pre-school setting where 3-year-old children play. The results from two independent observers not familiar with…

The same L2 speaking performance may be analysed and evaluated in very different ways by different teachers or raters. We present a new, technology-assisted research design which opens up to investigation the trajectories of convergence and divergence between raters. We tracked and recorded what different raters noticed when, whilst grading a…

In this study, generalizability theory was used to examine the extent to which (a) time-sampling methodology, (b) number of simultaneous behavior targets, and (c) individual raters influenced variance in ratings of academic engagement for an elementary-aged student. Ten graduate-student raters, with an average of 7.20 hr of previous training in…

Previous research shows inconsistency in clinician-assigned diagnoses of Autism Spectrum Disorder (ASD). We conducted an exploratory study that examined the concordance of diagnoses between a multidisciplinary assessment team and a range of independent clinicians throughout Australia. Nine video-taped Autism Diagnostic Observation Schedule (ADOS)…

Background & Aims: This study ran within the framework of the Thales Aphasia Project that investigated the efficacy of elaborated semantic feature analysis (ESFA). We evaluated the treatment integrity (TI) of ESFA, i.e., the degree to which therapists implemented treatment as intended by the treatment protocol, in two different formats:…

One of the most important considerations in designing clinical infant research studies is the selection of reliable and valid measurement procedures. Few measures of caregiver-child interactions have been studied with newborns, particularly premature infants. The main objective of this study was to examine psychometric properties of the National…

Large-scale audiovisual data that measure group learning are time consuming to collect and analyze. As an initial step towards scaling qualitative classroom observation, we qualitatively coded classroom video using an established coding scheme with and without its audio cues. We find that interrater reliability is as high when using visual data…

In this book the authors explain how to build, and over time improve, the elements of an observation system that equips all observers to identify and develop effective teaching. It is based on the collective knowledge of key partners in the Measures of Effective Teaching (MET) project--which carried out one of the largest-ever studies of classroom…

During the pre- and post-implant habilitation process, mothers of children using cochlear implants may be coached by clinicians to use appropriate communicative strategies during play according to the family's choice of communication approach. The present study compared observations made by experienced and inexperienced individuals in the analysis…

This study used generalizability theory to measure reliability on the Recognizing Effective Special Education Teachers (RESET) observation tool designed to evaluate special education teacher effectiveness. At the time of this study, the RESET tool included three evidence-based instructional practices (direct, explicit instruction; whole-group…

Psychometric properties and initial validity of the Brief Observation of Social Communication Change (BOSCC), a measure of treatment-response for social-communication behaviors, are described. The BOSCC coding scheme is applied to 177 video observations of 56 young children with ASD and minimal language abilities. The BOSCC has high to excellent…

Teacher observations have become a national education phenomenon, mandated by federal policies and promoted by philanthropists. They are crucial components of teacher evaluation systems that often have high stakes for teachers and school systems, but have sparked little innovation. Recent calls to make teacher evaluations better, faster, and…

This article investigates the link between teachers' appraisal of specific interpersonal situations in classrooms and their more general interpersonal identity standard, which together form their interpersonal role identity. Using semi-structured and video-stimulated interviews, data on teachers' appraisals and interpersonal identity standards…

This study investigates the potential of video analysis and a mathematical knowledge for teaching framework, the Knowledge Quartet (KQ), in mathematics teacher education programmes. It reports on the effectiveness of these tools in analysing and supporting secondary level pre-service mathematics teachers' subject matter knowledge and pedagogical…

This paper presents the voice emotion recognition part of the FILTWAM framework for real-time emotion recognition in affective e-learning settings. FILTWAM (Framework for Improving Learning Through Webcams And Microphones) intends to offer timely and appropriate online feedback based upon learner's vocal intonations and facial expressions in order…