The authors' purpose was to evaluate the multiple-gate Preschool Behavior Screening System (PBSS) for identifying children's social, emotional, and behavioral (SEB) difficulties. The sample included 122 triads of preschool children between 3 and 5 years old and their parents and teachers. The teacher and parent samples were predominantly women and…

The Psychoeducational Profile 3rd Edition (PEP-3) is a comprehensive assessment tool designed for children with autism spectrum disorder (ASD). Although its original English version has been validated, few validation studies have been conducted on translated versions including Chinese ones. Based on 554 Chinese children with ASD and 311 typically…

We determined inter-rater agreement for the VB-MAPP, an instrument sometimes used in planning educational goals and evaluating intervention effects for young people with autism. A pair of raters independently rated each of 32 children diagnosed with autism. Intraclass correlation coefficients for the total Milestones and Barrier scores were 0.876…

The purpose of study is to examine the reliability of analytical rubrics and checklists developed for the assessment of story writing skills by means of generalizability theory. The study group consisted of 52 students attending the 5th grade at primary school and 20 raters in Mersin University. The G study was carried out with the fully crossed…

In Australia, one of the key findings of the Teacher Education Ministerial Advisory Group was that not all graduating pre-service teachers possess adequate pedagogical content knowledge (PCK) to teach effectively. The concern is that higher education providers working with pre-service teachers are using pedagogical practices and assessments which…

Engagement in early childhood is defined as a child's level of participation with the environment. Engagement is an important construct in assessment and intervention of social and early learning competence given its link to school achievement. Few tools exist to assess engagement of young children in early education, and there is a need for a…

Research Findings: We evaluated the score stability of the Mathematical Quality of Instruction (MQI), an observational measure of mathematics instruction. Three raters each scored, independently, 100 video-recorded lessons taught by 20 kindergarten teachers in the spring. Using generalizability theory analyses, we decomposed the MQI's score…

The use of assessments that require rater judgment (i.e., rater-mediated assessments) has become increasingly popular in high-stakes language assessments worldwide. Using a systematic literature review, the purpose of this study is to identify and explore the dominant methods for evaluating rating quality within the context of research on…

Although inter-rater reliability is an important aspect of using observational instruments, it has received little theoretical attention. In this article, we offer some guidance for practitioners and consumers of classroom observations so that they can make decisions about inter-rater reliability, both for study design and in the reporting of data…

The purpose of this study was to find the effect of English Language Teachers' Methodology (ELTM) on the Career Growth (CG) of the Saudi students. In order to provide a solid basis for this research study, a cross-sectional-descriptive research design was employed. For scale development and tool standardization, inter-class correlation…

This paper explores the concept of dynamically generating definitions using a deep-learning model. We do this by creating a dataset that contains definition entries and contexts associated with each definition. We then fine-tune a GPT-2 based model on the dataset to allow the model to generate contextual definitions. We evaluate our model with…

Many classroom-observation instruments have been developed (e.g., Gleason et al., 2017; Nava et al., 2019; Sawada et al., 2002), but a very small number of studies published in refereed journals have rigorously examined the quality of the ratings and the instrument using measurement models. For example, Gleason et al. developed a mathematics…

The biggest challenge facing American public schools today is increasing student achievement, and this depends on the educational system's ability to improve the quality of instruction. Principals play a key role in improving the quality of instruction on their campus through the instructional leadership practices they engage in on a daily basis.…

The purpose of this study was to build a Random Forest supervised machine learning model in order to predict musical rater-type classifications based upon a Rasch analysis of raters' differential severity/leniency related to item use. Raw scores (N = 1,704) from 142 raters across nine high school solo and ensemble festivals (grades 9-12) were…

Rater cognition or "think-aloud" studies have historically been used to enhance rater accuracy and consistency in writing and language assessments. As assessments are developed for new, complex constructs from the "Next Generation Science Standards (NGSS)," the present study illustrates the utility of extending…