Classroom observations remain the predominant data source used in teacher evaluations, but little is known about how rater characteristics may affect teachers' scores. For special educators, whose instructional practice requires specialized knowledge and skills, school administrators (i.e., the raters) without experience in special education…

Implementation of research-based, Tier 1 behavior management strategies can be monitored to provide data-driven feedback and in support of integrity. The "Measure of Active Supervision and Interaction" (MASI) was developed to measure four behavior management practices (i.e., Praise, Correction, References to Behavior Expectations, Active…

Scores from noncognitive measures are increasingly valued for their utility in helping to inform postsecondary admissions decisions. However, their use has presented challenges because of faking, response biases, or subjectivity, which standardized third-party evaluations (TPEs) can help minimize. Analysts and researchers using TPEs, however, need…

Students with Emotional and Behavioural Disorders (EBD) have the poorest academic and social outcomes across the general and special education student populations, and are among the most likely to receive instruction in self-contained special education classrooms hallmarked by small teacher-student ratios, frequent transitions, extreme student…

Background & Context: Accurate, reliable, and scalable measurement of classroom quality is critical to ensure that children benefit from early childhood programs. Early childhood classroom quality is most commonly measured using classroom observation tools serving multiple purposes including (1) teacher support (e.g., coaching, professional…

In face-to-face surveys, interviewer observations are a cost-effective source of paradata for nonresponse adjustment of survey estimates and responsive survey designs. Unfortunately, recent studies have suggested that the accuracy of these observations can vary substantially among interviewers, even after controlling for household-, area-, and…

Bilingual literature for children is valuable in encouraging literacy in second language learners. Stories can enhance vocabulary and language abilities, learning encounters, subject content, social aptitude, and other skills in the early reader through text as well as illustrations. This paper explores issues in selecting quality dual language…

This study aimed to assess the validity and reliability of jump assessments using the "MyJump2" application. Eleven junior athletes (15 ± 1.4 years) performed five countermovement (CMJ) and drop jumps (DJ) measured simultaneously by a force platform and "MyJump2." Additionally, intra- and inter-day reliability was assessed over…

Purpose: This study presents a novel methodology/process for assessing inclusion of theoretically-based implementation factors within available adoption-ready health promotion programs. Methods: Classroom-based physical activity (CBPA) programs were used as an example to describe the process. Our team selected an implementation science framework…

Children from low-income households are at risk for entering school behind their more economically advantaged peers across major domains of school readiness. The Head Start program represents the federal government's response to these achievement gaps by mandating the use of scientifically based assessments and curricula to provide children with…

Though treatment integrity measurement is important for research intended to promote social and behavioral outcomes of children at risk for emotional and behavioral disorders (EBDs) in early childhood settings, measurement gaps exist in the field. This paper reports on the development and preliminary psychometric assessment of the treatment…

Drawing parallels to classical test theory, this article clarifies the difference between rater accuracy and reliability and demonstrates how category marginal frequencies affect rater agreement and Cohen's kappa. Category assignment paradigms are developed: comparing raters to a standard (index) versus comparing two raters to one another…

Assessing global interrater agreement is difficult as most published indices are affected by the presence of mixtures of agreements and disagreements. A previously proposed method was shown to be specifically sensitive to global agreement, excluding mixtures, but also negatively biased. Here, we propose two alternatives in an attempt to find what…

In the study, it was aimed to investigate the leniency/severity, bias and halo effect of the raters which were used in the scoring of the diagnostic tree prepared by the teacher candidates with the many-facet Rasch model. The research study group constitutes 24 teacher candidates who are taking measurement and evaluation lesson from the students…

In 2015, the American Council on Education surveyed undergraduate admission and enrollment management leaders at 338 four-year institutions to understand holistic admissions review (Espinosa, Gaertner, and Orfield, 2015). In the report titled, Race, Class and College Access: Achieving Diversity in a Shifting Legal Landscape, 92% of selective…