Though qualitative research has become more prevalent in practice over the last 30 years, there is still considerable uncertainty among researchers regarding how to ensure inter-rater consistency when teams are tasked with coding qualitative data. In this article, we offer an explanation of a methodology our qualitative team used to achieve…

Single-case design is a research methodology that entails repeated measurement to assess the influence of an independent variable on a dependent variable over time. Data collected in this manner are regularly analyzed using visual analysis of data displayed in a linear graph. Although there is agreement regarding critical elements of visual…

Background: Cleft lip and palate (CLP) treatment in Norway is centralized and multidisciplinary, with long-term follow-up from birth to adulthood. The Norwegian Registry of Cleft Lip and Palate was established to ensure high-quality care and enable systematic data collection. Speech data are a key component, assessed by speech--language therapists…

Single-case research design is a useful methodology for evaluating the presence of a functional relation between an intervention and the mathematical performance of students with a learning disability. However, a functional relation cannot be established with threats to internal validity of the design. External validity is impacted if researchers…

There is growing interest in using measures of teacher applicant quality to improve hiring decisions, but the statistical properties of such measures are poorly understood. We present evidence on structured ratings solicited from teacher applicants' references. We find that the reference ratings capture only one underlying dimension of applicant…

Background: Speech-language pathologists often multitask in order to be efficient with their commonly large caseloads. In stuttering assessment, multitasking often involves collecting multiple measures simultaneously. Aims: The present study sought to determine reliability when collecting multiple measures simultaneously versus individually.…

This paper discusses the challenges and lessons learned from conducting observations to measure the quality of classroom practice for a large-scale study of elementary teachers' mathematics instruction. Specifically, this paper shares our process for obtaining valid data for quality of elementary mathematics instruction; what we learned can inform…

We developed a brief, informant-report interview for assessing autism spectrum conditions (ASC) in adults, called the Developmental, Dimensional and Diagnostic Interview-Adult Version (3Di-Adult); and completed a preliminary evaluation. Informant reports were collected for participants with ASC (n = 39), a non-clinical comparison group (n = 29)…

The current study was designed to evaluate a rotating momentary time sampling (MTS) data collection system. A rotating MTS system has been used to measure activity preferences of preschoolers but not to collect data on responses that vary in duration and frequency (e.g., talking). We collected data on talking for 10 preschoolers using a 5-s MTS…

Second language (L2) researchers often have to develop or change the instruments they use to measure numerous constructs (Norris & Ortega, 2012). Given the prevalence of researcher-developed and -adapted data collection instruments, and given the profound effect instrumentation can have on results, thorough reporting of instrumentation is…

An ePortfolio Assessment Institute (AI) structured as a faculty development opportunity was undertaken to increase faculty confidence in teaching and assessing ePortfolios and to collect reliable data about student performance on four learning outcomes associated with an institutionwide ePortfolio initiative. Faculty raters participated in the…

Systematic reviews of literature are studies that strategically search for published research on a specific topic in order to synthesize what is known about the topic. This systematic review describes 157 articles on synchronous online learning (SOL) from thirty-four different countries on instructional setting, content areas, participant…

Through a year-long study of welfare-to-work students in the community college CalWORKs program, we investigated what self-authorship development looks like by examining developmental progress, and whether there are patterns in development along the three dimensions of self-authorship. Findings demonstrate progress toward self-authorship, but…

Out-of-school time (OST) youth programs are inherently difficult to assess. They are often very dynamic: Many youth interact with one another and with staff members in various physical environments. Despite the challenge, measuring quality is critical to help program directors and policy makers identify where to improve and how to support those…

Objective: Although health professionals increasingly are expected to be able to assess and manage patients' risk for suicide, few methods are available to evaluate this competency. This report describes development of a competency-assessment instrument for suicide risk-assessment (CAI-S), and evaluates its use in an objective structured clinical…