The use of inter-rater reliability (IRR) methods may provide an opportunity to improve the transparency and consistency of qualitative case study data analysis in terms of the rigor of how codes and constructs have been developed from the raw data. Few articles on qualitative research methods in the literature conduct IRR assessments or neglect to…

Purpose: This study presents a novel methodology/process for assessing inclusion of theoretically-based implementation factors within available adoption-ready health promotion programs. Methods: Classroom-based physical activity (CBPA) programs were used as an example to describe the process. Our team selected an implementation science framework…

The current study describes the development of a content analysis coding scheme to examine transcripts of online asynchronous discussion groups in higher education. The theoretical framework comprises the theories regarding knowledge construction in computer-supported collaborative learning (CSCL) based on a sociocultural perspective. The coding…

This study evaluated a process for training raters to reliably rate clinicians delivering the Adolescent Community Reinforcement Approach (A-CRA) in a national dissemination project. The unique A-CRA coding system uses specific behavioral anchors throughout its 73 procedure components. Five randomly selected raters each rated "passing"…

The authors present a model for program-wide assessment of the effectiveness of writing instruction in a chemistry laboratory course. This model, which involves collaboration between faculty from chemistry, the Writing Across the Curriculum (WAC) program, and the Faculty Center for Teaching and Learning, is based on several theories and…

Although interview formats support rich data collection in conceptual change studies, interview formats limit sample sizes. This study explores the possibility of using constructed-response formats as an alternative or supplement for collecting similarly rich data across larger pools of subjects in conceptual change studies. While research in…

We discuss the utility of a coding system designed to evaluate the amount and type of parent talk during individualized family service plan (IFSP) meetings. The iterative processes used to develop the "Parent Communication Coding System" (PCCS) and its associated codes are described. In addition, we explored whether PCCS codes could be…

Research Findings: The purpose of this study was to describe differences in the narratives produced by 3-, 4-, and 5- year old Spanish-speaking (SS) children. Narrative productions of 104 typically developing children were collected using a story-retelling task and coded using the Index of Narrative Complexity. The results of this study indicate…

Investigations of contributors to and consequences of the parent-child relationship require accurate assessment of the nature and quality of parenting. The present study describes the development and psychometric evaluation of the Laboratory Parenting Assessment Battery (Lab-PAB), an observational rating system that assesses parenting behaviors…

A coding method for measuring reflective thinking in student journals was tested twice, demonstrating acceptable reliability among evaluators and supporting the precision of the guidelines for coding. Coding categories were as follows: habitual action, introspection, thoughtful action, content reflection, process reflection, content and process…

The traditional kappa statistic in assessing interrater agreement is not adequate when multiraters and multiattributes are involved. In this article, latent trait models are proposed to assess the multirater multiattribute (MRMA) agreement. Data from the Third International Mathematics and Science Studies (TIMSS) are used to illustrate the…

Investigates how coder characteristics such as language skills, political knowledge, coding experience, and coding certainty affected inter-coder and coder-training reliability. Shows that language skills influenced both reliability types. Suggests that cross-national researchers should pay more attention to cross-national assessments of…

In this article, we extend previous work with respect to interrater reliability measure of computer-mediated conferencing and suggest coding categories relevant to problem-based learning. Calculating interrater reliability agreement by using a Transcript Reliability Cleaning Percentage (TRCP) approach is simple for academics with limited…

Networked learning aims to foster students' knowledge construction processes as well as the quality of knowledge construction. In this respect, it is crucial to be able to analyse both aspects of networked learning. Based on theories on networked learning and the empirical work of relevant authors in this domain, two coding schemes are presented…