Even when raters demonstrate agreement in the use of a measure, limited score variability or violation of often-ignored statistical assumptions can result in lower reliability estimates than intuitively expected. This article uses data drawn from two randomized controlled trials of schema therapy and cognitive behavioral therapy for the treatment…

Jeffrey Beall's blog listing of potential predatory journals and publishers, as well as his "Criteria for Determining Predatory Open-Access (OA) Publishers" are often looked at as tools to help researchers avoid publishing in predatory journals. While these "Criteria" have brought a greater awareness of OA predatory journals,…

This article describes the core features of outcome research and then explores issues confronting researchers who engage in outcome studies. Using an intervention research perspective, descriptive and explanatory methods are distinguished. Emphasis is placed on the counterfactual causal perspective, designing programs that fit culture and context,…

This guide is designed to help researchers make sure that their research reports include enough information about study measures so that readers can assess the quality of the study's methods and results. The guide also provides examples of write-ups about measures and suggests resources for learning more about these topics. The guide assumes…

Quantitative behaviour analysis requires the classification of behaviour to produce the basic data. This can be challenging when the theoretical taxonomy does not match observational limitations, or if a theoretical taxonomy is unavailable. Binary keys allow qualitative observation to be used to modify a theoretical taxonomy to produce a practical…

The reliability of a new scale to assess Autistic Disorder, Pervasive Developmental Disorder, Not Otherwise Specified (PDD-NOS), and Asperger's Disorder in children was examined. Parents or other caregivers rated symptoms of 207 children between 2 and 16 years of age. The scale, which had 40 items in the final version, correlated highly with…

Marking reliability is purported to be produced by having an effective community of practice. No experimental research has been identified which attempts to verify empirically the aspects of a community of practice that have been observed to produce marking reliability. This research outlines what that community of practice might entail and…

Until recently, research studies have implied that domestic violence does not affect Asian American and immigrant communities, or even Asians abroad, because ethnicity or culture has not been addressed. In this content analysis, the authors examined trends in publications in leading scholarly journals on violence relating to Asian women and…

The traditional kappa statistic in assessing interrater agreement is not adequate when multiraters and multiattributes are involved. In this article, latent trait models are proposed to assess the multirater multiattribute (MRMA) agreement. Data from the Third International Mathematics and Science Studies (TIMSS) are used to illustrate the…

Coding systems have become popular methods of cataloging the verbal and nonverbal interaction occurring during marital and family therapy. One such system, Pinsof's (1981) Family Therapist Coding System (FTCS), was the first designed explicitly to identify and differentiate specific verbal behaviors of family therapists independent of their…

This paper reviews the prominent concepts of the stuttering event and concerns about the reliability of stuttering event measurements, specifically interjudge agreement. Recent attempts to resolve the stuttering measurement problem are reviewed, and the implications of developing an improved measurement system are discussed. (Author/JDD)

In this study, 30 judges identified disfluency types they perceived in audiovisually recorded speech stimuli, first individually and then with a partner. Although intrapair and interpair agreement was higher in the partner than the individual condition, agreement for occurrences still averaged below 50 percent. Findings suggest caution in use of…

Identifies three sources of reporting deficiency for meta-analytic results: quality (adequacy) of publicizing; quality of macrolevel reporting, and quality of microlevel reporting. Reanalysis of 25 reports from the Smith, Glass and Miller (1980) psychotherapy meta-analysis established two sources of misinformation, interrater reliabilities and…

This paper contends that behavior observation data relating to speech-language pathology are reliable if they are not affected by differences among observers or other variations in the recording context. The theoretical bases of methods used to estimate reliability for observational data are reviewed, and suggestions are provided for improving the…

Sixteen listeners (10 expert, 6 naive) judged the dissimilarity of pairs of voices drawn from pathological and normal populations. Only parameters that showed substantial variability were perceptually salient across listeners. Results suggest that traditional means of assessing listener reliability in voice perception tasks may not be appropriate.…