Interviews can provide valuable information for social researchers, but problems that may affect the quality of the information gathered should be addressed. These include subject-researcher reactivity, role relations, truth telling, reporting of the information collected, and researcher characteristics. A profile of effective interviewer…

Investigated psychometric properties of three methods of scoring a Mixed Standard Scale performance evaluation: a patterned procedure, simple nonpatterned scoring procedure and procedure assigning differential weights to statements on the basis of scale values provided by subject matter experts. Found no differences in the score distribution…

A FORTRAN subroutine is presented to calculate a generalized measure of agreement between multiple raters and a set of correct responses at any level of measurement and among multiple responses, along with the associated probability value, under the null hypothesis. (Author)

Process Skills Assessment is an observational assessment designed to evaluate process skills as demonstrated during the performance of a self-selected task. Retest reliability results indicated that assessment can be used to measure changes in clients' process skills after occupational therapy if the same task is used upon readministration.…

Presents a method for measuring interrater agreement on checklists. The technique computes a single agreement score from the concordance of raters' check mark configurations and derives an overall coefficient of agreement called "phi." A medical education study illustrates the phi methodology. (SLD)

In reply to criticism of the Rorschach Comprehensive System (CS) by J. Wood, M. Nezworski, and W. Stejskal (1996), this article presents a meta-analysis of published data indicating that the CS has excellent chance-corrected interrater reliability. It is noted that the erroneous assumptions of Wood et al. make their assertions about validity…

G. Meyer (1997) attempts to refute the present authors' criticisms of the interrater reliability of the Rorschach Comprehensive System (CS) but misrepresents their position and offers a flawed meta-analysis in support of his own. Rorschach proponents need to undertake high-quality replicated studies of CS reliability and validity. (SLD)

Replies to Wood et al. and documents limitations of their conclusions about the Rorschach Comprehensive System (CS), supporting Meyer's own meta-analysis, which finds adequate interrater reliability for the CS. (SLD)

This study examines the average reliability of Hare Psychopathy Checklists (PCLs) adapted for use in samples of youthful offenders (aged 12 to 21 years). Two forms of reliability are examined: 18 alpha estimates of internal consistency and 18 intraclass correlation (two or more raters) estimates of interrater reliability. The results, an average…

Purpose: The purpose of this study was to describe the Medication Management Instrument for Deficiencies in the Elderly (MedMaIDE) and to provide results of reliability and validity testing. Design and Methods: Participants were 50 older adults, aged 65 and older, who lived in the community, took at least one prescription medication, and were then…

Background: Difficulties experienced by novices in clinical reasoning have been well documented in many professions, especially medicine (Boshuizen and Schmidt 1992, 2000; Elstein, Shulman and Sprafka 1978; Patel and Groen 1986; Rikers, Loyens and Schmidt 2004). These studies have shown that novice clinicians have difficulties with both knowledge…

Background: There is a history of over-prescription of antipsychotics to individuals with intellectual disability (ID), while antidepressants may be under-prescribed. However, appropriate treatment is best supported when the diagnosis of psychosis or depression is valid and carries good predictive validity. The present authors report a study…

The National Council for Accreditation of Teacher Education (NCATE, 2002) requires teacher education units to develop assessment systems and evaluate both the success of candidates and unit operations. Because of a stated, but misguided, fear of statistics, NCATE fails to use accepted terminology to assure the quality of institutional evaluative…

Science education reform has long focused on assessing student inquiry, and there has been progress in developing tools specifically with respect to experimentation and argumentation. We suggest the need for attention to another aspect of inquiry, namely "mechanistic reasoning." Scientific inquiry focuses largely on understanding causal…

This article examines the influence of keyboarding versus handwriting in a high-stakes writing assessment. Conclusions are based on data collected from a pilot project to move Old Dominion University's Exit Exam of Writing Proficiency from a handwritten format into a dual-option format (i.e., the students may choose to handwrite or keyboard the…