Observation of counsellor skills through a one-way mirror, video or audio recording followed by supervisors and peers feedback is common in counsellor training. The nature and extent of agreement between supervisor-peer dyads are unclear. Using a standard scale, supervisors and peers rated 32 interviews by psychology trainees observed through a…

This paper outlines the process and results of an authentic assessment of student work using a revised version of the AAC&U's Information Literacy VALUE rubric. This rigorous assessment, which included the scoring of nearly 900 student papers from four different stages across the undergraduate curriculum, revealed much about the process of…

The Mini-Mental State Examination (MMSE) is one of the most commonly used instruments to screen for cognitive deficits within the hospital setting. However training in how to administer this widely used tool is scarce with little, if any, formal training for nursing staff. Scores are also often misused with over reliance on results and cut-offs to…

Possible sources of intrajudge inconsistency in standard setting are reviewed, and approaches are presented to improve the accuracy of rating. Procedures for providing judges with feedback through discussion or computerized communication are discussed. Monitoring and maintaining judges' consistency throughout the rating process are essential. (SLD)

An approach is presented to the definition of minimal competence for judges to use in standard setting. Panelists in standard setting must receive training to ensure that differences in rating result from differences in perceptions of item difficulty, not in differences of opinion about the definition of minimal competence. (SLD)

Training judges to generate item ratings in standard setting once the reference group has been defined is discussed. It is proposed that sensitivity to the factors that determine difficulty can be improved through training. Three criteria for determining when training is sufficient are offered. (SLD)

Although arbitrary, whenever multiple judgmental standard-setting procedures are utilized by different groups concurrently, stability across raters can be achieved and decisions can be made in a relatively judicious manner. Greater stability across methods (Ebel, Nedelsky, Angoff) may be effected by slightly modifying the Ebel approach. (Author/PN)

The impact of overall test length and difficulty on the expert judgments of item performance by the Nedelsky method were studied. Five university-level instructors predicting the performance of minimally competent candidates on a mathematics examination were fairly consistent in their assessments regardless of length or difficulty of the test.…

A panel of 24 public school teachers and 37 college/university faculty members provided recommendations on minimal standards for the essay portion of the National Teacher Examinations Communication Skills Test. Public school judges' recommendations were significantly more variable than were those of college/university judges. (TJH)