Statistical thinking is required for good statistical analysis. Among other things, statistical thinking involves identifying sources of variation. Students in introductory statistics courses seldom recognize that one of the largest sources of variation may come in the collection and recording of the data. This paper presents some simple exercises…

Although the rubric has emerged as one of the most popular assessment tools in progressive educational programs, there is an unfortunate dearth of information in the literature quantifying the actual effectiveness of the rubric as an assessment tool "in the hands of the students." This study focuses on the validity and reliability of the rubric as…

Online or web-based peer assessment is a valuable and effective way to help the learner to examine his or her learning progress, and teachers need to be familiar with the practice before they use it in their classrooms. Therefore, the purpose of our study was to design an online peer assessment activity for 37 inservice science and mathematics…

This qualitative study analyzes how the most recent community college film, Evolution (2001) depicts and portrays the image of a community college as interpreted by attending community college students. Previous community college research suggests that college choice, enrollment, and funding may be affected by perceived image. Image is greatly…

This study evaluated a "substantively driven" method for scoring NAEP writing assessments automatically. The study used variations of an existing commercial program, e-rater[R], to compare the performance of three approaches to automated essay scoring: a "brute-empirical" approach in which variables are selected and weighted solely according to…

The psychometric properties of a version of the Course Experience Questionnaire revised for students currently enrolled at the University of Sydney, the Student Course Experience Questionnaire (SCEQ), were assessed, gathering students' perceptions on a number of scales, including Good Teaching, Clear Goals and Standards, Appropriate Assessment,…

A model-based approach to rater reliability for essays read by multiple readers is presented. Variation of rater severity (between-rater variation) and rater inconsistency (within-rater variation) is considered in the presence of between-examinee variation. An additive variance component model is posited and the method of moments for its…

Administration of the Test of Spoken English (TSE) yields tapes of oral performance on items within six sections of the test. Trained scorers subsequently rate responses using four proficiency scales: pronunciation, grammar, fluency, and overall comprehensibility. This project examined the consistency of statistical relations among TSE scores with…

This report addresses the reliability of the American Council on the Teaching of Foreign Languages (ACTFL) Oral Proficiency Interview (OPI), not as a measure of speaking ability, but rather as practiced by testers trained by the ACTFL, such as by the Interagency Language Roundtable (ILR), in English as a Second Language (ESL), French, German,…

The "Writing What You Read" (WWYR) rubric was designed for large-scale assessments, and differs from most narrative rubrics in its narrative-specific content and its developmental framework. The rubric contains five analytic subscales for theme, character, setting, plot, and communication, and a sixth holistic scale for overall…

The Quebec Adaptive Behavior Scale (QABS) is widely used in Quebec (Canada) to assess behavior of people with mental retardation in educational, vocational, residential or hospital settings. This study estimated the interrater agreement and test-retest reliability of the QABS. To determine test-retest reliability, the QABS was completed by 27…

This overview of the 40 year development of language proficiency rating at the Foreign Service Institute (FSI) and elsewhere in the federal government focuses on three issues pertaining to the scale and the raters: (1) the number of levels of differentiation in the scale; (2) the relation of the scale to the rating task; and (3) the calibration of…

This study examined the relationship of teachers' and parents' ratings of the employability skills of 15 students (ages 8-16) with multiple disabilities. Five teachers and 14 sets of parents used the Ohio Employability/Life Skills Assessment, Parent Form to assess student mastery of employability attitudes and skills. Significant differences were…

A four-level taxonomy is proposed to define the usefulness of rater written comments for supporting letters of recommendation. The taxonomy is used to classify comments on 220 rating forms by 25 raters from two surgery departments regarding performance by third-year medical students. Written comments were classified by the following taxonomy: (1)…

The reliability and validity of teacher ratings, the relationship between teacher ratings and principal self-reports of instructional leadership, and the degree to which they are influenced by demographic factors are examined in this study. Methodology involved completion of the Instructional Leadership Inventory, a self-report measure, by 81…