ERIC Number: EJ1460959
Record Type: Journal
Publication Date: 2025-Dec
Pages: 26
Abstractor: As Provided
ISBN: N/A
ISSN: N/A
EISSN: EISSN-2229-0443
Available Date: 2025-01-28
Revisiting the Effectiveness of a Performance Decision Tree-Style Rubric Compared to a Grid-Style Rubric
Language Testing in Asia, v15 Article 8 2025
The performance decision tree (PDT; Fulcher et al., 2011) is a rubric style that is applicable to performance assessment, with origins in Upshur and Turner's (1995) empirically derived binary-choice, boundary-definition (EBB) scale. It is easier for raters to assess performance by evaluating multiple binary-choice descriptors. Additionally, learners receive easy-to-understand feedback. The most important advantage is that the number of feedback types can be increased to the nth power of 2 (n = number of items). Therefore, this study compared the validity, reliability, and practicality of the PDT and grid-style rubrics. Three Japanese university English instructors participated in the preliminary work to create the rubric, and the contributors scrutinized the information to create a pilot version. Five raters (two English lecturers and three student raters) then scored 64 Japanese university students' performances in four English-speaking tasks. Next, analyses were conducted using the many-facet Rasch measurement and generalizability theory to observe the validity and reliability of the PDT. The results confirmed that both the PDT and the grid had validity within the range that could be examined and that the reliability was sufficiently high. However, interviews with raters revealed that the PDT had superior practicality. Additionally, the PDT took less time to complete the evaluation than the grid and was judged to be an effective form of the rubric. This article is part of a study aimed at demonstrating the usefulness of the PDT by creating and evaluating a pilot version. It was used to partially examine the validity and reliability of the PDT. The remaining aspects of validation shall be presented in a separate article.
Descriptors: Scoring Rubrics, Second Language Learning, Second Language Instruction, Language Teachers, English (Second Language), Feedback (Response), Foreign Countries, Performance Based Assessment, Evaluators, College Faculty, College Students, Student Evaluation, Validity, Decision Making, Reliability, Oral Language, Speech Communication
Springer. Available from: Springer Nature. One New York Plaza, Suite 4600, New York, NY 10004. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-460-1700; e-mail: customerservice@springernature.com; Web site: https://link-springer-com.bibliotheek.ehb.be/
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Japan
Grant or Contract Numbers: N/A
Author Affiliations: 1Fukushima University, Fukushima, Japan