In the present study, I examined the effects of rater characteristics, in particular, raters' familiarity with a foreign accent, on the assessment of second language (L2) pronunciation. Forty-three native English-speaking teachers were divided into three groups according to their reported types of familiarity with Korean accents: heritage,…

L2 pragmatics have explored the effects of different factors on different aspects of learners' pragmatic performance, but often not simultaneously. In addition, syntactic complexity is rarely examined in L2 pragmatics. This cross-sectional study aimed to conduct a multidimensional analysis to explore the effects of proficiency and study-abroad…

Background: Although teachers of English are required to assess students' speaking proficiency in the Common European Framework of Reference for Languages (CEFR), their ability to rate is seldom evaluated. The application of descriptors in the assessment of English speaking on CEFR in the context of English as a foreign language has not often been…

As a result of the fact that judgments of non-native speech are closely tied to social biases, oral proficiency ratings are susceptible to error because of rater background and social attitudes. In the present study we seek first to estimate the variance attributable to rater background and attitudinal variables on novice raters' assessments of L2…

The emergence of computer technology in English language teaching has paved the way for teachers' application of Mobile Assisted Language Learning (mall) and its advantages in teaching. This study aimed to compare the effectiveness of the face to face instruction with Telegram mobile instruction. Based on a toefl test, 60 English foreign language…

The present study aimed at characterizing what skilled or more proficient ESL college writing is in the Philippine setting through a contrastive analysis of three groups of variables identified from previous studies: resources, processes, and performance of ESL writers. Based on Chenoweth and Hayes' (2001; 2003) framework, the resource level…

Much research has investigated the role of planning time in second language writing; however, the results show that there are inconsistent findings about the effects of planning time conditions on the complexity of the EFL learners' textual output. The current study attempted to consider the differential effects of planning time conditions in…

This study attempted to unveil the effect of metadiscourse instruction on the improvement of the speaking ability of Iranian EFL learners. After the administration of a language proficiency test, 34 homogeneous participants were assigned into the experimental and control groups. Then, the two groups were compared on their speaking ability. After…

This study investigated the interrater reliability of proficiency-level judgments of graduate student trainee raters on oral proficiency interviews (OPIs). Trainees' ratings were compared with the judgments of a certified American Council on the Teaching of Foreign Languages (ACTFL) tester for 150 interviews. (Author/JL)

This paper examines whether individuals who train themselves to score a performance assessment will rate acceptably when compared to known standards. Research on the efficacy of rater self-training materials developed by the Center for Applied Linguistics for the Texas Oral Proficiency Test (TOPT) is examined. Rater self-materials are described…

A study of the reliability of the proficiency ratings scale and techniques used by three federal government agencies--the Central Intelligence Agency, the Defense Language Institute, and the Foreign Service Institute (FSI)--to test employees' oral language proficiency in French and German had two randomly selected two-person teams of testers from…

Differences in procedures used by academic institutions and government agencies in administering the American Council on the Teaching of Foreign Languages' Oral Proficiency Interview test are examined, and results and implications of two studies of interrater reliability are discussed. (MSE)

A preliminary study is reported of the use of new multifaceted Rasch measurement mechanisms for investigating rater characteristics in language testing. Ratings from four judges of scripts from 50 candidates taking the International English Language Testing System test, a test of English for Academic Purposes, are analyzed. The analysis…

A group of seven Japanese-language Oral Proficiency Interview (OPI) testers licensed by the American Council on the Teaching of foreign Languages (ACTFL) conducted research related to ACTFL-OPI criteria. They first examined 24 audiotaped interview tests to see what kind of consistency there would be when individual testers applied general criteria…

A prototype revised form of the Test of Spoken English (TSE) was compared with the current version of the same test, comparing interrater reliability, frequency of rater discrepancy at all score levels, component task adequacy, scoring efficacy, and other concurrent and construct validity evidence, including the oral proficiency interview…