ERIC Number: EJ1200970
Record Type: Journal
Publication Date: 2018-Dec
Pages: 9
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0731-1745
EISSN: N/A
Available Date: N/A
Measuring Widening Proficiency Differences in International Assessments: Are Current Approaches Enough?
Rutkowski, David; Rutkowski, Leslie; Liaw, Yuan-Ling
Educational Measurement: Issues and Practice, v37 n4 p40-48 Win 2018
Participation in international large-scale assessments has grown over time with the largest, the Programme for International Student Assessment (PISA), including more than 70 education systems that are economically and educationally diverse. To help accommodate for large achievement differences among participants, in 2009 PISA offered low-performing systems the option of including an easier set of items in the assessment with an aim of providing improved achievement estimates. However, there remains a lack of evidence on the performance of this design innovation. As such, we simulate a design that closely mirrors the PISA 2015 math assessment in order to empirically examine the benefits of including easy items for low-performing countries. We extend the PISA design to include increased numbers of easy items and items that are easier than currently implemented. Findings show that the current PISA approach provides little advantage compared to a common test for all participants. Our study also demonstrates persistent bias, low coverage rates, and low correlations between generating and estimated proficiency under current designs. Through our simulation we also show that to improve achievement estimation for low performers about half of the items would need to be made significantly easier.
Descriptors: Educational Assessment, Foreign Countries, Achievement Tests, Secondary School Students, International Assessment, Test Items, Difficulty Level, Mathematics Tests, Test Bias, Simulation, Mathematics Achievement
Wiley-Blackwell. 350 Main Street, Malden, MA 02148. Tel: 800-835-6770; Tel: 781-388-8598; Fax: 781-388-8232; e-mail: cs-journals@wiley.com; Web site: http://www.wiley.com.bibliotheek.ehb.be/WileyCDA
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Program for International Student Assessment
Grant or Contract Numbers: N/A
Author Affiliations: N/A