NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED642242
Record Type: Non-Journal
Publication Date: 2021
Pages: 135
Abstractor: As Provided
ISBN: 979-8-7806-5939-6
ISSN: N/A
EISSN: N/A
Available Date: N/A
Comparison of DIF Methods for the Student Experience in the Research University Survey: A Validity and Methodological Study
Thapelo Ncube Whitfield
ProQuest LLC, Ph.D. Dissertation, The University of Iowa
Student Experience surveys are used to measure student attitudes towards their campus as well as to initiate conversations for institutional change. Validity evidence to support the interpretations of these surveys' results, however, is lacking. The first purpose of this study was to compare three Differential Item Functioning (DIF) methods on Likert-scaled survey data as one facet of internal structure validity evidence for the Student Experiences in the Research University (SERU) survey. Specifically, DIF analyses were performed to provide evidence for the comparison of survey scores across administrations (years), and between student demographic groups (race, gender, first-generation student status). The second goal of the study was to compare the performance of DIF aggregation methods (Combined SMD and Average SMD) to a commonly used traditional DIF method (SMD/Mantel).The study included both real data analysis and a simulation study component to evaluate the nature of the three DIF methods under various conditions. In the real data analysis, data from three administrations of the SERU survey were evaluated for DIF using each of the three DIF methods. For the simulation portion, the power and Type-I error rates for each method were evaluated under varying conditions of trait (ability) distribution, sample size, and DIF level (small, medium, and large).The results found that the DIF aggregation methods may be useful in providing more accurate information regarding DIF on survey instruments, especially when there may be concerns that traditional DIF methods ignore (e.g., low focal group sample sizes for a single administration). The final chapter of this study provides SERU survey developers with information on the validity evidence supporting certain subgroup score comparisons as well as information on potentially biased items that may need to be reassessed or field tested. Furthermore, both SERU survey developers and developers of similar student experience surveys can use this dissertation as a template for collecting internal structure validity evidence by way of DIF. Lastly, education and psychometrics researchers can use the results of this study to assess the behavior of DIF aggregation methods on survey data. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com.bibliotheek.ehb.be/en-US/products/dissertations/individuals.shtml.]
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com.bibliotheek.ehb.be/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A