ERIC Number: ED521965
Record Type: Non-Journal
Publication Date: 2009
Pages: 13
Abstractor: ERIC
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Evaluating the Effect of Random Selection on Virtual Comparison Group Creation
Ma, Lingling; Cronin, John
Northwest Evaluation Association
Virtual Comparison Groups (VCG) were developed by the Northwest Evaluation Association as an alternative to conventional controlled experiments for social science researchers working in the field of education. The VCG is generally a group of up to 51 students who are matched, based on key characteristics of the student and school, to a single student who is part of a study group. This study evaluated the procedures for selecting students to be used as Virtual Comparisons for a study group of 1,000 students in an effort to determine whether the random selection procedures created differences between the VCG sample and the population from which it was drawn. In summary, this study finds that the procedures for randomly selecting cases for membership in a VCG do not create samples that differ significantly from the population group from which they are drawn. The study also shows that while VCG samples above 40 do not necessarily improve the stability of VCG estimates across a large population of students, they greatly improve the stability of individual VCG estimates, thus making the individual comparisons more valuable to teachers. The study informs the need for future research into how the VCG criteria themselves and the order in which they are applied are likely to impact the nature and performance of a Virtual Comparison Group and its validity as a means for comparing student performance to a control in a quasi-experimental setting. Virtual Comparison Group Selection Procedure is appended. (Contains 2 tables and 6 figures.)
Descriptors: Social Science Research, Comparative Analysis, Sampling, Student Characteristics, Institutional Characteristics, Computer Assisted Testing, Research Methodology, Evaluation Methods, Test Validity, Quasiexperimental Design, Grade 5, Elementary School Students, Mathematics Achievement
Northwest Evaluation Association. 121 NW Everett Street, Portland, OR 97209. Tel: 503-624-1951; Fax: 503-639-7873; Web site: http://nwea.org
Publication Type: Reports - Research
Education Level: Elementary Education; Grade 5
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: Northwest Evaluation Association
Grant or Contract Numbers: N/A
Author Affiliations: N/A