ERIC Number: ED422386
Record Type: Non-Journal
Publication Date: 1998-Apr-15
Pages: 19
Abstractor: N/A
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Comparative Approach to Evaluating Systemic Reform Policies: Applying Objective Measurement and Multilevel Analysis Methods.
Lee, Jaekyung
This study explores an alternative approach to educational program and policy evaluation by using two major educational measurement/analysis methods, and illustrates their integrated applications to evaluating state reform policies. Most evaluations have been done one program at a time, but it is desirable to design evaluation research in a way that compares the effectiveness of several programs that have the same objectives but different content or function on the same set of outcome measures. Applying item response theory to policy and practice survey provides an innovative solution to objective measurement of policies and practices. In addition, multilevel analysis methods would not only provide a means for formulating school and state-level regression models simultaneously but also provide more precise estimates of the extent to which state policies affect school practices. An illustrative study of state policy examines the multilevel linkages between state policies and educational outcomes. First, objective measures of state policies are created through application of the Rasch model. Then the multilevel education policy-practice linkages are examined through the application of the hierarchical linking model. As the results illustrate, the idea of comparing two groups of states on their policy outcome measures is similar to the nonequivalent control group design. However, the research design proposed in this paper differs from the nonequivalent control group design in some significant ways: (1) treatment is not a single program, but a set of programs; (2) group exposure is a matter of degree; (3) all of the programs that constitute treatment do not have to occur between pretest and posttest; and (4) subjects examined on pretest and posttest do not have to be the same, but can be sampled independently. The proposed approach should give more flexibility for evaluation design in real-life settings, but at the same time more difficulties for interpretation of evaluation results. Some concerns are reviewed. (Contains three tables, one figure, and eight references.) (SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A
Author Affiliations: N/A