ERIC Number: ED155215
Record Type: Non-Journal
Publication Date: 1978-Feb
Pages: 37
Abstractor: N/A
ISBN: N/A
ISSN: N/A
EISSN: N/A
Available Date: N/A
Impact on What?: The Importance of Content Covered. Research Series No. 2.
Porter, Andrew C.; And Others
Defining practical significance in program evaluations is a difficult measurement problem which can only be solved by an intimate familiarity with the measures on which effects are estimated and their content relationship to the program goals. Past attempts to provide general solutions to the size of effect problem have relied on standardized indices which can be estimated and reported without any knowledge of what was measured. Such efforts are viewed here as steps in the wrong direction. Instead, what is called for is a procedure whereby the content goals of the program, the content implied by a test, and the interrelationship between the two are made explicit. The procedure should investigate treatment-by-item interactions and at the same time, describe the measures used so that persons other than the evaluator can reach their own decisions about practical significance. Analysis of the mathematics sections of four major intermediate level standardized tests (Iowa Tests of Basic Skills, Metropolitan Achievement Tests, Stanford Achievement Tests, and California Test of Basic Skills) with their taxonomies indicated rather substantial differences in content tested. It was clear that standardized tests are not well suited to the task of estimating item domain by treatment interactions. (Author/CTM)
Publication Type: Reports - Research
Education Level: N/A
Audience: N/A
Language: N/A
Sponsor: National Inst. of Education (DHEW), Washington, DC. Basic Skills Group. Teaching Div.
Authoring Institution: Michigan State Univ., East Lansing. Inst. for Research on Teaching.
Grant or Contract Numbers: N/A
Author Affiliations: N/A