The reason for conducting the present case study was the problems the researchers encountered during data collection for another research project (Primary Study) entitled "The effects of virtual versus traditional flipped classes on EFL learners' grammar knowledge, self-regulation, and autonomy." Two online questionnaires were…

The results of large-scale international assessments such as Programme for International Student Assessment (PISA) have attracted a considerable attention worldwide and are often used by policy-makers to support educational policies. To ensure that the published results represent the actual population, these surveys go through a thorough scrutiny…

Background/Context: Two key uses of international assessments of achievement have been (a) comparing country performances for identifying the countries with the best education systems and (2) generating insights about effective policy and practice strategies that are associated with higher learning outcomes. Do country rankings really reflect the…

This article reflects on key methodological issues emerging from children and young people's involvement in data analysis processes. We outline a pragmatic framework illustrating different approaches to engaging children, using two case studies of children's experiences of participating in data analysis. The article highlights methods of…

This article explores the statistical methodologies used in demonstration and effectiveness studies when the treatments are applied across multiple settings. The importance of evaluating and how to evaluate these types of studies are discussed. As an alternative to standard methodology, the authors of this article offer an empirical binomial…

A year-long case study of the effects of an educational and vocational rehabilitation program in a youth training center was discussed in terms of fourteen issues which might threaten the validity of an evaluation or research project. This correctional institution housed women aged 14 to 21 who were long-term wards of the state, held on remand, or…

Rather than automatically presuming explicit conditions exist when designing an evaluation to fit the summative or formative mold, evaluators should think of an evaluation design as fitting between endpoints on an evaluation process continuum. Evaluators can blend techniques from both the formative and summative evaluation, matching actual program…

Two innovative methods for collecting and analyzing qualitative data are vignettes and pre-structured cases. Vignettes are descriptions of situations or problems written by a professional, with a suggested outline and comments provided by a researcher. Advantages of this method are strength of impact of the written descriptions and efficiency of…

This case study describes an unsuccessful research proposal submitted in response to a request for applications for projects to implement and evaluate up to 18 innovative programs for homeless persons with alcohol or other drug problems. This unsuccessful proposal reflects several tensions inherent in evaluation research. (SLD)

In 1989 the Department of Education contracted with Development Associates, Inc., to conduct a review of the evaluation and improvement practices of projects funded under Title VII of the Elementary and Secondary Education Act, often known as the Bilingual Education Act. The study focused on local transitional bilingual education and special…

A case study illustrates the ways in which local reactions to a program evaluation directly threaten its validity, and shows that not all participants in a qualitative evaluation study understand or defer to the commitments of democratic evaluation. Ethical concerns in program evaluation are considered. (SLD)

Outlines problems involved in information retrieval evaluation and suggests a more user-centered interpretive approach. A longitudinal study that examines evaluation through the information-seeking behavior of 22 case studies of actual users is described; and relevance judgments, effectiveness ratios, and analysis of document failures are…

This article summarizes the findings of the National Research Council (NRC) report "On Evaluating Curricular Effectiveness" and examines the reviews in middle grades mathematics undertaken by the What Works Clearinghouse (WWC). The NRC report reviewed and assessed 147 key evaluations of 13 National Science Foundation-supported K-12 mathematics…