In the current article, we consider the influential position of the Programme for International Student Assessment (PISA) and discuss several methodological areas that demonstrate the need for caution when using and interpreting PISA results. We motivate our argument by briefly describing the program's increased influence in educational policy…

This study critically investigates the suitability of United Nations' composite indices and other related measures--among which the Gender Inequality Index just released in 2010--to capture gender inequalities in the context of "highly developed" countries, focusing on the case of Europe. Our results indicate that many of the gender gaps…

Current economic conditions and the increasing competition for government funding are leading to an increased focus on the impact of research. Measuring the impact of research is difficult because not all impacts are direct and some can be negative or result from the identification of problems that require a non-research response. The time between…

Although the Delphi is widely used, research on certain methodological issues is somewhat limited. After a brief introduction to the strengths, limitations, and methodological challenges of the technique, we share our experiences (as well as problems encountered) with an electronic Delphi of educational program evaluation (EPE) in the Asia-Pacific…

Rapidly declining response rates and the associated threat of nonresponse bias call into question the validity of data obtained through telephone surveys, a tool often used in evaluation. This article explores changes in nonresponse bias over time by examining three data points (1991, 1996, and 2002) from an annual household telephone survey…

This report critically examines studies of how scientific research drives innovation which is then translated into socio-economic benefits. It focuses on research evaluation insights that are relevant not only to the academic community, but also to policymakers and evaluation practitioners--and particularly to biomedical and health research…

A year-long case study of the effects of an educational and vocational rehabilitation program in a youth training center was discussed in terms of fourteen issues which might threaten the validity of an evaluation or research project. This correctional institution housed women aged 14 to 21 who were long-term wards of the state, held on remand, or…

Global economic transformations have never been as dramatic as in the twentieth century. Most countries have experienced radical changes in the standards of income per capita, technology, fertility, mortality, income inequality and the extent of democracy in the course of the past century. It is the goal of many disciplines--economics, history,…

Considers the competing demands placed on evaluators in relation to the service (characteristics of the intervention), the sample (people participating), and methodology or research design. Explores compromises that may be required between scientific ideals and real-world limitations, and assesses implications for obtaining meaningful results in…

Describes government funding of university-based research in Australia, Canada, and the United Kingdom. Examines problems in research evaluation and related public policy, particularly the link between evaluation and funding, issues of efficiency and incentive compatibility, and impact of performance evaluation on researcher behavior. Appendix…

This article emerges from a recent review of evidence, conducted by the authors and others, on the lifelong barriers to widening participation in higher education in England. This has led us to a consideration of the quality and relevance of the research activity in this large field of endeavour, and to the creation of a typology of the kinds of…