This qualitative study utilized episodic narrative interviews to investigate assessment thresholds involved in the development of assessment literacy. The goal of the study was to inform efforts toward quality improvements in higher education. Thirty-five academic staff from universities in Australia, Canada and Sweden shared stories of…

Based on analyses of Maclean's ranking data pertaining to Canadian universities published over the last 24 years, we present a summary of statistical findings of annual ranking exercises, as well as discussion about their current status and the effects upon student welfare. Some illustrative tables are also presented. Using correlational and…

Principals and teachers do not use large-scale assessment results because the lack of distinct and reliable subtests prevents identifying strengths and weaknesses of students and instruction, the results arrive too late to be used, and principals and teachers need assistance to use the results to improve instruction so as to improve student…

This paper addresses the determination of statistically desirable response rates in students' surveys, with emphasis on assessing the effect of underlying variability in the student evaluation of teaching (SET). We discuss factors affecting the determination of adequate response rates and highlight challenges caused by non-response and lack of…

In high-stakes writing assessments, rater training in the use of a rating scale does not eliminate variability in grade attribution. This realisation has been accompanied by research that explores possible sources of rater variability, such as rater background or rating scale type. However, there has been little consideration thus far of…

Background/Context: Two key uses of international assessments of achievement have been (a) comparing country performances for identifying the countries with the best education systems and (2) generating insights about effective policy and practice strategies that are associated with higher learning outcomes. Do country rankings really reflect the…

This report critically examines studies of how scientific research drives innovation which is then translated into socio-economic benefits. It focuses on research evaluation insights that are relevant not only to the academic community, but also to policymakers and evaluation practitioners--and particularly to biomedical and health research…

We present a data-based perspective concerning recent (e.g., 2008) "Maclean's" magazine rankings of Canadian universities, including cluster analysis of the 2008 data. Canadian universities empirically resemble and relate to each other in a manner different from their formal classification and final rank ordering in the…

In attempting to use a realistic evaluation approach to explore the role of Community Parents in early parenting programs in Toronto, a novel technique was developed to analyze the links between contexts (C), mechanisms (M) and outcomes (O) directly from experienced practitioner interviews. Rather than coding the interviews into themes in terms of…

This paper reports on research that suggests a new view of assessment of the practicum in teacher education. By transcending the stereotypes of "failing" student teachers who do not achieve traditional benchmarks, this new lens highlights the complexity, diversity and inequality of experiences through listening to the voices of the…

Career development practitioners understand that the value of career and employment services extends beyond attachment to the labor market. However, "soft" outcomes such as increased self-efficacy and improved ability to manage transitions are difficult to measure. Convincing funders and policy makers of the significance of such outcomes…

Issues related to the establishment of formal standards for student achievement on large-scale assessments are discussed, and two approaches to standard setting that have been used in Saskatchewan (Canada) are described. In spite of the general success of these programs, numerous issues to consider in future evaluations are raised. (SLD)

Discusses Bauman's (1991) evaluation of stability of Wechsler Intelligence Scale for Children (Revised) IQs in children with learning difficulties. Argues that some procedures produced magnified estimates of instability. Presents reanalysis of data that suggests that, although stability coefficients were high, approximately 25 percent of sample…

Describes government funding of university-based research in Australia, Canada, and the United Kingdom. Examines problems in research evaluation and related public policy, particularly the link between evaluation and funding, issues of efficiency and incentive compatibility, and impact of performance evaluation on researcher behavior. Appendix…

Seventeen Canadian teachers and administrators were interviewed to determine their perspectives on reading education. Forty-two interviews were transcribed and evaluated. Preliminary analysis reveals that assessment as a social-psychological phenomenon is filled with tensions, contradictions, and dilemmas that cannot be solved simply through…