This article presents the Renger System Test (RST) as a method for assessing whether a system evaluation approach is suitable for evaluating complex interventions. The RST has three criteria: (1) the intervention includes multiple components, (2) these components operate interdependently, and (3) their interdependence produces an outcome that no…

Debates about appropriate, fair and effective ways of assessing research and researchers have raged through the scientific community for decades, recently mostly concerned with discussing the merits and limitations of metric-based, quantitative assessments versus peer review-based, qualitative alternatives. Ample attention has been paid to formal…

Remembering information about others is important but challenging in various social contexts. For instance, in long-term collaborative educational settings, students often need to choose peers for academic support. In different contexts, the selection process can depend on group awareness, i.e., the state of being informed about relevant social or…

Current approaches used in educational research and practice to evaluate the quality of written arguments often rely on structural analysis. In such assessments, credit is awarded for the presence of structural elements of an argument, such as claims, evidence, and rebuttals. In this article, we discuss limitations of such approaches, including…

This paper describes, and lays out an argument for, the use of a procedure to help groups of reviewers to judge the quality of prior research reports. It argues why such a procedure is needed, and how other existing approaches are only relevant to some kinds of research, meaning that a review or synthesis cannot successfully combine quality…

In this article we attempt to reconstruct the tacit and implicit notions of quality in the humanities. This reconstruction is based on a series of semi-structured qualitative interviews with 33 humanities scholars. Applying Max Weber's theory of authority, we argue the quality notions have two different sources--external and internal. External…

Critical appraisal of evidence is performed to assess its validity, trustworthiness and usefulness in evidence-based practice. There currently exists a large number and variety of critical appraisal tools (also named risk of bias tools and quality assessment instruments), which makes it challenging to identify and choose an appropriate tool to…

This paper presents a framework for utilizing Student Evaluations of Teaching (SET) in faculty evaluations. Recognizing the ongoing debate about the validity of SET as a measure of teaching effectiveness, the authors agree with scholars who propose viewing SET as a tool for gauging 'student perceptions of learning'. They present a method that…

Surveys are typical for student evaluation of teaching (SET). Survey research consistently confirms the negative impacts of careless responses on research validity, including low data quality and invalid research inferences. SET literature seldom addresses if careless responses are present and how to improve. To improve evaluation practices and…

Prospective students and their families use college rankings to navigate their higher education options. Rising tuition and fees have made the college decision more fraught. Recently, the major college ranking providers have revised their methodologies to reflect costs and other considerations. These revisions raise important questions about the…

We analysed the full text of all journal articles returned to the education subpanel of the 2021 Research Excellence Framework (REF2021). Using a latent Dirichlet allocation topic model, we identified 35 topics that collectively summarise the journal articles that research units, typically schools of education, selected for submission. We found…

Several AI-aided screening tools have emerged to tackle the ever-expanding body of literature. These tools employ active learning, where algorithms sort abstracts based on human feedback. However, researchers using these tools face a crucial dilemma: When should they stop screening without knowing the proportion of relevant studies? Although…

A key responsibility of higher education providers is the accurate certification of the knowledge and skills attained by their students. However, despite an intense focus on developing relevant quality assurance regulations, academic standards in higher education have remained resistant to explication and consistent application. In this paper, we…

This study examined the perspectives of accreditation liaison officers, institutional accreditor peer-reviewers, and chief library officer perspectives on the role that the academic librarian, academic library, and information literacy should play in institutional accreditation. The study was framed using a conceptual model based on New…

The assessment of workplace learning by educators at the workplace is a complex and inherently social process, as the workplace is a participatory learning environment. We therefore propose seeing assessment as a process of judgment embedded in a community of practice and to this purpose use the philosophy of inferentialism to unravel the judgment…