Those asked to make valid decisions with data don't have the technical knowledge to understand nuance around data quality, assessment aims, and statistical limitations that influence how they should interpret the data. This reality is what Greg Thompson, Leslie Rutkowski, and David Rutkowski call the validity paradox. Educators can surmount this…

Background: Traditional survey efforts to gather outcome data at scale have significant limitations, including cost, time, and respondent burden. This pilot study explored new and innovative large-scale methods of collecting and validating data from publicly available sources. Taking advantage of emerging data science techniques, we leverage…

This article presents the findings of a systematic qualitative analysis of research in the ethics of digital trace data use in learning and education. From the resulting analysis of 77 peer-reviewed studies, we (1) map the characteristics of research by study type, academic community, institutional setting, and national context; (2) identify the…

Participatory modeling (PM) is an engaged research methodology for creating analog or computer-based models of complex systems, such as socio-environmental systems. Used across a range of fields, PM centers stakeholder knowledge and participation to create more internally valid models that can inform policy and increase engagement and trust…

This article examines the concept of credible evidence in Extension evaluations with specific attention to the measures and measurement strategies used to collect and create data. Credibility depends on multiple factors, including data quality and methodological rigor, characteristics of the stakeholder audience, stakeholder beliefs about the…

This study evaluates the influence of ranking reports on university brands' credibility and perceived differentiation. Signaling theory is applied to link ranking with credibility and perceived differentiation. An experimental approach was used to collect data and to test the hypothesis. Data was collected from 328 participants in the UK regarding…

These are the appendixes for the report, "Strengthening the Pennsylvania School Climate Survey to Inform School Decisionmaking." This study analyzed Pennsylvania School Climate Survey data from students and staff in the 2021/22 school year to assess the validity and reliability of the elementary school student version of the survey;…

Evidence-based assessment (EBA) requires that investigators employ scientific theories and research findings to guide decisions about what domains to measure, how and when to measure them, and how to make decisions and interpret results. To implement EBA, investigators need high-quality assessment tools along with evidence-based processes. We…

Reform efforts in statistics education emphasize the need for students to develop statistical thinking. Critical to this goal is a solid understanding of design in the process of collecting data, evaluating evidence, and drawing conclusions. We collected survey responses from over 700 college students at the start of an introductory statistics…

Teacher data-driven decision making (DDDM) is a professional practice of great prominence in the current K-12 education system. Moreover, teacher self-efficacy and anxiety around DDDM represent important measurement targets in both research and practice. This study consequently examined the validity, reliability, and invariance of data collected…

Evidence-based algorithms can improve both lay and professional judgements and decisions, yet they remain underutilised. Research on advice taking established that humans tend to discount advice--especially when it contradicts their own judgement ("egocentric advice discounting")--but this can be mitigated by knowledge about the…

Purpose: The purpose of this study was to ascertain the relationship between EFL teachers' perception of the intended and unintended consequences of formative assessment (FA) decisions and their sense of self-efficacy and anxiety toward data-driven decision-making (DDDM). Design/methodology/approach: A correlational research design and…

In this study, teacher education students' concerns, sense of efficacy, and anxiety related to the future use of data to drive educational decision-making were explored. In alignment with prior research with practicing teachers, this sample of pre-service teachers reported concerns (thoughts, preoccupations, and feelings) that indicate they are…

This publication is the fourth in a series of practical Cedefop skills anticipation guides for policymakers, analysts, and expert professionals. The three previous guides presented a rich mosaic of conventional and emerging methods for identifying technological change and its impact on skills. They assessed the merits and challenges of using…

Purpose: This study aims to analyse and evaluate the methodology followed by the Times Higher Education Impact Rankings (THE-IR), as well as the coverage obtained and the data offered by this ranking, to determine if its methodology reflects the degree of sustainability of universities, and whether their results are accurate enough to be used as a…