Evaluating the performance of Large Language Models (LLMs) is a critical yet challenging task, particularly when aiming to avoid subjective assessments. This paper proposes a framework for leveraging subjective metrics derived from the class textual materials across different semesters to assess LLM outputs across various tasks. By utilizing…

The synthetic control method (SCM) is a popular approach for estimating the impact of a treatment on a single unit with panel data. Two challenges arise with higher frequency data (e.g., monthly versus yearly): (1) achieving excellent pre-treatment fit is typically more challenging; and (2) overfitting to noise is more likely. Aggregating data…

Evaluators widely agree that the involvement of key actors is a central aspect of quality practice. However, a scoping review of to what extent and how youth are included has not been explored. This is an important omission because many evaluations are done on programs that directly serve youth. We examined 159 evaluation reports in a publicly…

The exponential growth of scholarly publications in recent years has presented a daunting challenge for researchers to keep track of relevant articles within their research field. To address this issue, we examined the capabilities of InfraNodus, an AI-Powered text network analysis platform. InfraNodus promises to provide insights into any…

According to architectural designers, although they are aware of the sound environment when designing spaces, in many cases visual (design) and cost (cost-effectiveness, etc.) were the predominant factors. In many cases, the visual (design) and cost aspects (cost-effectiveness, etc.) were dominant. The tacit rule for evaluating the sound…

Automatically assessing classroom discussion quality is becoming increasingly feasible with the help of new NLP advancements such as large language models (LLMs). In this work, we examine how the assessment performance of 2 LLMs interacts with 3 factors that may affect performance: task formulation, context length, and few-shot examples. We also…

This study aims to examine the studies on curriculum commitment in Turkey. The descriptive content analysis method was used in the research. Google Scholar and YÖK national databases were searched using the keywords "curriculum" and "fidelity." Twenty-four studies using these words in their titles were included in the research.…

Knowledge Tracing (KT) is a task to model students' knowledge based on their coursework interactions within an Intelligent Tutoring System (ITS). Recently, Deep Neural Networks (DNN) showed superb performance over classical methods on multiple dataset benchmarks. While most Deep Learning based Knowledge Tracing (DLKT) models are optimized for…

This paper reviews 38 studies conducted between 2015 and 2022 on collaborative assessments in open-distance and e-learning (ODeL) contexts, focusing on the benefits, types, challenges, and strategies to improve collaborative assessments. This qualitative review aims to investigate collaborative assessments within the ODeL comprehensively. The…

The purpose of this systematic review was to survey available measures of creativity for pk12 students for assessments characteristics and reporting of psychometric properties. Using the PRISMA framework, we identified 42 unique articles with 48 assessments meeting our inclusion criteria. Then, two coders independently coded all articles using a…

Gifted students are future leaders in society. In order to support and encourage them, it's important to have a thorough understanding of the nature of giftedness and the perceptions, and anticipations of gifted students. Therefore, the purpose of this study is to investigate the perceptions and anticipations of 6th-grade gifted students about…

Classroom discourse is an essential vehicle through which teaching and learning take place. Assessing different characteristics of discursive practices and linking them to student learning achievement enhances the understanding of teaching quality. Traditional assessments rely on manual coding of classroom observation protocols, which is…

The roles in traditional mentoring dyads are well known across both academic and professional contexts (Dawson, 2014). Despite the universality of these relationships, the way mentorship is evaluated in these relationships is fractured. Evaluation is limited to singular voices, singular points in time and simplified metrics to capture the journey…

Idiomatic expressions (IEs) play an essential role in natural language. In this paper, we study the task of idiomatic sentence paraphrasing (ISP), which aims to paraphrase a sentence with an IE by replacing the IE with its literal paraphrase. The lack of large scale corpora with idiomatic-literal parallel sentences is a primary challenge for this…

Automated essay scoring (AES), where natural language processing is applied to score written text, can underpin educational resources in blended and distance learning. AES performance has typically been reported in terms of correlation coefficients or agreement statistics calculated between a system and an expert human examiner. We describe the…