This study compares how the generative AI (GenAI) large language model (LLM) ChatGPT performs in grading university exams compared to human teachers. Aspects investigated include consistency, large discrepancies and length of answer. Implications for higher education, including the role of teachers and ethics, are also discussed. Three…

Background: Individuals with dementia have communication limitations resulting from cognitive impairments that define the syndrome. Whereas there are numerous cognitive assessments for individuals with dementia, there are far fewer communication assessments. The Profiling Communication Ability in Dementia (P-CAD) was developed to address this gap.…

Student evaluations of teaching (SET) act as the primary means to gauge instructor effectiveness. Likewise, SETs provide the primary qualitative feedback to instructors via student comments. However, mostly students with strong feelings tend to write comments. Among the most recallable are toxic comments: comments that are unhelpful/hurtful in…

This study investigates the reliability and consistency of a custom GPT-based scoring system in comparison to trained human raters, focusing on B1-level opinion paragraphs written by English preparatory students. Addressing the limited evidence on how AI scoring systems align with human evaluations in foreign language contexts, the study provides…

Open access to novel AI tools offers unprecedented opportunities for human-AI collaboration in writing instruction and assessment. While research on using generative AI tools like ChatGPT in these contexts is emerging, more is needed to understand their effectiveness as Automated Writing Evaluation (AWE) tools. This study explores the potential of…

The "consensual assessment technique" (CAT) is a reliable and valid method to measure (product) creativity and often considered "the" gold standard of creativity assessment. The reliability measure traditionally applied in CAT studies--inter-rater reliability--cannot capture time-sampling error, which is a particular relevant…

Purpose: This study compared the interrater reliability of adult naïve listeners' perceptual assessments of different speech variables in children with a cleft palate with or without a cleft lip (CP ± L). In addition, the study investigated whether the listeners were able to perceive differences in these speech variables before and after speech…

As there is a paucity of instrument investigating a hybrid teaching conception, the current study is seen as part of attempt to fill this gap. The subjects in the study were 310 University participants--instructors in Socialist Republic of Viet Nam (Vietnam). The survey was implemented with the use of Cognitive Constructivism-oriented Teaching…

Purpose: The current standard for clinical diagnosis of childhood apraxia of speech (CAS) is expert clinician judgment. The psychometric properties of this standard are not well understood; however, they are important for improving clinical diagnosis. The purpose of this study is to determine the extent to which experts agree on the clinical…

Single-case design is a research methodology that entails repeated measurement to assess the influence of an independent variable on a dependent variable over time. Data collected in this manner are regularly analyzed using visual analysis of data displayed in a linear graph. Although there is agreement regarding critical elements of visual…

There is a vast body of literature documenting the positive impacts that rater training and calibration sessions have on inter-rater reliability as research indicates several factors including frequency and timing play crucial roles towards ensuring inter-rater reliability. Additionally, increasing amounts research indicate possible links in…

The typical process for assessing inter-rater reliability is facilitated by training raters within a research team. Lacking is an understanding if inter-rater reliability scores "between" research teams demonstrate adequate reliability. This study examined inter-rater reliability between 16 researchers who assessed fundamental motor…

Despite the widespread use of school-based universal screening systems for social, emotional, and behavioral risk, limited research has examined discrepancies in ratings provided by teachers and their secondary students. Using the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS; teacher report) and mySAEBRS (student report) scores…

Purpose: Assessing social communication and measuring its changes among young autistic children presents significant challenges, particularly when tracking intervention effects within short timeframes. Existing measures, mostly validated in Western contexts, may not be suitable for culturally diverse populations. Addressing this gap, the Social…

Background: The need for the adaptation of instruments to other native languages to promote culture-fairness framed this study. Aim: This article reports on the adaptation of the locally developed Emotional Social Screening Tool for School Readiness (E3SR-R) into isiXhosa. Setting: This adaptation study was conducted in South Africa. Methods: The…