Automated Essay Scoring (AES) tools aim to improve the efficiency and consistency of essay scoring by using machine learning algorithms. In the existing research work on this topic, most researchers agree that human-automated score agreement remains the benchmark for assessing the accuracy of machine-generated scores. To measure the performance of…

As generative artificial intelligence tools like ChatGPT become increasingly integrated into educational environments, understanding their impact on critical thinking is crucial. Despite growing concerns about AI's potential to diminish students' independent reasoning, there is a lack of research tools specifically designed to evaluate students'…

This study aimed to develop a valid and reliable scale to measure individuals' and organizations' attitudes toward the use of ChatGPT, emphasizing the necessity for organizations to adapt to rapidly evolving information and technology environments. The methodology consisted of three stages. In the first stage, a 13-item draft scale was…

The test view is on the Duolingo English Test (DET), an alternative online English proficiency test with a machine-driven characteristic. The review covers essential information of the DET such as test purpose, usage, score-mapping with CEFR scale, price, and publisher. Meanwhile, the test usefulness is discussed with focuses on reliability,…

Recent advances in machine learning have made automated scoring of learner speech widespread, and yet validation research that provides support for applying automated scoring technology to assessment is still in its infancy. Both the educational measurement and language assessment communities have called for greater transparency in describing…

Since researchers investigated automatic scoring systems in writing assessments, they have dealt with relationships between human and machine scoring, and then have suggested evaluation criteria for inter-rater agreement. The main purpose of my study is to investigate the magnitudes of and relationships among indices for inter-rater agreement used…

Critical thinking, along with other skills, is included as a basic skill in the constructive education program that has been in use in Turkey since 2005. Therefore, a large increase has been observed in studies on critical thinking skills since 2005. In this frame, the present study was conducted in order to systematically examine research papers…

The purpose of this research is to examine the effect of the level of trust of primary school teachers towards their organization in relation to their perceptions of the school having a culture of teacher leadership. Participants of the study consisted of 378 teachers working in Burdur public primary schools. The data collection tool used two…

This paper reports on the development of a scale for determining the quality of the student e-learning experience at the degree level when the student learning context is predominately a campus-based experience. Rapid developments in the use of information and communication technologies (ICT) in higher education require methods for evaluating the…

The INTASC Principles, when used as the basis for developing appropriate measurement instruments to assess teacher dispositions, provide a viable approach to the diagnosis and remediation of skill-related affective performance in teacher candidates and also to meeting NCATE requirements for Standard 1. In this symposium, the development and use of…

In an effective e-learning game, the learner's enjoyment acts as a catalyst to encourage his/her learning initiative. Therefore, the availability of a scale that effectively measures the enjoyment offered by e-learning games assist the game designer to understanding the strength and flaw of the game efficiently from the learner's points of view.…

Three numerical coefficients for analyzing the validity and reliability of ratings are described. Each coefficient is computed as the ratio of an obtained to a maximum sum of differences in ratings. The coefficients are also applicable to the item analysis, agreement analysis, and cluster or factor analysis of rating-scale data. (Author/BW)

The Air Force Occupational Measurement Center conducts task-based occupational surveys of Air Force specialties that include supervisor ratings on recommended training emphasis for entry-level airmen. Priorities are input to the Instructional System Development training model, which guides the development and revision of specialty training…

On-line performance assessment was developed to maximize the usefulness of performance assessment and to minimize the time and labor costs incurred. This paper reports on the development of an on-line performance assessment instrument, focusing on the establishment and validation of the scoring rubric and its implementation in the Rasch model, the…

A method for understanding and controlling the multiple facets of an oral examination (OE) or other judge-intermediated examination is presented and illustrated. This study focused on determining the extent to which the facets model (FM) analysis constructs meaningful variables for each facet of an OE involving protocols, examiners, and…