This study addressed the cognitive impacts of providing correct and incorrect machine learning (ML) outputs in support of an object detection task. The study consisted of five experiments that manipulated the accuracy and importance of mock ML outputs. In each of the experiments, participants were given the T and L task with T-shaped targets and…

This study explores the novice programmers' intention to use chat generative pretrained transformer (ChatGPT) for programming tasks with emphasis on performance expectancy (PE), risk-reward appraisal (RRA), and decision-making (DM). Utilizing partial least squares structural equation modeling (PLS-SEM) and a sample of 413 novice programmers, the…

In two experiments, we examine how features of an imperfect automated decision aid influence compliance with the aid in a simplified, simulated nautical collision avoidance task. Experiment 1 examined the impact of providing transparency in the pre-task instructions regarding which attributes of the task that the aid uses to provide its…

Much has been made of the apparent capacity for creativity of generative AI. However, as research expands the knowledge base regarding the capabilities and performance of this technology, the prevailing view is shifting away from "AI is creative" and towards a more balanced model of Human-AI co-creativity. Nevertheless, even this…

The aim of the research is to investigate the academic achievement of ChatGPT, an artificial intelligence based chatbot, in a national mathematics exam. For this purpose, 3.5 and 4 versions of ChatGPT were asked mathematics questions in a national exam. The method of the research is a case study. In the research, 3.5 and 4 versions of ChatGPT were…

This research aimed to develop and validate a rubric to assess Artificial Intelligence (AI) chatbots' effectiveness in accomplishing tasks, particularly within educational contexts. Given the rapidly growing integration of AI in various sectors, including education, a systematic and robust tool for evaluating AI chatbot performance is essential.…

Generative artificial intelligence brings opportunities and unique challenges to nontraditional higher education students, stemming, in part, from the experience of the digital divide. Providing access and practice is critical to bridge this divide and equip students with needed digital competencies. This mixed-methods study investigated how…

Predicting and analyzing multimodal dialog user experience (UX) metrics, such as overall call experience, caller engagement, and latency, among other metrics, in an ongoing manner is important for evaluating such systems. We investigate automated prediction of multiple such metrics collected from crowdsourced interactions with an open-source,…

The feasibility and limitations of speaker adaptation in improving the performance of a "fixed" (speaker-independent) automatic speech recognition system were examined. A fixed vocabulary of 55 syllables is used in the recognition system which contains 11 stops and fricatives and five tense vowels. The results of an experiment on speaker…

The work of a three-year series of experimental studies of human cognition is summarized in this report. Proglem solving and learning in man-machine interaction was investigated, as well as relevant variables and processes. The work included four separate projects: (1) computer-aided problem solving, (2) computer-aided instruction techniques, (3)…

This study, part of a larger effort to develop a methodology for evaluating intelligent computer systems (Artificial Intelligence Systems), explores the use of benchmarking as an evaluation technique. Benchmarking means comparing the performance of intelligent computer systems with human performance on the same task. Benchmarking in evaluation has…