ERIC Number: EJ1461993
Record Type: Journal
Publication Date: 2025-Feb
Pages: 20
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0022-0175
EISSN: EISSN-2162-6057
Available Date: 2024-06-17
Automated Scoring of Figural Tests of Creativity with Computer Vision
Selcuk Acar1; Peter Organisciak2; Denis Dumas3
Journal of Creative Behavior, v59 n1 e677 2025
In this three-study investigation, we applied various approaches to score drawings created in response to both Form A and Form B of the Torrance Tests of Creative Thinking-Figural (broadly TTCT-F) as well as the Multi-Trial Creative Ideation task (MTCI). We focused on TTCT-F in Study 1, and utilizing a random forest classifier, we achieved 79% and 81% accuracy for drawings only (r = 0.57; 0.54), 80% and 85% for drawings and titles (r = 0.59; 0.65), and 78% and 85% for titles alone (r = 0.54; 0.65), across Form A and Form B, respectively. We trained a combined model for both TTCT-F forms concurrently with fine-tuned vision transformer models (i.e., BEiT) observing accuracy on images of 83% (r = 0.64). Study 2 extended these analyses to 11,075 drawings produced for MTCI. With the feature-based regressors, we found a Pearson correlation with human labels (rs = 0.80, 78, and 0.76 for AdaBoost, and XGBoost, respectively). Finally, the vision transformer method demonstrated a correlation of r = 0.85. In Study 3, we re-analyzed the TTCT-F and MTCI data with unsupervised learning methods, which worked better for MTCI than TTCT-F but still underperformed compared to supervised learning methods. Findings are discussed in terms of research and practical implications featuring Ocsai-D, a new in-browser scoring interface.
Descriptors: Scoring, Computer Assisted Testing, Models, Correlation, Creativity Tests, Creative Thinking, Task Analysis, Accuracy, Test Format, Computer Software
Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www-wiley-com.bibliotheek.ehb.be/en-us
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Assessments and Surveys: Torrance Tests of Creative Thinking
Grant or Contract Numbers: N/A
Data File: URL: https://osf.io/q82ax/?view_only=69bff72648c64c6c992420b971e716ad
Author Affiliations: 1University of North Texas; 2University of Denver; 3The University of Georgia