A recent study published in the journal Scientific Reports suggests that Large Language Model (LLM) AI chatbots may outperform ordinary humans in creative tasks, such as brainstorming alternative uses for common items – which reflects divergent thinking. However, the individuals who scored highest on these tasks still outperformed the results of the best-performing chatbots.
Divergent thinking is a thought process often associated with creativity, emphasizing the generation of many different ideas or solutions for a specific task.
It is typically assessed through alternative use tasks (AUT), in which participants are asked to come up with as many alternative uses for everyday objects as possible in a short period of time. The responses were divided into four different categories: fluency, flexibility, originality, and exhaustiveness.
Mika Koivisto and Simone Grassini compared the responses of 256 human participants with those of three AI chatbots (ChatGPT3, ChatGPT4, and Copy.Ai) and reacted to the AUTs of four objects (a rope, a box, a pencil, and a candle). Authors assess the originality of responses by rating semantic distance (how relevant the response is to the original purpose of the object) and creativity.
Semantic distance is quantified on a scale of 0 to 2 using computational methods, while human evaluators are blind to the initiator of the response, subjectively rating creativity on a scale of 1 to 5. On average, chatbots generated responses that scored significantly higher than human responses to semantic distance (0.95 vs. 0.91) and creativity (2.91 vs. 2.47).
Human responses have a larger range on both metrics — the lowest score is much lower than the AI response, but the highest score is usually higher. In seven of the eight scoring categories, the best response from humans outperformed the best response from each chatbot.
These findings suggest that AI chatbots can now generate ideas at least like ordinary people. However, the authors note that they only considered the performance of individual tasks related to the assessment of creativity. The authors propose that future research could explore how to integrate AI into the creative process to improve human performance.