Human writers still outperform ChatGPT in creativity, study finds

Humans outperform AI in creative writing, for now: A recent study by UC Berkeley researcher Nina Beguš reveals that human-written stories are more creative and nuanced than those generated by AI, though AI’s capabilities are rapidly advancing.

• Beguš, a lecturer in UC Berkeley’s School of Information and Department of History, conducted a comparative study between hundreds of human participants and AI platforms like ChatGPT and Llama.
• The study, published in Humanities and Social Sciences Communications, offers insights into the current limitations of generative AI tools in creative writing.

Study methodology and key findings: Beguš used a common storytelling structure based on the Pygmalion myth to compare human and AI-generated responses, revealing significant differences in narrative quality and complexity.

• Participants were given prompts to write a story about a human creating an artificial human and falling in love with it.
• The study analyzed 250 human-written responses and 80 AI-generated stories, evaluating aspects such as gender, sexuality, race, ethnicity, culture, and overall narrative complexity.
• Human-written stories consistently demonstrated richer and more varied narratives, while AI-generated stories were often formulaic, lacked tension, and were filled with clichés.

AI’s evolving capabilities: Despite overall human superiority in creative writing, the study uncovered interesting developments in AI-generated content, particularly in terms of inclusivity and representation.

• Newer AI models, such as ChatGPT 4, produced more inclusive writing, with 25% of AI-generated stories featuring same-sex love interests, compared to only 7% of human-written stories.
• This trend reflects the integration of more progressive human values in recent AI training data and demonstrates how AI can be influenced by technologists’ choices during the value alignment process.

The emergence of “artificial humanities”: Beguš’s work is part of a new field she calls “artificial humanities,” which aims to incorporate insights from history, literature, and other humanities subjects into AI development.

• This interdisciplinary approach seeks to leverage centuries of expertise in language, writing, and human nature to inform and enhance AI capabilities.
• Beguš argues that scholars from the humanities should play a crucial role in developing future AI tools, given their deep understanding of language and human expression.

Implications for education and writing: The study highlights the growing influence of AI on writing and education, prompting discussions about the future of human creativity and expression.

• Universities are increasingly providing access to ChatGPT and teaching students how to use it effectively in their work.
• Some professors are incorporating Beguš’s research on AI and the humanities into their course syllabi, recognizing the importance of understanding AI’s role in writing and creativity.

The future of human-AI collaboration in writing: While AI currently falls short in creative writing, its rapid advancement raises questions about the future relationship between human writers and AI tools.

• Beguš speculates about how future generations might view traditional writing methods, noting that writing has been an essential human activity taught from an early age.
• The researcher emphasizes the need for quality writers to create compelling stories and expresses curiosity about potential insights writers might gain from AI in the future.

Broader implications: As AI continues to evolve, the study underscores the importance of interdisciplinary collaboration between humanities scholars and AI developers to create more sophisticated and nuanced AI writing tools.

• The research highlights the potential for AI to complement human creativity rather than replace it, suggesting a future where AI and human writers may work in tandem to produce high-quality content.
• Beguš’s work opens up new avenues for exploring the intersection of technology and human expression, potentially leading to more advanced and culturally aware AI systems in the future.