The Cambridge University Press study led by Villanova University tested whether adults aged 18–81 could distinguish human-written short fiction from stories produced by AI. Across three experiments — including one with 1,682 participants and two paired-comparison trials with 424 and 481 participants — readers often failed to tell AI from human writing. Pieces labeled as human but generated by AI received the highest ratings, and higher self-reported AI literacy (but not literary expertise) improved detection accuracy.
Readers Rate AI Stories Higher — But Only When Told They're Human, Study Finds
Researchers used three fictional short stories written by published authors, alongside three generated by ChatGPT [EPA/Shutterstock]
New research published by Cambridge University Press and led by Villanova University finds that readers often prefer short stories generated by AI when those stories are presented as human-written. Across three experiments involving adults aged 18 to 81, participants struggled to distinguish AI-generated fiction from work by published authors, and they sometimes rated AI pieces more highly when they believed a human had written them.
Study Design and Findings
The researchers compared three published short stories by human authors with three stories produced by ChatGPT. The project comprised three experiments:
- Experiment 1: 1,682 participants each read one story and were either correctly or incorrectly told whether it was written by a human or by AI; they then rated how engaging and high-quality the story was.
- Experiments 2 and 3: Paired-comparison tests in which 424 and 481 participants respectively read one human-written story and one AI-generated story without being told the source, then attempted to identify which was which.
Overall, participants were generally unable to reliably identify AI-written stories. Notably, stories that were actually written by AI but labeled as human received the highest ratings for engagement and quality.
The study tested to see whether people could tell the difference between human-written and AI-generated short stories [BBC]
Dr Deena Weisberg, senior author and psychologist at Villanova University, commented: "I'd say that public assumptions about AI's capabilities are increasingly out of date."
Additional Results and Interpretation
The study also explored possible predictors of detection skill. Participants who self-reported greater AI literacy were better at recognizing AI-generated text; by contrast, those who reported stronger expertise in literature were not more accurate. Readers additionally rated AI-generated stories as clearer, more direct, and easier to process, while human-written stories were often described as subtler and more complex. The authors suggest readers may prefer predictability and ease of processing over complexity that demands more cognitive effort.
Implications
These findings have implications for publishing, education, and content moderation. If AI text can match or exceed readers' quality ratings when framed as human, standards for disclosure, media literacy, and tools for detecting synthetic text may become increasingly important. The results also challenge assumptions about the uniqueness of human creativity in fiction.
Study Source: Cambridge University Press; Research Led By Villanova University (USA).
Help us improve.
