Bot or not: Can people tell the difference between stories written by a human or by an AI system?

Abstract

How do people perceive artificial intelligence (AI)-generated fictional stories? In Study 1, participants (1,682 adults recruited from Prolific) read either a human-written or AI-generated story. Half were told that their story was written by a human, and half were told that their story was written by ChatGPT. Participants rated the AI-generated story as higher quality and more absorbing, but they rated the story better on both measures when they were told it was written by a human. In Studies 2 and 3, participants (905 adults recruited from Prolific) read both types of story and guessed each story’s origins. Participants were no better than chance at differentiating between the human-written and AI-generated stories. Self-reported expertise with AI, but not with fictional literature, was positively correlated with correct story identification. AI programs thus can generate creative work that is viewed as being at least as good as human-generated work but are not perceived as being able to do so.