AI-generated fiction is flooding literary magazines — but not fooling anyone

A short story titled “The Last Hope” first hit Sheila Williams’ desk in early January. Williams, the editor of Asimov’s Science Fiction magazine, reviewed the story and passed on it.

At first, she didn’t think much of it; she reads and responds to writers daily as part of her job, receiving anywhere from 700 to 750 stories a month. But when another story, also titled “The Last Hope,” came in a couple weeks later by a writer with a different name, Williams became suspicious. By the time yet another “The Last Hope” came a few days later, Williams knew immediately she had a problem on her hands.

“That’s like the tip of the iceberg,” Williams says.

Since that first submission, Williams has received more than 20 short stories all titled “The Last Hope,” each coming from different author names and email addresses. Williams believes they were all generated using artificial intelligence tools, along with hundreds of other similar submissions that have been overwhelming small publishers in recent months.

Asimov’s received around 900 stories for consideration in January and is on track to get 1,000 this month. Williams says nearly all of the increase can be attributed to pieces that appear to be AI-generated, and she’s read so many that she can now often tell from the first few words whether something might not be written by a human.

Besides repeating titles, there are certain character names that tend to appear often, Williams says. Sometimes the manuscript will contain a different title than the one indicated in the online form. Author names often appear to be amalgamations of first and last names. In optional cover letters, some authors include instructions on how to wire them money for their story that has not yet been accepted. At times, the submitter hasn’t even bothered to replace “[name]” with their own.

Using ChatGPT, The Verge was able to replicate some elements of submissions Williams has seen. A prompt to write a short science fiction story — plus copy-and-pasted information from Asimov’s submission guidelines — produced stories with dozens of similar titles in succession, like “The Last Echo,” “The Last Message,” “The Last Day of Autumn,” and “The Last Voyager.” 

Willams and her team have learned to spot AI-generated works, but the influx of submissions has been frustrating all the same. Outlets like Asimov’s are getting overwhelmed by AI chum, taking up the time of editors and readers and potentially crowding out genuine submissions from newer writers. And the problem could only get worse, as the wider availability of writing bots creates a new genre of get-rich-quick schemes, where literary magazines with open submissions have discovered themselves on the receiving end of a new surface for spammy submissions trying to game the system.

“I just basically go through them as quickly as I can,” Williams says of the pieces she suspects are AI-generated. “It takes the same amount of time to download a submission, open it, and look at it. And I’d rather be spending that time on the legitimate submissions.”

For some editors, the influx of AI-generated submissions has forced them to stop accepting new work.

Last week, the popular science fiction magazine Clarkesworld announced it would temporarily close submissions due to a flood of AI-generated work. In an earlier blog post, editor Neil Clarke had noted that the magazine was forced to ban a skyrocketing number of authors because they had submitted stories that were generated using automated tools. In February alone, Clarkesworld had received 700 submissions written by humans and 500 machine-generated stories, Clarke says.

Clarke believes the spammy submissions are coming from people looking to make a quick buck and who found Clarkesworld and other publications through “side hustle” influencers and websites. One website, for example, is loaded with SEO bait articles and keywords around marketing, writing, and business and promises to help readers make money quickly. An article on the site lists nearly two dozen literary magazines and websites — including Clarkesworld and Asimov’s, as well as larger outlets like the BBC — with pay rate and submission details. The article encourages readers to use AI tools to help them and includes affiliate marketing links to Jasper, an AI writing software. 

Most of the publications pay small per-word rates, around 8 to 10 cents, while others pay flat fees of up to a few hundred dollars for accepted pieces. In his blog, Clarke wrote that a “high percentage of fraudulent submissions” were coming from some regions but declined to name them, concerned that it could paint writers from those countries as scammy. 

But the possibility of being paid is a factor: in some cases, Clarke has corresponded with people who’ve been banned for submitting AI-generated work, saying they need the money. Another editor told The Verge that even before the AI-generated stories, they’d get submissions and emails from writers in countries where the cost of living is lower and an $80 publication fee goes much farther than it does in the US.

Clarke, who built the submission system his magazine uses, described the AI story spammers’ efforts as “inelegant” — by comparing notes with other editors, Clarke was able to see that the same work was being submitted from the same IP address to multiple publications just minutes apart, often in the order that magazines appear on the lists.

“If this were people from inside the [science fiction and fantasy] community, they would know it wouldn’t work. It would be immediately obvious to them that they couldn’t do this and expect it to work,” Clarke says.

The issue extends beyond science fiction and fantasy publications. Flash Fiction Online accepts a range of genres, including horror and literary fiction. On February 14th, the outlet appended a notice to its submission form: “We are committed to publishing stories written and edited by humans. We reserve the right to reject any submission that we suspect to be primarily generated or created by language modeling software, ChatGPT, chat bots, or any other AI apps, bots, or software.”

The updated terms were added around the time that FFO received more than 30 submissions from one source within a few days, says Anna Yeatts, publisher and co-editor-in-chief. Each story hit cliches Yeatts had seen in AI-generated work, and each had a unique cover letter, structured and written unlike what the publication normally sees. But Yeatts and colleagues had had suspicions since January that some work they had been sent had been created using AI tools.

Yeatts had played around with ChatGPT beginning in December, feeding the tool prompts to produce stories of specific genres or in styles like gothic romance. The system was able to replicate the technical elements, including establishing main characters and setting and introducing conflict, but failed to produce any “deep point of view” — endings were too neat and perfect, and emotions often spilled into melodrama. Everyone has “piercing green eyes,” and stories often open with characters sitting down. Of the more than 1,000 works FFO has received this year, Yeatts estimates that around 5 percent were likely AI-generated.

“We put that scary little warning up [on the submissions page],” Yeatts says. Enforcing it, though, could prove to be challenging.

In the past, FFO has published mainstream work that has a more conventional writing style and voice that is accessible to a range of reading levels. For that, Yeatts says stories generated using AI tools could get past baseline requirements. 

“It does have all the parts of the story that you try to look for. It has a beginning, middle, and end. It has a resolution, characters. The grammar is good,” Yeatts says. The FFO team is working to train staff readers to look for certain story elements as they’re taking a first pass at submissions.

Yeatts is concerned that a growing wave of AI-generated work could literally shut out written work. The outlet uses Submittable, a popular submission service, and FFO’s plan that includes a monthly cap on stories, after which the portal closes. If hundreds of people send ineligible AI-generated work, that could prevent human authors from sending in their stories.

Yeatts isn’t sure what the magazine can do to stop the stories from coming. Upgrading the Submittable plan would be costly for FFO, which runs “on a shoestring budget,” Yeatts says. 

“We’ve talked about soliciting stories from other authors, but then that also doesn’t really feel true to who we are as a publication because that’s going to deter new writers,” Yeatts says. “We really don’t have good solutions.”

Others in the community are keeping an eye on the problem that’s inundating other publishers and are thinking through ways to respond before it spreads further. Matthew Kressel, a science fiction writer and creator of Moksha, an online submission system used by dozens of publications, says he’s started hearing from clients who have received spammy submissions that appear to be written using AI tools.

Kressel says he wants to keep Moksha “agnostic” when it comes to the value of submissions generated using chatbots. Publishers have the ability to add a checkbox where writers can confirm that their work doesn’t use AI systems, Kressel says, and is considering adding an option for publications that would allow them to block or partially limit submissions using AI tools.

“Allowing authors to self-affirm if the work is AI-generated is a good first step,” Kressel told The Verge via email. “It provides more transparency to the whole thing, because right now there’s a lot of uncertainties.”

For Williams, the editor of Asimov’s, being forced to use her time to sift through the AI-generated junk pile is frustrating. But even more concerning is that legitimate new authors might see what’s happening and think editors won’t ever make it to their manuscript.

“I don’t want writers to be worried that I’m going to miss their work because I’m inundated with junk,” Williams says. The good stories are obvious very early on. “The mind that crafts the interesting story is not in any danger.”