Can ChatGPT write a better novel than I can?

By Stephen L Carter

I’m no enemy of artificial intelligence, and no stranger to the notion of combined human-computer authorship. I’ve written about the goofy appeal of movies scripted by neural nets. For a class project in college, I submitted a computer program that generated outlines for “Star Trek” episodes. But as a working novelist, I’m naturally concerned at the prospect that ChatGPT and its cousins might displace human authors. That’s been the smart talk lately, as large language models herald a new era of AI.

The novel’s demise has been predicted often, but after a series of chats with ChatGPT, I think this time the voices of gloom might have a point.

Well, half a point.

Novels matter. Reading serious literature increases empathy and an appreciation of human complexity. That’s why I’ve long argued that novels are crucial to making democracy work.

So how good is ChatGPT at fiction? I tried dozens of tests, from asking the bot to imitate the voice of a known writer to inviting it to create on its own.

Read | ChatGPT takes on the tough US medical licensing exam

The results were mixed.

The bot was dreadful at reproducing the voices of a great novelists of earlier eras and today’s big sellers. For instance, its version of Stephen King began like a bad book jacket: “One day, strange things began to happen in Millfield. People started to disappear, and strange whispers echoed through the streets at night.”

Fine. ChatGPT can’t (yet) keep up with the bigs. Neither can the rest of us. But when we allow the bot to flex its own imaginative muscles, things start to get interesting.

For example, when I asked the software to write scary stories, the results astonished me. ChatGPT has clearly learned a key page-turning formula or two. Here’s one opening paragraph:

< Sarah had always been a runner. She had been on the track team in high school and college, and even now, in her late twenties, she still enjoyed going for a run around the neighborhood. But today, Sarah was running for her life.

Not bad! Though the prose won’t win prizes, I defy any editor or agent to ignore a query that begins that way.

Read | Google engineer develops Bhagavad Gita-based 'GitaGPT' to answer life's queries

But I suppose the plot-driven story is exactly what we’d expect an LLM to be good at. The bot is trained on existing texts to predict which string would probably follow which string. Gertrude Stein famously wrote that in the true novel we don’t read to find out what happens next. But that’s exactly what most readers do, and kindling that desire is what makes contemporary fiction go. ChatGPT, though rough around the edges, is starting to understand how it’s done.

I’m not saying the bot is ready to produce a decent novel. It gets the elements of fiction but isn’t sure how to arrange them. Its endings are uniformly weak. But the near-term goal of AI researchers isn’t authorship; it’s transforming fiction into a collaborative enterprise between human and machine.

In November, researchers at Google reported on experiments with Wordcraft, a bot designed to assist creative writing. The participants, all published authors of poetry or fiction, could at moments of their choosing ask Wordcraft for advice or proposed text. Though the advice was often helpful, the participants reported problems, among them a difficulty in getting the bot to maintain a distinctive voice.

Perhaps, given sufficient time and training, the LLMs will figure that one out. Certainly Microsoft thinks so. The company’s decision to invest $10 billion in OpenAI, the startup that created ChatGPT, signals a belief that as the bot learns, the collaborative future will arrive. Under the deal, the bot will be integrated not only into Bing but into Office. A writer who’s feeling blocked will be able to ask the program to continue the story.

To test ChatGPT’s current capacity to assist a novelist, I tried the following prompt:

> Finish this paragraph: When I looked out the window I was terrified. They had found me after all. There was nowhere left to hide.

Here’s the response:

< I saw a group of dark figures approaching my house, their faces obscured by hoods and masks. I knew they were here to take me away and there was nothing I could do to stop them. I felt my heart pounding in my chest as I realized that my fate was sealed. The end was near.

Impressive. Again, the response isn’t exactly deathless prose, but neither was the prompt. I’d certainly be inclined to read on.

With more literary elements, however, the program (so far) remains weak. I asked for a description of a “beautiful sunset” and was treated to a long, convoluted paragraph that included this passage — “a breathtaking spectacle in which the sky is painted with a vibrant array of colors” — a phrase that reads like a middle-schooler who’s trying too hard. Moreover, in my test runs, ChatGPT generated countless pounding hearts and moths drawn to flame and other cliches aspiring writers are warned to avoid.

Which is not to say that ChatGPT and its competitors won’t get better. Already, the bot understands literature well enough to write an essay that passes the AP English exam. If it can analyze novels, there’s no reason to think it can’t learn to write them.