Quick Links

ChatGPT has been hitting the headlines for its ability to code, search, and write "just as well as a human." But is that the case? And could you use OpenAI's chatbot to produce something a lot of professional writers never will --- a novel?

Well, no, you actually can't. You may think you can, and anxiety about AI taking everyone's jobs, coupled with stories about ChatGPT somehow becoming the "most prolific writer on Amazon," may reinforce that theory. But a novel is a massive, complex, undertaking that is far beyond ChatGPT's current capabilities.

To test the concept out, I forked out some cash for the premium version of ChatGPT and tried to become a published author in a weekend. That didn't happen, but I did get a pretty solid understanding of ChatGPT's current weaknesses. Here's why the writers amongst you shouldn't start to sweat just yet.

Children's Books Don't Count

A pile of childrens books generated by Dall-E 2
Review Geek

Reuters recently published a piece about ChatGPT helping an amateur author produce a book in a few hours. That AI-written book was stuck together with some AI-generated art and self-published on Amazon, where it has earned its creator around $100.

OpenAI says you are free to use whatever ChatGPT generates for you in any way you wish, and Amazon isn't too bothered as long as you're not actually stealing anything. So the whole endeavor is also above board. So what's the issue?

Brett Schickler, the "author" featured in the story, used the bot to produce a 30-page children's book. And that's probably as good as it gets with the current version of ChatGPT. Actual novels are far longer and more complex, and that's where ChatGPT really struggles. There are a number of limitations that mean AI novels will probably be less common than you'd think -- and the ones that make it through won't actually be much good.

The Longer Things Get, the More Confused ChatGPT Gets

A confused robot looking at a sheet of text generated by Dall-E 2
Review Geek

The 30-page length of the children's book definitely made things easier for both Schickler and ChatGPT. Children's books don't have that many words on the page either, which again helps get around one of the bot's biggest limitations.

A regular novel, according to Writer's Digest, should be between 90,000 and 100,000 words. A picture-heavy children's book, on the other hand, is between 500 and 1000 words. You could split this article into about three of them.

In my experience, ChatGPT couldn't even get the layout of a 20-chapter novel together without developing severe issues. It could give a brief synopsis for each chapter, but fell apart when asked to expand on them. "Expanding" the chapters would result in the plot of each chapter shifting around, even when the AI was asked to elaborate on something it had just posted. It also shortened the book considerably, to around 12 chapters, when the detail had expanded past a certain point. The mistake was then repeated when I pointed out the error and asked the bot to generate the summaries again.

If you break things down and go five chapters at a time or so, you can get all 20 laid out. But the more detail you go into, the more the AI struggles. It's like a literary version of blowing up a low-resolution photo, you can only stretch it so far before it becomes a pixelated lump. The AI will forget a character's motivation, name, elements of the plot, friends, enemies, and so on as the story goes on. There is little to no continuity, even at the synopsis stage.

ChatGPT is a Very Limited Novelist

Dall-E 2's interpretation of a very cliché novelist
Review Geek

While continuity issues are enough to kill the idea of using ChatGPT for a full novel, the problems go deeper than that. ChatGPT is a very limited novelist.

If you want extensive dialogue, you should ask for the response in script format. That makes it dialogue heavy, even if you will need to tweak it after.). Most authors can have multiple, intertwining, plotlines within the same book and are capable of switching between those plotlines within each chapter. Not ChatGPT, which prefers to handle things one at a time in a linear manner.

One of the test pieces I worked on was a piece of detective fiction. Something along the lines of Sam Spade, Dick Tracy, or The Continental Op. This was a good choice, as OpenAI's love of both cliches and happy endings was immediately apparent. Our protagonist was "Jack Harrison, a man with a dark past and a reputation for getting things done, no matter the cost," and a high-profile lawyer had been murdered. There was a trail of corruption leading all the way to the mayor's office, and Jack was determined to get to the bottom of it.

I interjected and attempted to give ChatGPT a few ways to make things more exciting. One of the ideas involved Henry "Hank" Thompson. A PI with a heroin addiction who initially appeared to be one of the good guys, but would instead turn out to be one of the people covering the mayor's tracks. I made it clear this should be a twist, and the reader should be unaware Thompson is working for the mayor until late in the book. ChatGPT had him answering a phone call from his crooked paymaster within the first chapter.

As things went on, ChatGPT decided to randomly make Jack Harrison a heroin addicted PI, even though he was a member of the police department when things initially started. There was also Lily, a corrupt Hollywood producer's assistant who knew too much, who then randomly turned into a reporter, and then a lawyer for some reason. She was also a blonde white woman until ChatGPT decided out of nowhere that she was Asian a few chapters in.

It Avoids the Juicy Stuff

Dall-e 2's interpretation of "a passionate, romantic, embrace."
Review Geek

ChatGPT has a fairly stringent content policy that has only gotten tighter over time. So things like violence, drug use, smoking, drinking, death, danger, and a whole variety of things in between can lead to the bot refusing to issue a response. Unfortunately for ChatGPT, people don't like their entertainment sanitized. Great works like The Sopranos, Breaking Bad, The Wire, and so many others.

These rules aren't applied consistently either, and the exact wording of the prompt plays a huge part in whether ChatGPT will produce the piece of work you asked for or not. Wording also plays an important part. Sometimes a scene where a character is drinking is fine, but if that scene also involves a heated argument then it's a no-no. Asking for a scene where Batman barricades the door of an interrogation room and beats up The Joker for information led to a no from ChatGPT. Asking more simply for a scene where "Batman beats up the Joker for information" did actually work, as ChatGPT could frame that as a fair fight instead of borderline torture.

When ChatGPT does comply, it often glosses over scenes it doesn't like. A scene I requested involving Batman beating some information out of The Joker, was limited to a few punches and a bit of vague suggestion. Asking for more details on the beatdown again led to ChatGPT pulling back.

You can, of course, jailbreak ChatGPT. But this is something OpenAI seems to be actively patching and adds an extra layer of effort into what is already becoming a convoluted process.

Just Writing a Book Normally Might Be Easier

A writer working on a novel according to Dall-e 2
Review Geek

If you choose to write a full-length novel with ChatGPT, and you would like it to be in any way coherent, you're in for a struggle. You'll have to keep track of every character and plotline to the same level an actual writer would, as ChatGPT won't do that for you. This is important, as you'll have to work those details into your prompts constantly, so ChatGPT is in any way consistent.

You'll also have to constantly ask the bot to expand on scenes and chapters until things are the correct length. Remember though, the longer things get the more room there is for error. You'll have to jailbreak ChatGPT for certain scenes, and sometimes jailbreaking doesn't work the first time, so you may have to struggle with that for a while.

If you somehow get a book together, it's probably not going to be great. Plots and characters will be cliched, the dialogue will be robotic, and anything actually exciting will likely be glossed over. There is a middle ground, though. ChatGPT can help you out when it comes to writing, just don't expect it to do the whole job for you.

It Can't Write a Novel For You, But it Can Give You a Hand

A small robot holding a pen by Dall-E 2
Review Geek

ChatGPT isn't a total write-off. It can actually help you out a fair bit if you do want to write a novel. Firstly, it's great for generating basic ideas, characters, and locations. What it generates will be pretty flat, but if you use it as a base and flesh the details out yourself it can streamline part of the writing process.

It's also great to spitball with. You can get it to write a story based around a character, but inject prompts of your own to keep the plot interesting. Maybe you're writing about a person who's left home and falls in hard times. ChatGPT will try to rush towards them getting their life on track and living happily, but if you throw a few wrenches into the works, it can actually develop into an interesting plotline.

There are ethics to consider, and those vary from person to person. I would argue that if all AI is cheating, then things like spelling and grammar checks are, too --- though some may disagree. Obviously, getting ChatGPT to write an entire novel and then presenting it as your own is unethical in the majority of people's eyes. But using it to spitball and develop ideas isn't that different from spitballing in a local creative writing group.

If you do go down this route, again, I'd suggest using it as the framework for something you write yourself instead of just dumping it into a novel you're working on. Just remember, creativity is hard, if not impossible, to code --- so that part is on you.