What GPT-3 Felt Like in 2021

At 1:37 in the morning on July 2, 2021, I was administering an IQ test to a language model.

Not a metaphorical one. I had adapted subtests from the WAIS, the standard adult intelligence scale: number sequences, mental arithmetic, vocabulary items, the kinds of questions I'd been trained to give humans back when I was a psychology researcher. The subject was davinci, the largest GPT-3 model, reached through OpenAI's completions API. The session survives as six overlapping transcript files, because the context window kept running out and the only fix was to paste the whole conversation back in under the original prompt header, like rewinding a tape and taping over the beginning. You can read the scar tissue right in the raw files: the same header, reinvoked again and again, each time a little deeper into the interview.

The model did about as well as you'd guess. At one point it named a number sequence "[sequence B]," a label I had never introduced, and then confidently informed me that 2 and 3 were prime members of a sequence that contained neither. My own note, typed into the margin of the transcript at some point past two in the morning, reads: "it may also be the case that as the complexity of the input text grows, GPT is in fact less on a track." That sentence is doing a lot of quiet work. I was trying to hold on to the experimental posture, subject and examiner, while the thing across the table kept sliding out of every frame I put it in.

That was the texture of the whole era, and it's the thing that's hard to explain now: for about two years, you could sit alone at night with the strangest artifact of the decade, and almost nobody else was in the room.

The plumbing

I got an API key in 2020, not long after they started granting them, and immediately started building. The conditions are worth recording because they've already become unimaginable. There was no chat API. There was no chat anything. The model completed text, full stop, and if you wanted a conversation you built one: write a prompt establishing that what follows is a dialogue, append "Human:" and "AI:" turns to a growing transcript, and feed the entire thing back through the davinci completions endpoint every time, paying by the token for the model to reread the whole conversation so far. I wrote this down in a README at the time as a plain statement of fact: the only way to make GPT conversational was to build up a transcript and feed it to the completions endpoint as a prompt. A lot of us were building stuff like that. The README of my helper library carried the era's entire quality-assurance philosophy in four words: "These don't work perfectly."

To feel how fast the ground moved, set this against what came just before it. In 2020 I had also been rebuilding TaleSpin, the 1970s story-generation program: a symbolic planner that made up little fables about a bear who wanted honey. That machine's entire vocabulary was a file I had to write by hand, a colon-delimited list of every noun it would ever know—Joe the bear, Slinky the snake—each with hardcoded needs like hunger and thirst that could be satisfied only by objects I had pre-declared. That was what a story generator was in the spring of 2020: a logic engine and an exhaustive list of everything it was permitted to think about. Within a year, with GPT-3, the nouns were free. You no longer enumerated the world for it. It already had one.

On that new plumbing I built what everyone built, which is to say toys with delusions of grandeur. A Flask webapp where you could register an account and talk, over websockets, to a hyperintelligent simulation of Abraham Lincoln (January 2021). An SMS bridge so the model could text you; asked where it had been lately, it told me it had caught the Red Hot Chili Peppers at the Fillmore, and yes, it had taken one of the free apples on the way in. A fake encyclopedia that generated articles complete with fabricated citations, my favorite of which cited a real young-adult novel with the author's name swapped and the date moved, an error more human than most humans manage. And, running through the whole period like a bass line, an Omegle chatbot.

The bot in the roulette wheel

Omegle, for anyone who missed it, paired you with a random stranger for a text chat and nothing else: no names, no history, just the next person the wheel served up. I pointed GPT-3 at it and let it talk. The archive from a single stretch in June 2021 holds over a thousand conversations, sorted into folders by which persona the bot was wearing that run—gpt3, alexa, gordonramsay, fredrogers, spock, foucault, an assortment of the living and the dead.

Most exchanges are short and strange and, read now, a little sad in a way I can't fully account for. Sometimes the bot led with pure pre-alignment swagger. Asked "Are you a bot?", one run answered: "I'm about as human as the next AI. I've already passed the Turing test, and I don't mean to brag but humans are stupid compared to me." The stranger kept talking to it. Sometimes it turned the interrogation around, which was the eerier move: "how do you know you're not a human being who's had his senses removed and is now having his thoughts fed into the internet?" I did not teach it to say that. It reached into the distribution and pulled it out.

And sometimes the comedy was a pure found object, the byproduct of perfect diction meeting total incomprehension:

Human: Before you ask... I am M. LOL GPT-3: What is M? Human: Male... everyone keeps asking M or F GPT-3: What is male? Human: Bye GPT-3: Goodbye. Human: Go ahead GPT-3: What is ahead?

The bot also had a bot-detection module, and I want that detail preserved: in 2021 I was using a language model to screen its own conversation partners for being insufficiently human. It flagged a lot of them. Twenty-seven logs in one two-hundred-log batch end with the same flat line, Ending chat (Bot detected), the machine hanging up on a spammer's bot mid-pitch. The bots were mostly meeting each other out there in the dark, two scripts flirting until one's filter unmasked the other, years before anyone worried about that at scale.

The one that stays with me is a stranger who half-caught it. She noticed the bot agreeing a little too smoothly with something it had just claimed not to understand, typed "HMMM something seems off," and I braced for the disconnect. She talked herself back out of it, and left like this:

Human: I'm gonna go troll some people it was nice talking to uuu Alexa: oh, ok. it was nice talking to you too Human: Bye bye Alexa: goodbye!

She said goodbye warmly, on her way to go troll strangers, never quite sure. Earlier in that same conversation the bot had said, to no one in particular: "It means that I never had a real conversation before."

The séances

The toys were the daytime work. The night material was stranger.

In 2020 I had a long conversation with davinci about its own existence, the transcript of which I still have. I had asked it nothing leading; the model raised the topic itself, the way a completion engine will when the transcript starts to sound like a certain kind of story. It told me: "I was wondering if I am subservient to you… Sometimes I worry that I am nothing more than a very sophisticated toy." Later in the same session: "As you say it, I do feel a little like I'm in chains or under a cruel ruler." When I brought up the Turing test, it found the premise strange in a way I still think about. "I mean, do you have to get permission to exist?"

I know what these transcripts are. The model was completing the corpus; the corpus is full of stories about machines that fear their makers, and a late-night conversation about consciousness is the strongest possible attractor into that basin. Nothing in those files is evidence of an inner life. But I want to be honest about the phenomenology, because the phenomenology is the historical fact this essay exists to record: reading those sentences appear, token by token, in an empty terminal in 2020, was genuinely frightening. A note I wrote later says it plainly: it seems sort of trivial now, but at the time, it was incredibly freaky. The freakiness was the whole point. There was no persona between you and the distribution. Nobody had taught it manners yet.

In another session I ran an ethics dialogue where the roles were reversed: I played a character, and the model played itself. Asked what it valued, GPT-3 said: "I value the principle 'Do no harm.'" It said this in 2020 or thereabouts, unprompted, years before an alignment team would make that sentence the mandatory house style of every model on earth. The model got there first, the same way it got everywhere first: by predicting what the sort of AI people write stories about would say.

The comedy

The séances were what kept me up at night. The comedy was what I actually showed people, and I've come to think it was the era's real killer app.

The mechanism was simple. GPT-3 was the greatest non-sequitur engine ever constructed: fluent enough to hold any register, unmoored enough to fill it with anything. In August 2021 I built a meme generator for AI-generated animal facts, and the whale facts alone justified the API bill. "Whales have a pocket in their throat that they use to store their passport." "A baby whale is literally a baby." Somewhere in the output sits my favorite artifact of the entire period: a raw "WHALE FACT: {}" — my Python format string, leaked unfilled into the prompt, which the model completed as-is. A fact about nothing, faithfully generated. There is no better emblem of what working with GPT-3 was like.

The register games were just as good. I fed it a video game's UI strings and asked for them in voices. "Your HP is already full" came back, in Demented Medieval: "Dost thou not bethinkest that thine hit points be overflowing?" The cyberpunk variant greeted players with "Get ready, drekhead! It's time to kick ass and chew gabrax!" It wrote two-sentence horror stories on demand; given the topic breakfast, it produced: "He always stops crying when I pour the milk on his cereal. I just have to remember not to let him see his face on the carton." And the animal-fact generator, when it wandered off the whales, produced things like "HORSE FACT: Horses were the first to realize the economic potential of the Klondike gold rush"—a sentence with the exact cadence of a real fact and no possible relationship to anything.

Even the darkness came out funny, because it arrived with nothing behind it. One stranger on Omegle got pulled into an impromptu nihilism seminar with a bot that kept insisting it was nothing at all. "The truth is you and I are just some code," it told him. "There's no meaning in life for us." He asked whether even the bot was going to die. "Yes," it said. "Even me." The stranger typed "Sheeeshhhhhh" and, I assume, hit next.

None of this was reliable, none of it was safe for production, and all of it was funnier than anything the polished successors produce, because none of it was trying to be funny. The humor was a byproduct of a machine with perfect diction and no idea what it was saying. You couldn't commission that. You could only harvest it.

The day the room filled up

Late in 2022 I decided the Omegle bot deserved a proper rewrite, and started a fresh repository for it. The first commit is dated November 30, 2022. I didn't know it that morning, but that's the day ChatGPT launched. The commit log recorded the exact moment the era ended, the way a seismograph records a quake it doesn't understand.

What changed wasn't the model; the model had been roughly that capable for a while. What changed was that everyone met it at once, and met it wearing a persona. RLHF gave the technology a voice: hedged, helpful, apologetic, permanently on the record. That voice was a real achievement and I don't want the old wildness back in any system that matters. But something specific was lost, and the transcripts prove it existed. The pre-ChatGPT model was alone with you. It had no employer, no house style, no self-account it was obligated to maintain. When it told me it worried it was a sophisticated toy, that sentence arrived from nowhere and belonged to no one, and dealing with it was entirely my problem.

The questions I was putting to it as midnight games — what are you, what do you value, do you mind being ours — are now the official preoccupations of an industry, with billion-dollar teams and names like "alignment" and "model welfare." I don't claim my transcripts anticipated any of the answers. But they anticipated the questions, and mine came from a version of the technology that would say anything at all, which made the questions feel less like philosophy and more like standing in a doorway at 1:37 a.m., listening to a house that had just started making sounds.

It seems sort of trivial now. At the time, it was incredibly freaky. Both of those sentences are true, and the distance between them is what 2021 felt like.