Subject: It's a feature, not a bug.
Author:
Posted on: 2023-03-26 21:35:08 UTC

(Disclaimer: The below is my personal understanding, presented as fact. Just call me ChathS.)

ChatGPT - all the Internet-taught AIs, in fact - really is a predictive text engine. It has been explicitly built to predict what the Internet would say next, in a conversation or a sentence or an essay. It's been calibrated so it doesn't straight-up regurgitate existing text, but adds something like it, based on a whole Internet's worth of like.

It can't analyse - it can just write analysis. It can't create - it can just remix creations. And, crucially, it can't give you facts - it can give you sentences that are shaped like facts.

It doesn't know what facts are. All it knows is what the Internet would say next. If that's something that the Internet agrees on, it will state it as a fact and resist argument - I tried in vain to get it to say Jupiter wasn't a planet. If it's something that isn't in its data at all, it might say so - it wouldn't give me a story about someone betraying the PPC, because nobody has. But for something that has the same shape as something in its data, like PPC agents fighting or falling in love? It will happily predict what the Internet would say next.

It can't do anything else, not and remain an "AI" . A prediction engine can't simultaneously create novel-seeming analysis, and not create novel-seeming facts - because to the engine, those are the same thing. They're both just Internet text.

(You can cheat by giving it instructions before the user gets to it; ChatGPT does that, that's why it keeps saying it's a language model. I'm sure they have told it not to claim certain things as facts, and to refuse certain topics. But that's not gonna help us here.)

If you've not run into it before, I highly recommend Janelle Shane's "AI Weirdness" blog, https://www.aiweirdness.com/. She's been making AIs demonstrate their limits since they were trained on a few thousand words, and continues to do so with the Internet-trained generation. It's good fun.

hS

Reply Return to messages