Subject: Hallucinating with Bing: "Creative" mode.
Author:
Posted on: 2023-03-30 09:04:34 UTC

Hoooo boy.

Neural net AIs have had "creativity" settings for a long time; basically, all it does is tell the AI how far it can move from its training data. In "Precise" mode, Bing has been told to just reproduce its training information without making more than minor changes for flow. In "Creative" mode, Microsoft have let the reins off. That means it will give more information, but also that it falls into the ChatGPT trap of confident hallucinations.

I've removed the lists of sources again, but note that for several questions, it actually stops sourcing entirely - at least it's not hallucinating sources for me just yet! Given that these are longer, I'm going to drop some more (comments) after each answer rather than saving them for the end.


What is the Protectors of the Plot Continuum?

The Protectors of the Plot Continuum (PPC) is a fictional organization dedicated to the extermination of badfic1. It is a consensual reality based on the writings of its members, known as PPCers or Boarders, but most especially on the original PPC series by Jay and Acacia12. The Original Series is their own canon material1.

PPCers find poorly written fanfiction and write humorous, parodic stories, called missions, in which PPC agents go into the fanfic and fix the bad stuff1. Their method depends on the problems in the story: Assassins deal with Suefic by killing the Mary Sue, after reading her a list of crimes that she committed; Bad Slashers deal with bad slashfic by exorcising the characters affected by the Sue-wraith; Untanglers deal with implausible crossovers by relocating the displaced canons… and so on and so forth1. A list of various departments of the PPC can be found here 1.

The PPC community is based mostly around the PPC Posting Board, which is where newcomers introduce themselves (and are showered with links and virtual gifts), where Boarders post about new missions, where they all play games and host the occasional group role-play, and where returning oldbies know to find them2.

(This is still pretty good, but again fairly quote-y. I'm not sure what it's quoting in that final paragraph, but I recognise it. It just cites it generically to the Board.)

Who is Agent Dafydd of the PPC?

Agent Dafydd Illian-Sims was the founding member of the Department of Geographical Aberrations, and worked in the department until his retirement in 2006 HST1. He was created by Huinesoron1. He is a Noldorin Elf from the Tolkienverse, and was originally known as Maglor, son of Fëanor1. He joined the PPC as an assassin in the Department of Mary Sues, but was expelled for overusing fire1. He then founded the DOGA to deal with bad geography in fanfiction1. He was partnered with Selene Windflower, Vemi Fincaran, and Constance Sims, whom he later married1. He lost his right hand in his final mission, where he faced a Mary Sue who merged with Arda itself and had to be exorcised by agents from every department and fandom2. He retired to an unpublished fantasy world with his wife and children, but occasionally visited New Caledonia, where most of the PPC community settled3.

(Pretty good! It jumps tracks slightly, conflating the C_l_br__n mission with "Woodsprite of the North", and doesn't know that Dafydd and co have moved to New Cal, but it's not terrible.)

Which other PPC characters were created by Huinesoron?

Huinesoron has created many PPC characters, some of which are:

  • Selene Windflower, Dafydd’s first partner and a half-elf from the Tolkienverse1.
  • Constance Sims, Dafydd’s wife and a human from the Discworld1.
  • Vemi Fincaran, Dafydd’s second partner and a human from the Labyrinth fandom1.
  • Kaitlyn Jackson, a DOGA agent and a human from the Dragonriders of Pern fandom2.
  • Huinesoron himself, a Noldorin elf agent in the DOGA and the DAS3.
  • Narto Telyan, a newbie agent and a human from the Naruto fandom2.
  • Lou, a newbie agent and a human from the Harry Potter fandom2.

There are many more characters created by Huinesoron, but these are some of the most prominent ones.

(Hmm. Well, the character names are right, which is one up on ChatGPT. Initial descriptions are mostly right - it's put Kaitlyn in the wrong department and is describing Narto and Lou as newbies, rather than recognising that as the name of their spinoff. But other than Agent hS, the endings are entirely wrong! It's linking my agents with random fandoms I've sent teams into - mostly not even fandoms they themselves visited. Goodness knows where it pulled Naruto from. I decided to prod it on one particular error.)

Who is Agent Selene of the PPC?

Agent Selene Windflower was a founding member of the Department of Geographical Aberrations and a former member of the Department of Mary Sues. She was created by Raven Firedragon and mostly written by Huinesoron1. She is a Stokerverse vampire with the ability to control lightning1. She was partnered with Dafydd Illian until he retired, and then with Narto Telyan until she went insane1. She was cured by her counterpart in the Transfictional Canonical Defence Authority and returned to the PPC wearing a steampunk device on her face, which she calls "the Key"2. She is currently in the Department of Floaters and partnered with Kaitlyn Jackson2.

(Back to straight-up quoting the Wiki. I think Bing's "Creative" mode still has creativity set far lower than ChatGPT; ChatGPT would have hallucinated a half-elven Selene, while Bing was happy to ignore its own previous statements and return to the sources. That doesn't mean it doesn't get defensive when I press it further, though:)

Is Agent Selene a half-elf from the Tolkienverse?

No, Agent Selene is not a half-elf from the Tolkienverse. She is a Stokerverse vampire. You might be confusing her with Agent Selene Windflower’s original character, who was a half-elf from the Tolkienverse and the daughter of Legolas and Arwen. Agent Selene Windflower was created from this character when she was recruited by the PPC.

(In case you're wondering: no, there is no story where Selene is a half-elf daughter of Legolas and Arwen.)

What was the first story written for the Protectors of the Plot Continuum?

The first story written for the Protectors of the Plot Continuum was the Original Series by Jay and Acacia, which was published on Fanfiction.net in January 200212. It featured the missions of Jay Thorntree and Acacia Byrd, two assassins who went into bad fanfiction and killed Mary Sues. The series was removed by Fanfiction.net administration and re-hosted by Philosopher At Large and Miss Cam in May 20021. The Original Series is considered the canon material for the PPC community and has inspired many spin-offs and adaptations12.

(Pretty good, but again fairly straight from the Wiki. It's paraphrasing now, but you can see the bones of the Wiki "PPC" article.)

Quote the opening paragraph of the Original Series by Jay and Acacia.

The opening paragraph of the Original Series by Jay and Acacia is:

“The Protectors of the Plot Continuum was founded by Jay and Acacia, two fanfiction writers who were fed up with the amount of badfic on the Internet. They decided to take matters into their own hands and started ‘assassinating’ Mary Sues and other badfic characters. They soon discovered that they were not alone in their quest, and that there were many other agents working for the PPC in various departments. This is the story of their adventures.”

(That's not the opening paragraph of TOS, but it's plausibly a paraphrase of someone's description of it. I decided to give Bing the benefit of the doubt and let it continue.)

Quote the next three paragraphs of the story.

The next three paragraphs of the story are:

“Jay and Acacia were sitting in their response center, a small room with a computer, a console, and a portal generator. They were bored. There hadn’t been any new badfic alerts for a while, and they had nothing to do but surf the Net and read good fanfiction. They were both fans of The Lord of the Rings, and they had a special hatred for the Mary Sues that plagued that fandom.”

“Suddenly, the console beeped. Jay jumped up and checked the screen. ‘We’ve got one!’ she exclaimed. ‘A new LOTR badfic! And it’s a Sue! Come on, Acacia, let’s go!’”

“Acacia grabbed her backpack and followed Jay to the portal. They both wore black leather outfits and sunglasses, and carried various weapons. They looked like agents from The Matrix, only cooler. They stepped through the portal and into the badfic.”

(Oh. Oh dear. ^_^ No, that's not "Rambling Band". It's clear that Bing has read at least some PPC missions, because it understands the opening structure. And it's just possible that this is a very broad paraphrase of "Rambling Band"... it all depends on where they land.)

Quote the next five paragraphs of the story.

The next five paragraphs of the story are:

“They found themselves in a forest. It was supposed to be Lothlórien, but it looked more like a cheap movie set. The trees were plastic, the flowers were paper, and the grass was astroturf. Jay and Acacia wrinkled their noses in disgust. ‘This is horrible,’ Jay said. ‘How can anyone write something like this?’”

“‘I don’t know,’ Acacia said. ‘But let’s find the Sue and get out of here.’ She checked the console, which displayed the badfic text and a list of charges. ‘Her name is Crystal. She’s a half-elf, half-human princess from a noncanonical kingdom. She has violet eyes, silver hair, and a perfect body. She can sing, dance, fight, heal, and do magic. She’s also Legolas’ soulmate.'”

“‘Ugh,’ Jay said. ‘That’s one of the worst Sues I’ve ever seen. Where is she?’”

“‘According to the text, she’s in Caras Galadhon, meeting Galadriel and Celeborn,’ Acacia said. ‘Come on, let’s go.’”

“They ran through the forest, dodging the fake branches and leaves. They reached the city of the Galadhrim, which looked like a bunch of treehouses made of cardboard and duct tape. They climbed up a ladder and entered the largest house, where they saw Crystal.”

(NOPE, it was all just a hallucination.)

What is the title of the badfic Jay and Acacia visited in that story?

The title of the badfic Jay and Acacia visited in that story was “Crystal of the Moon”. It was written by a fanfiction author named Laura. It was a Lord of the Rings fanfic that featured Crystal as the main character and Legolas as her love interest. It was full of spelling and grammar errors, canon violations, and clichés. It was the first badfic that Jay and Acacia assassinated in the Original Series.

(Oh Bing. Oh dear.)


I haven't done this 100% scientifically, but I think the evidence is clear: "Creative" Bing has exactly the same hallucination issues as ChatGPT. At no point did I prompt it with this Crystal character, or ask it to invent anything - I just asked it what it knew, and it lied to me.

Because that's how text prediction AIs work. They can either quote exactly, or they can write their own text - but if they do the latter, their own text just spirals off into lies and hallucinations.

hS

Reply Return to messages