Certain names make ChatGPT grind to a halt, and we know why

In particular the “David Mayer” block (now resolved) raised more questions, Posted first on Reddit On November 26, multiple people shared this name. Reddit users speculate with David Meyer Rothschildalthough there is no evidence to support these theories. Tuesday, OpenAI tell the guardian Blacklisting David Mayer is a glitch.

“One of our tools incorrectly flagged this name and prevented it from appearing in responses when it shouldn’t. We are working on a fix,” an OpenAI spokesperson told the Guardian.

Problem with hard-coded filters

Allowing a certain name or phrase to consistently corrupt ChatGPT output could cause a lot of trouble for some ChatGPT users, making them vulnerable to adversarial attacks and limiting the usefulness of the system.

Scale AI Tips engineer Riley Goodside has discovered how attackers disrupt ChatGPT sessions Use visual cues to inject The name “David Mayer” is embedded in the image in a light, almost illegible font. When ChatGPT sees an image (in this case, a mathematical equation), it stops, but the user may not understand why.

This filter also means ChatGPT may not be able to answer questions about this article when browsing the web, e.g. via ChatGPT and search. If someone adds a banned name to the website text, someone could use it to potentially prevent ChatGPT from browsing and processing the website.

There is also the inconvenience factor. Preventing ChatGPT from mentioning or processing certain names, such as “David Mayer” (which may be a popular name shared by hundreds or even thousands of people) means that people who share that name will have difficulty using ChatGPT. Or, say you’re a teacher and you have a student named David Mayer, and you need help sorting the class list, ChatGPT will reject the task.

AI assistants, LL.M.s, and chatbots are still in their very early stages. Their use has opened up many opportunities and vulnerabilities that are still being explored on a daily basis. How OpenAI solves these problems remains an open question.

This article was updated at 3:50 pm on December 3, 2024, with a new statement from OpenAI about the failure of “David Mayer” in its ChatGPT block (sourced from The Guardian).