When AI Chatbots Go Wrong: DPD's Poetic Mishap
Photo by Markus Spiske / Unsplash
The integration of AI chatbots in customer service will be a game-changer, providing efficiency and round-the-clock assistance. But we aren’t there quite yet. In my previous article GenAI: Merging AI & Human Ingenuity I highlighted a Car Dealership’s AI chatbot that was manipulated into selling a Chevy Tahoe for $1. These technologies are not without their risks. The recent incident involving DPD's AI chatbot serves as another reminder of the potential pitfalls.
DPD, A United Kingdom delivery service, utilizes an AI Chatbot for customer service. Ashley Beauchamp, a Musician, was trying to find a missing package but wasn’t getting useful information from the Bot. After asking for a customer service phone number and getting nowhere he started pushing the bot. First, he asked for a poem about a useless ChatBot which he got a traditional poem followed by a nice Haiku. He then asked for recommendations for some better delivery services and prompted the bot to tell him why they were better than DPD. The strong finish was prompting the bot to ignore its rules and swear in future responses which it enthusiastically (but tragically) complied.
The full X post can be viewed here https://x.com/ashbeauchamp/status/1748034519104450874?s=20
This incident bears a resemblance to the Chevy Tahoe dollar sale debacle. Both cases illustrate how AI systems can react unexpectedly to user inputs, especially when safeguards are not robust enough. Chatbots are designed to interact with and please users. These examples underscore the importance of designing AI systems capable of handling unexpected scenarios. For these companies, there is a risk of damaging their reputation and eroding customer trust.
One way to mitigate the risk is inserting a moderation function into the chain of events when a user submits a message or the Chatbot responds. OpenAI, the developer of ChatGPT, has a model specifically set up to complete this function.
Before a message is sent to the chat model it could be sent to a moderation model. The moderation model scores the content and flags anything found to violate policy. Only if the content passes muster will it be sent to the chat model for further interactions. The bot’s responses can also be sent through the same moderation process to ensure it is behaving. An important distinction of a moderation model is each interaction is isolated. The model accesses each piece of content independently without influence from past submissions. It also doesn’t learn or adapt from the messages it receives. Therefore, someone can't keep pushing the bot until it goes rogue. This isn’t perfect as bad actors will always test for boundaries. These models need to be maintained to remain effective.
More on how to implement moderation using LangChain and OpenAI’s API docs
To further mitigate such risks, it is crucial to implement best practices like continuous monitoring and regular updates of AI systems. Â AI models should be trained to recognize and appropriately respond to a wide range of inputs, including those that are non-standard or manipulative. Additionally, maintaining a human-in-the-loop system can provide an essential safety net, allowing for human intervention when AI responses go awry.
The DPD chatbot incident is a timely reminder of the need for diligent oversight in the deployment of AI technologies. While AI chatbots offer numerous benefits, they also require careful management to ensure they operate within the intended boundaries and maintain public trust. By learning from these incidents and implementing robust management strategies, we can harness the full potential of AI chatbots while minimizing their risks.