AI chatbots are getting worse over time — academic paper

A recent research study titled “Larger and more instructable language models become less reliable” in the Nature Scientific Journal revealed that artificially intelligent chatbots are making more mistakes over time as newer models are released.

Lexin Zhou, one of the study’s authors, theorized that because AI models are optimized to always provide believable answers, the seemingly correct responses are prioritized and pushed to the end user regardless of accuracy.

These AI hallucinations are self-reinforcing and tend to compound over time — a phenomenon exacerbated by using older large language models to train newer large language models resulting in “model collapse.”

Editor and writer Mathieu Roy cautioned users not to rely too heavily on these tools and to always check AI-generated search results for inconsistencies:

“While AI can be useful for a number of tasks, it’s important for users to verify the information they get from AI models. Fact-checking should be a step in everyone’s process when using AI tools. This gets more complicated when customer service chatbots are involved.”

To make matters worse, “There’s often no way to check the information except by asking the chatbot itself,” Roy asserted.

Related: OpenAI raises an additional $6.6B at a $157B valuation

The stubborn problem of AI hallucinations

Google’s artificial intelligence platform drew ridicule in February 2024 after the AI started producing historically inaccurate images. Examples of this included portraying people of color as Nazi officers and creating inaccurate images of well-known historical figures.

Unfortunately, incidents like this are far too common with the current iteration of artificial intelligence and large language models. Industry executives, including Nvidia CEO Jensen Huang, have proposed mitigating AI hallucinations by forcing AI models to conduct research and provide sources for every single answer given to a user.

However, these measures are already featured in the most popular AI and large language models, yet the problem of AI hallucinations persists.

More recently, in September, HyperWrite AI CEO Matt Shumer announced that the company’s new 70B model uses a method called “Reflection-Tuning” — which purportedly gives the AI bot a way of learning by analyzing its own mistakes and adjusting its responses over time.

Magazine: How to get better crypto predictions from ChatGPT, Humane AI pin slammed: AI Eye