What are AI Hallucinations?
If you use GenAI tools like ChatGPT, you know that some results can be a bit strange, even wrong at times. This is called a hallucination. What causes these tools to hallucinate and how can we prevent that from happening?
Many of us think of hallucinations as seeing something that doesn’t exist or hearing noises that no one else can hear. In the context of AI, hallucinations occur when the model makes a prediction that doesn’t come from the input data.
Why Do AI Hallucinations Occur
Data Issues
If the model was trained with an insufficient or biased data set, the outputs are also going to be biased. This can lead to predictions that can create issues with trust, human safety, or user wellbeing. If the model was trained with the perfect data set, then you have an overfitting problem. This means that the model is not going to be very good at predicting things that relate to a real world data set. Both bad data and perfect data are not good and there needs to be a carefully balanced data set used to train your AI model.
Lack of Real-World Grounding
We haven’t quite reached general intelligence and certainly haven’t experienced super intelligence. Because of this, AI models often lack a fundamental understanding of the real world which leads to mistakes when generating or predicting outputs.
Model Issues
Models can run into many issues including their architecture and risk of being attacked. Since many AI models are complex in nature, they require an architecture that can handle depth and capacity to produce the appropriate level of responses. Hallucinations could also occur because the model has been victim of an adversarial attack, which involves manipulating input data with a goal of producing incorrect outputs.
What Can be Done to Get Rid of Hallucinations?
For one, we can never fully get rid of the risk of hallucinations, but we can reduce the likelihood of them occurring.
Common controls that are being implemented to help reduce AI hallucinations are to implement guardrails on input and output data. Specifically, all data inputs should be checked for data quality, diversity, and relevancy to the goal of the particular model. Also, all inputs and outputs could be filtered through a template to provide structure and ensure consistency.
You can also reduce the impact and risk of these hallucinations by using professional skepticism. This is a term that’s widely used in the audit community that basically means to review and question everything and to apply a critical mindset. You can’t just ask ChatGPT a question and copy/paste the answer into your workpapers.
A perfect example of what NOT to do comes from a situation when a New York lawyer used ChatGPT to conduct research for a court filing which referenced cases that were not real. The lawyer apparently didn’t understand the tool’s full capabilities as they had only read articles about the benefits of AI in professional settings. Legal professionals said that this was a basic mistake while other experienced counsel members called out the failure to verify.
Biggest takeaways here are to be mindful of the inputs you are using to not increase the risk of generating hallucinations in the first place and to conduct independent verification on all outputs. Emphasis on the verify!
Have you experienced an AI hallucination?
Originally published at www.medium.com.
I love working with Internal Audit teams to help them develop strategies to leverage analytics and AI to modernize their audit approach. If you’re interested in leveling up your audits, let’s chat!
