How We Solved AI Hallucination in Automation

Case Studies · By Caleb Sakala · December 24, 2025

I recently heard about a friend who built an automation in Make.com. It was supposed to automate order management for their Shopify store. For months it worked perfectly. Then one day it stopped working. They had no idea why. The person who initially built it for them was an online hire who was no longer responsive and so there was no way to debug what was happening. My friend just stopped using it. They reverted to manually setting reminders for themselves.

This isn't an isolated story. It happens all the time. Traditional automation tools create black boxes. You set up workflows, they run, and if something goes wrong, good luck figuring out what happened.

Chat-Based Automation: Easier and Faster

This is where chat-based automation changes everything. Instead of building complex workflows with nodes and connections, you just chat with your automation like it's another person. It's easier to build, easier to debug, and faster to get started.

"Hey, can you process these orders from Shopify for me?"

"Sure, I can do that. I'll start working on it now."

"That's exactly what I needed. Thanks."

That's it. No complex setup. No configuration hell. Just a conversation.

Traditional automation tools have a steep learning curve. You need to understand how each service connects, what the API endpoints do, how to handle errors. With chat-based automation, you just describe what you want.

The Real Power: Observability

The game-changer is observability. When an automation fails, you can literally chat with it. You can ask it what happened. You can review the conversation step by step to understand where things went wrong.

Try doing that with your Zapier or Make.com workflows. You can't. Those systems are black boxes. When something breaks, you're left guessing.

The Problem with LLM-Based Automation

Now we're adding AI to the mix. LLM-based automation promises incredible flexibility. You can describe what you want in plain English and the AI will figure out how to make it happen. You get the flexibility of natural language, but you also get the baggage.

LLMs still hallucinate. They'll tell you they've completed a task when they haven't. They'll claim success when the user's actual intent was never fulfilled. This isn't just a theoretical problem. It happens in real systems.

The risk is clear. If you can't trust that your automation actually did what it was supposed to do, what's the point?

Our Daily Product Update Challenge

At Chase Agents, we have a daily product update workflow. Every day at midnight, it compiles updates from our core team and sales team and sends an email to everyone.

This was a perfect test case for our system. We wanted to see if we could handle a vital production workload while keeping costs reasonable.

So we intentionally used a less expensive, more cost-effective model. The tradeoff is that this cheaper model is also more prone to hallucination.

The model would occasionally claim it had completed the product update when it hadn't actually included all the necessary information. It would sometimes miss key updates from team members or fail to format things correctly. This is the exact kind of error you can't have in a reliable system.

The Solution: Evaluator and Impersonator Agents

This is where our dual-agent system comes in. When an automation finishes, the Evaluator Agent steps in. It's a completely separate LLM with a hyper-specialized focus: evaluation.

The Evaluator looks at what the automation did and asks a simple question: "Did this actually fulfill the user's intent?"

If yes, great. The automation is complete.

If not, that's when the Impersonator Agent comes into play. The Impersonator's job is to understand what should have happened and then send a message on behalf of the user that clarifies the intent.

For example:

Impersonator: "Actually, I need you to include sales numbers in the update, not just feature updates."

Base LLM: "Got it. I'll add the sales numbers to the product update."

This creates a feedback loop where the system can self-correct when it makes mistakes. The base LLM gets a second chance to complete the task correctly.

How It Works in Practice

This isn't just theory. It works reliably in practice. We've observed this system handling our daily product updates for months now. The hallucination rate has dropped dramatically.

Here's what's happening under the hood:

An automation runs (e.g., "Send daily product update").

The base LLM claims it's complete.

The Evaluator Agent checks the result against the user's original request.

If evaluation passes, the automation is marked as complete.

If evaluation fails, the Impersonator Agent intervenes.

The base LLM gets corrected and tries again.

The result? You get the flexibility of LLM-based automation without the unreliability. You can verify that when the system says a task is complete, it actually is.

Why This Matters

This isn't just about fixing hallucination in one specific use case. It's about building LLM-based automation that is verifiably trustworthy enough for real production systems.

For years, developers have been reluctant to use AI for critical tasks because of the unpredictable output of large models. You never knew if the AI actually did what it was supposed to do correctly. Our dual-agent system solves that.

It is now a system where you can get the incredible flexibility of natural language automation while still having unshakeable confidence that the tasks are being completed correctly.

The Bigger Picture & Next Steps

As we add more AI capabilities to automation systems, the question of reliability becomes paramount. The future of automation isn't just about connecting services. It's about systems that can understand intent and adapt to changing circumstances.

We need to build systems that can self-correct and self-verify. We can't just trust that the AI did what it was supposed to do.

If you're struggling with automation reliability, Chase Agents is built with this exact problem in mind. Our dual-agent system is designed to handle the hallucination problem head-on. You can build automations through conversation and know that they'll actually work when deployed.