AI agent vs chatbot: the difference is one while loop

AI Agents · By Caleb Sakala · April 30, 2026

Every vendor article explaining the AI agent vs chatbot distinction draws the same picture. Chatbots respond. Agents act. Chatbots are reactive. Agents are proactive. These descriptions are accurate in the way a horoscope is accurate: vague enough to feel true, specific enough to be useless. The AI agent vs chatbot debate deserves a more concrete answer.

The real difference is structural, and you can see it in about 20 lines of Python. A chatbot takes a message, calls an LLM, returns the response. An agent wraps that same LLM call in a loop that checks whether the task is done. That loop is the entire distinction.

A chatbot is a single LLM call with no memory of what it decided

Here is a chatbot:

from openai import OpenAI
client = OpenAI()

def chatbot(user_message: str) -> str:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": user_message}]
    )
    return response.choices[0].message.content

One request in, one response out. The LLM has no way to take action, verify its answer, or change course halfway through. Ask it to "book a flight to Denver" and you get a paragraph about how to book flights. The model knows what booking a flight involves. It cannot book one.

Most chatbots in production are slightly fancier versions of this. They prepend a system message, maybe stuff some retrieved documents into the context. Retrieval-augmented generation makes a chatbot more knowledgeable, but it does not make it an agent. The execution path is still linear: gather context, call the model, return text.

An agent is the same LLM call inside a loop that can use tools

Here is the same model promoted to an agent:

import json
from openai import OpenAI
client = OpenAI()

tools = [
    {
        "type": "function",
        "function": {
            "name": "search_flights",
            "description": "Search available flights between two airports on a date",
            "parameters": {
                "type": "object",
                "properties": {
                    "origin": {"type": "string"},
                    "destination": {"type": "string"},
                    "date": {"type": "string", "format": "date"}
                },
                "required": ["origin", "destination", "date"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "book_flight",
            "description": "Book a specific flight by ID",
            "parameters": {
                "type": "object",
                "properties": {
                    "flight_id": {"type": "string"},
                    "passenger_name": {"type": "string"}
                },
                "required": ["flight_id", "passenger_name"]
            }
        }
    }
]

def agent(user_message: str) -> str:
    messages = [{"role": "user", "content": user_message}]

    while True:
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=messages,
            tools=tools
        )
        msg = response.choices[0].message
        messages.append(msg)

        if msg.tool_calls:
            for call in msg.tool_calls:
                result = execute_tool(call.function.name,
                                      json.loads(call.function.arguments))
                messages.append({
                    "role": "tool",
                    "tool_call_id": call.id,
                    "content": json.dumps(result)
                })
        else:
            return msg.content

The model, the API, and the prompt format are identical. What changed is the while True loop and the tool definitions. Instead of returning on the first response, the agent checks whether the model requested a tool call. If it did, the agent executes the tool, feeds the result back, and lets the model decide what to do next. The loop only breaks when the model responds with plain text, meaning it considers the task complete.

That loop is doing four things on each iteration: observe (read the latest tool result), reason (decide what to do next), act (call a tool or respond), and evaluate (is the task done?). Some people call this the ReAct pattern. Others call it an agentic loop. The name does not matter. The loop does.

Tool access without a loop is still a chatbot

Here is where most vendor comparisons get sloppy. They equate "has tools" with "is an agent." OpenAI's function calling documentation defaults to a single-pass pattern where the developer checks for tool calls, executes them, and makes one more API call to get a final response. That two-step pattern handles simple cases like weather lookups. It breaks on anything requiring multiple steps.

Booking a flight requires searching, selecting, confirming, and handling errors if the selected flight is full. A two-step pattern cannot do that because the model needs to see the search results before choosing, and it needs to see the booking confirmation before reporting success. Without the loop, the developer has to hardcode the sequence: search first, then book. At that point the LLM is a fancy text formatter, and the developer is the actual agent.

A real test for whether something qualifies as an agent: can the model decide at runtime how many tool calls to make and in what order? If yes, agent. If the developer pre-determines the sequence, chatbot with tool access.

Memory is the other half of the gap

The code above has a subtle feature: the messages list grows with every iteration. Each tool result and each model response accumulates in the conversation history. The model sees its own prior reasoning when deciding what to do next.

Chatbots typically start fresh every turn. A support chatbot answers your question, forgets the conversation (or relies on a truncated window), and treats your next message as unrelated. Agents need to remember what they already tried, which tools they called, and what the results were, because the current step depends on the last one.

Memory comes in different shapes. Short-term memory is the conversation history within a single task, like the messages list in the code above. Long-term memory persists across sessions, often stored in a vector database or a structured log. Salesforce's Agentforce documentation, for instance, describes "Topics" that maintain context across conversations. Anthropic's Claude supports a 200K token context window, which means a sufficiently short task can keep its entire reasoning history in a single prompt.

The practical constraint is cost. A 10-step agent loop with GPT-4o might accumulate 15,000-20,000 tokens across all the tool calls and intermediate reasoning. At $2.50 per million input tokens, each agent run costs roughly $0.04-0.05. A chatbot handling the same question in one pass costs $0.003. That 10-15x cost difference is why many production systems deliberately keep their "agents" to two or three tool calls, blurring the line between a true agent and a chatbot with extras.

Most production "agents" are chatbots wearing a lanyard

The marketing around AI agents in 2026 has outpaced the reality. Gartner predicted in late 2025 that 33% of enterprise software would include agentic AI by 2028, but the definition of "agentic" in those predictions is generous. A customer support bot that looks up your order number and pastes a tracking link is being counted as an agent. The model made one tool call in a pre-defined sequence. That is a chatbot with API access.

The honest AI agent vs chatbot spectrum looks more like this. At one end, a pure chatbot: no tools, no memory, one request-response cycle. In the middle, a tool-augmented chatbot: one or two pre-defined tool calls, developer-controlled sequence, minimal reasoning. At the other end, a full agent: dynamic tool selection, multi-step reasoning, the model decides when it is done.

Most deployed systems sit in the middle. Calling them agents is optimistic. Calling them chatbots is dismissive. "Workflow with LLM-assisted steps" is more accurate, if less marketable.

Where this gets interesting is in platforms that build automation infrastructure around LLM calls. In Chase Agents, for example, each automation step defines an input schema and a set of available tools. The platform's action-type routing means a research step can return a purchase_recommendation object, but only the orchestrator step holds the tool access to execute the purchase. That architecture makes the boundary between "decide" and "act" explicit at the workflow level rather than burying it inside a single agent loop.

The AI agent vs chatbot decision comes down to one question

When someone asks "should we build an AI agent or a chatbot?" they are usually asking the wrong question. The right question: does the task require the model to decide what to do next based on the result of what it just did?

If a customer asks a question, the model looks up the answer, and the conversation ends, that is a chatbot with a tool. Build it as a chatbot. The engineering is simpler, the latency is lower, and the failure modes are predictable.

If a customer asks "rebook my canceled flight for the cheapest available option tomorrow" and the model needs to look up the cancellation, search alternatives, compare prices, check seat availability, and confirm the booking, that task requires a loop. Build it as an agent.

The 2025 State of AI Agents report from LangChain found that among teams running agents in production, the median number of tool calls per task was four. Not forty. Not four hundred. Four. The gap between a chatbot and an agent is real, but it is smaller than the marketing suggests. An agent with a four-step loop and a chatbot with two hardcoded tool calls differ by about 15 lines of code and a philosophical commitment to letting the model decide.

For automation workflows that connect multiple services, the better architecture often skips the "single all-knowing agent" pattern entirely. A sequence of specialized steps, each with its own LLM call and its own tool set, can be more reliable than one agent trying to do everything. Chase Agents takes this approach by default: each step validates its inputs against a typed schema, uses only the tools it needs, and passes structured output to the next step. The LLM does the reasoning within each step. The workflow does the orchestration between steps.

Where the line moves next

The AI agent vs chatbot distinction is already getting blurry, and it will get blurrier. OpenAI's Realtime API lets models maintain persistent connections and respond to events without explicit prompts. Anthropic's tool-use implementation in Claude allows the model to chain tool calls within a single API response. Google's Gemini 2.0 introduced "agentic capabilities" that include proactive tool use.

As models get better at multi-step reasoning within a single call, the explicit while True loop may disappear into the model itself. The distinction will shift from "does it have a loop?" to "how much autonomy does the workflow grant the model at each step?" That question has no clean binary answer. It is a slider, and every production system sets it somewhere different.

Build the simplest thing that solves the task. If that means a chatbot with one tool call, ship it. If that means a five-step agent loop, add the loop. The label matters less than whether the user's flight gets booked.