Article · Agentic AI Fundamentals
Chatbot or AI Agent: How to Tell Them Apart Before You Buy
A chatbot converses. An agent resolves the case. The word “agent” has become so commercial that almost any interface that talks now gets labeled one, and a good share of the “customer service agents” on the market are chatbots with a new name. The difference is not vocabulary but whether the system touches your data and does work, or merely holds a better conversation. This article gives you the criteria to tell them apart in a demo, before you sign.
In the pillar article on AI agents for business I sum the difference up in one sentence. Here I open it into three categories the market blurs, and into a concrete method for knowing what you are being sold.
Three things that share a name
It pays to separate three kinds of system, because each does something different with the case it receives.
The classic chatbot, scripts and intents. It runs on a decision tree and a catalog of predefined intents. It recognizes the user’s phrase, matches it to a known intent, and returns the answer prepared for that intent. Ask something the script did not anticipate and it gets lost or sends you back to the menu. It never leaves its playbook and touches no business system.
The LLM chatbot. It swaps the engine. Instead of an intent tree, it uses a language model that understands badly phrased questions, holds the thread, and writes natural answers. It converses far better and is far less embarrassing. But in most implementations it still does what the classic one did, answer. It can explain your return policy elegantly, but it does not process your return.
The agent. It takes the case and resolves it: it looks up the order in the system, checks whether it meets policy, triggers the return, generates the label, updates the record, and, if the case exceeds what it can do, escalates it to a person with the context already prepared. The conversation is only the surface, and underneath sit access to systems, encoded rules, and the ability to act. How that system works inside is in AI agents in operations.
At a glance:
| Classic chatbot | LLM chatbot | Agent | |
|---|---|---|---|
| Converses | Rigid, scripted | Fluent, natural | Fluent, natural |
| Touches your systems | No | Rarely | Yes, with permissions |
| Resolves the case | No, deflects | No, informs | Yes, or escalates with context |
| Leaves a trace of what it did | Little | Little | Yes, every action |
Why so many “agents” are rebranded chatbots
The reason is commercial and also technical. Commercial, because “agent” sells and “chatbot” sounds like 2018. Technical, because wiring a model to a conversation is relatively cheap today, while wiring it to your systems with permissions, rules and traces is the expensive, specific work, the part that actually encodes your operation. Many vendors do the cheap thing, call it an agent, and hope nobody looks underneath.
The result is predictable. You buy expecting the system to take work off your team, and what arrives is a very polished conversational search box that still deflects every case that matters to a person. That pattern, the demo that dazzles on the happy path and the system that cannot hold real operations, is the same one that sinks so many pilots, and I have analyzed it in why enterprise AI pilots fail.
How to spot it in a demo: three questions
When someone shows me a supposed agent, I do not argue about the quality of the conversation. I ask three questions, and the three answers tell me what I am being sold.
Does it touch my systems? Ask to see an action that reads from or writes to a real system: looking up an order status in the ERP, updating a ticket, generating a document. If all it does is talk about information you pasted into the prompt or it pulled from a knowledge base, it is a chatbot, however well it converses. An agent enters your platforms with concrete permissions, and what controls that demands is detailed in AI agent permissions and controls.
Does it resolve or deflect? Give it a full case, not a question. “I want to return this order.” A chatbot explains how to do it or opens a ticket for a person. An agent checks the policy, executes the return within its rules, and escalates only the cases that exceed its mandate. The share of cases it resolves end to end versus the ones it deflects is the number that truly separates one category from the other.
Does it leave a trace? Ask what gets recorded for each interaction: what it queried, which rule it applied, which action it took, why it escalated. A serious agent produces that trace because without it there is no audit, and a system that acts on your data without leaving a record is a governance problem, not a feature. How that evidence is reviewed is in auditing AI agent decisions.
The number that separates the two categories, and how it gets dressed up
The third question leaves a figure on the table, and that figure is the one to ask for in writing. Call it the end-to-end resolution rate: out of every hundred cases arriving through the channel, how many come out settled without a person stepping in. It looks simple and it is not, because the number changes completely depending on what goes into the denominator.
These are the four ways of inflating it I keep running into, all of them defensible on paper:
- Taking out of the denominator whatever the system does not understand. If out-of-perimeter cases do not count as cases, the rate measures how well it resolves what it already knew how to resolve. Ask for the number over everything that arrives through the channel, including what the system rejects on reading it.
- Counting as resolved the case that never came back. A customer not pushing again does not mean they were served. That is deflection, and it belongs in its own row.
- Counting as resolved what a person finished. If the system prepares the case and somebody else closes it, the person resolved it. Preparing it well is valuable, and it is measured in time saved per case, not in cases resolved.
- Measuring over the demo sample. The demo cases were chosen by the seller. Ask for the number over a real month of operation, seasonality included.
Measuring by resolution rather than deflection is what I argue for running the operation in AI in customer service, where the natural denominator is the cases the system accepts. Comparing bids needs the other one, the full denominator.
One more check closes the conversation: ask for the figure broken down by case type. A high overall rate can hide near-perfect resolution on the trivial types and near-zero on the type that is costing you money, which is exactly the one you wanted taken off your hands. How an evaluation set gets built and which other metrics go alongside this one is in AI agent evaluation. For this conversation one idea is enough: a provider who cannot give you the number broken down by case type is not measuring it.
What to buy for what you need
Not everything needs to be an agent. If your problem is that people cannot find information and swamp the team with repeated questions, a good LLM chatbot over your knowledge base solves a lot and costs little. Paying for an agent there would mean paying for capacity you will not use.
The agent is justified when the bottleneck is not informing but executing, when your team spends the day doing repetitive transactions on systems (processing, reconciling, updating, tracking) and you want that work done, not explained better. The neighboring boundary, the one between an agent and a copilot that assists your people instead of executing on its own, is in copilot or AI agent. And if you are comparing vendor proposals, the full buying criteria are in how to choose an AI agent provider.
Frequently Asked Questions
What is the difference between a chatbot and an AI agent?
The chatbot converses: it recognizes what you ask and returns an answer or deflects you to a person. The agent resolves the case. It queries your systems, applies your rules, executes the action, and escalates only what exceeds its mandate, leaving a trace of every step. One informs, the other works.
Is an LLM chatbot already an agent?
Not necessarily. Swapping the engine for a language model improves the conversation, but if the system still only answers and deflects, it is still a chatbot. It becomes an agent when it accesses your systems with permissions and executes work, not when it merely talks better.
How do I know if I am being sold a real agent?
Ask three questions in the demo: does it touch my systems with real permissions, does it resolve the case end to end or deflect it to a person, does it leave a trace of every action? If all three answers are not clear and demonstrable, it is probably a chatbot with an agent’s name.
Which number do I ask a provider for to tell an agent from a chatbot?
The end-to-end resolution rate: out of every hundred cases arriving through the channel, how many come out settled without a person stepping in. Ask for it over everything that arrives, not only over the cases the system accepts, and broken down by case type. If all you get is one overall figure, or the out-of-perimeter cases have vanished from the denominator, the number says nothing.
Is the resolution rate shown in the demo any use?
No, because the demo cases were chosen by the seller. What counts is the figure from a real month of operation, with its seasonality and its odd cases in it, and with the denominator written down. A short pilot on your own volume gives you that measurement before you sign anything.
Do I always need an agent for customer service?
No. If your problem is answering questions and locating information, an LLM chatbot over your knowledge base performs well and costs less. The agent is justified when the real work is processing and executing transactions on your systems, not just informing.