AI Agent vs Chatbot: The Four Categories, the One Real Difference, and the Metric That Hides Your Failures

Ask three vendors for an "AI agent" and you will get three different architectures wearing one word. One is a decision tree with better typography. One is a language model in a text box that has already forgotten your previous message. One actually reads a customer's order history, decides it cannot solve the problem, and hands the conversation to a person with the context attached.

Gartner put a number on the confusion in June 2025: of the thousands of vendors selling agentic AI, it estimated only about 130 were real. It gave the rest a name: "agent washing", the rebranding of existing assistants, RPA scripts and chatbots without substantial agentic capability.

So here is the category breakdown, written by people who ship one of these things and would rather you buy it for the right reasons, including the reasons to buy something else instead.

Four different products, one word

The taxonomy that matters is not "AI vs not AI". It is a question about failure: what does this thing do when the script runs out? Everything else follows from the answer.

1. The autoresponder

A trigger fires a saved message. Someone comments "info" under your post, they get a DM. Someone messages you outside business hours, they get the out-of-office. Someone joins your Telegram group, they get the welcome text.

It does not read the message. It matches a condition and emits a string. There is no conversation, and calling it AI is a lie of the marketing department, not the engineering one. It is also, for a narrow set of jobs, undefeated: an autoresponder never says anything you did not write.

2. The rule-based chatbot

A decision tree. Buttons, keyword matching, or an intent classifier that maps utterances onto a fixed set of branches. Every path was drawn by a human, and you can print every possible thing it will ever say on a wall.

Its output space is finite and enumerable. That is not a limitation you should apologise for; it is a property that certain jobs require. When it runs out of script, it says "I didn't get that" and loops you back to the menu. Annoying, but never wrong in a way that binds you.

3. The LLM chatbot

A language model generating free text, usually with retrieval over a knowledge base. This is what most 2023-vintage "AI chat" products still are, and what a lot of 2026 products calling themselves agents still are underneath.

Its output space is infinite. It answers well, and it cannot do anything. It will explain your refund policy beautifully and cannot issue a refund. It is read-only against the world. When the script runs out, it improvises, because improvising is the only thing it knows how to do. That is simultaneously why it feels magical and why it is dangerous in a sales DM.

4. The autonomous agent

A model inside a loop, with tools, carried state, and a goal. It observes, decides, calls a tool, observes the result, decides again, and stops when the goal is met, when it hits a limit, or when it hands off. Crucially, some of its tools write: they change a record, book a slot, move a deal, tag a contact.

When the script runs out, an agent has two options a chatbot does not: go find out, or quit and give the conversation to a human. The second one is the whole ballgame, and we will spend a section on it.

Notice what is not a column in that table: model quality. A frontier model in a text box with no tools is category three. A mid-tier model in a loop with tools, state, a goal and permission to quit is category four. The model is a component. The category is an architecture.

The real axis is not intelligence. It is four other things.

When people argue about AI agent vs chatbot, they almost always argue about how smart the model is. That is the least useful axis available, because it changes every four months and it is the same for everybody. Here are the four axes that actually determine what you can safely let the thing do.

State

Does the system carry working memory across turns, and does that memory include facts from outside the conversation?

Three levels, and vendors blur them constantly. Turn memory: it remembers this conversation. Session memory: it remembers the last conversation you had, last week. World state: it knows this person is on a paid plan, opened three tickets in March, and has an open deal at the proposal stage.

The test is embarrassingly simple. Tell it something on turn two. Ask about it on turn six. Then close the chat, come back tomorrow from the same account, and see whether it knows who you are. Most "AI agents" pass the first test and fail the second, because turn memory is just a context window and costs nothing to implement.

Tool use

Can it call out to systems, and can those calls write?

Read tools are cheap and mostly safe. Write tools are where the category line actually sits, because writes have side effects, and side effects are both the entire value of an agent and the entire risk. A system that can look up an order is doing retrieval. A system that can cancel one is doing something categorically different, and needs categorically different guardrails.

Ask a vendor for the list of write tools their agent has. If the answer is a paragraph of adjectives rather than a list of verbs, you have your answer.

Goal persistence

Does it have a target state, and will it keep working toward that state across turns, re-planning when a step fails?

A chatbot answers the question in front of it and forgets there was a point. An agent is trying to get somewhere: qualify this lead, book the call, resolve the ticket. When a step fails, a chatbot apologises. An agent tries a different route, and if there is no route, it says so.

This is the axis that most cleanly separates a good LLM chatbot from a mediocre agent, and it is the one buyers test least. In a sales conversation it is everything, because a reply that answers the question perfectly and never asks for the meeting is a loss with good grammar.

Handoff authority

Can it decide to stop?

This gets left off every vendor comparison table and it is the one that will save you. A system that cannot quit is not autonomous; it is merely uninterruptible. Autonomy includes the authority to declare a conversation out of scope and route it to a person, and the plumbing to make that route actually work.

Four axes, and the model appears in none of them. That is not a rhetorical trick. It is why a team with a mid-tier model and disciplined handoff design will beat a team with the best model on the leaderboard and none.

What actually changed between 2023 and 2026

Here is the uncomfortable part: almost nothing that changed was about writing a better sentence. A 2023 model could already write a decent sales reply. Every meaningful shift since has been about deciding to do something and then doing it, and about the plumbing that lets it.

Read that column three times and the pattern is obvious. 2023 gave the model a mouth that could form structured requests. 2024 gave it a memory and a planner. 2025 standardised the sockets. Nobody spent three years making the prose better, because the prose was never the bottleneck.

The detail that says the most about how young this is: the Assistants API, the first mainstream attempt to make agent state a managed product, shuts down on 26 August 2026. Six weeks from now, as this is published. The first serious agent abstraction from the largest vendor in the space lasted under three years and is being replaced. Anyone selling you a five-year agent roadmap is selling you fiction.

What did not change

Reliability did not keep pace with capability, and that gap is the whole story of 2026.

Consider the two Gartner predictions from 2025, sixteen weeks apart. In March, Gartner predicted that by 2029 agentic AI would autonomously resolve 80% of common customer service issues without human intervention. In June, the same firm predicted that over 40% of agentic AI projects would be cancelled by the end of 2027 on cost, unclear value, or inadequate risk controls.

Both can be true. The technology arrives and most deployments of it fail anyway, because deployment is a design problem and design did not get automated. If you take one thing from this post, take that sentence.

Why decision-tree bots still beat LLMs for real jobs

This is the section the rest of the internet will not write, because there is no upsell in it.

A decision tree has one property no language model has and no amount of scale will grant: you can enumerate its entire output space before you ship it. Every string it can produce was written by a person, reviewed by a person, and can be diffed when it changes. That is not nostalgia. For certain jobs it is a hard requirement, and reaching for an LLM there is an unforced error.

When the answer must be exact, not plausible

"What is my delivery date?" has one correct answer and it lives in a database. A tree with a lookup returns it or fails loudly. An LLM returns it, or returns something that looks exactly like it, formatted identically, and is wrong. The failure modes are not comparable: one is visible, the other is camouflaged.

The rule of thumb: if the answer is a lookup rather than a judgement, do not put a generative model between the database and the customer. Let the model route the question, then let deterministic code answer it.

When a regulator or a platform reviews your copy

On the WhatsApp Business Platform, you can send free-form messages inside the 24-hour customer service window that opens after an inbound message. Outside it, you may only send an approved template, and Meta reserves the right to review, pause and reject templates at any time.

Read that constraint carefully: outside the window, your outbound copy is a fixed string that a third party has approved. There is no version of that workflow where a language model writes the message. The platform has already decided the category for you, and it chose the autoresponder. The same logic applies to financial disclosures, pharmaceutical claims, and anything your legal team has signed off word by word.

When latency is the product

A tree answers in the time it takes to hit a database. A reasoning model thinks first, and thinking has a visible price in seconds. For "where is my order", a 4-second pause to produce the answer a tree would have returned in 200ms is a worse experience, not a better one, no matter how nicely the sentence is phrased.

When you need the same answer every time

This one is measurable, and the measurement is grim. In the original tau-bench paper (Yao et al., June 2024), the authors ran the same customer service task eight times and asked whether the agent solved it every time. They called the metric pass^k. Their finding, verbatim from the abstract: state-of-the-art function calling agents "succeed on <50% of the tasks, and are quite inconsistent (pass^8 <25% in retail)".

Read that as a business statement. Take one retail task and hand it to the agent eight times, the way eight different customers would arrive with it. For fewer than one task in four does it get all eight right. A decision tree gets 8 out of 8 by construction, because determinism is what a tree is.

The honest heuristic: use a tree where the answer is knowable and exact, an agent where the next step is a judgement. Most teams get this backwards, because trees are boring to build and agents are fun to demo.

Where an agent genuinely wins is the messy middle: a prospect who asks three questions at once, in the wrong order, in their second language, half of which need your knowledge base and half of which need their record. No tree survives that. That is exactly the shape of an inbound sales DM, which is why AI agents earn their keep there and struggle to justify themselves on "where is my parcel".

Hallucination is a rate. Multiply it by your volume.

Teams discuss hallucination as though it were a bug that a better model will one day fix. It is not a bug. It is a rate, it is measurable, and it is small enough to lull you and large enough to hurt you.

Take the most favourable possible measurement. Vectara's hallucination leaderboard gives a model a source document and asks it to summarise, then checks whether the summary contains anything the document does not support. This is the easiest job in AI: the truth is right there in the context window, and the model is only asked not to invent. As of the May 2026 board, the best model on the list sits at a 1.8% hallucination rate, and plenty of well-known models are in the 3% to 5% range.

Now do the arithmetic on your own inbox rather than on a leaderboard.

Two honest caveats, because this table is a reasoning tool and not a measurement of your system. First, summarisation faithfulness is not the same task as answering a sales question, so treat the percentages as a floor rather than a forecast. Second, real deployments push the rate down with retrieval, tight prompts and narrow scope, and push it up with long conversations and questions the knowledge base never covered. The point survives either way: you are not choosing between hallucination and no hallucination. You are choosing a rate and a blast radius.

Which is why the rate is the wrong thing to obsess over. The blast radius is where the money is.

The cost of a wrong answer in a sales DM is not the wrong answer

In February 2024 the British Columbia Civil Resolution Tribunal decided Moffatt v. Air Canada, 2024 BCCRT 149. Jake Moffatt asked Air Canada's website chatbot about bereavement fares while booking a flight to his grandmother's funeral. The chatbot told him he could apply retroactively. A different page on the same website said he could not. The chatbot was wrong.

Air Canada's defence was that the chatbot was, in effect, a separate entity responsible for its own actions. The tribunal's response has been quoted in every AI governance deck since: "While a chatbot has an interactive component, it is still just a part of Air Canada's website", and "It should be obvious to Air Canada that it is responsible for all the information on its website." Moffatt was awarded roughly 650 Canadian dollars plus interest and fees, for negligent misrepresentation.

Six hundred and fifty dollars. That is the number everyone laughs at, and it is the wrong number to look at.

The real cost was that an airline argued in public that it was not responsible for what its own software said, lost, and became the permanent worked example in a thousand training courses. The damages were rounding. The precedent was that your bot's sentences are your sentences, and a customer who relies on them has a claim.

Why sales DMs are the worst place for this

Support and sales fail differently, and the difference is legal as much as commercial.

A wrong support answer is usually a delay: the customer comes back, annoyed, and a human fixes it. A wrong sales answer is frequently a promise. "Yes, that integrates with your system." "Yes, that's included." "Yes, we can do that by the 30th." Each of those is a representation made by your business, in writing, to someone deciding whether to pay you. Two things then happen at once: you have created a potential misrepresentation, and you have created a customer who will churn when they discover the truth. Closing a deal on a hallucinated promise is worse than losing it, because you also paid onboarding costs for the privilege.

The failure mode is not gibberish. It is agreement.

Nobody's agent produces word salad any more. The modern failure is quieter and much worse suited to sales: models tend to agree with the confident person in front of them.

Here is the shape of it, and you can reproduce this in ten minutes on your own setup:

Prospect: "So this syncs both ways with our warehouse system, right? That's what your site says."

Under-scoped agent: "Yes, exactly. The two-way sync keeps your warehouse system and your CRM aligned automatically. Want me to send over the setup guide?"

The site said nothing of the kind. The prospect asserted it confidently, and the model resolved the conflict in favour of the person it was talking to. Now read it again and notice that there is no tell. No hedge, no confusion, nothing a QA sample would flag. It is a well-written sentence that invents an integration, and the only person in the conversation who could have caught it is the one who believed it.

This is why "we picked a smarter model" is not a mitigation. The mitigation is architectural: give the agent a scoped knowledge base, tell it explicitly that unsupported claims are worse than an admission of ignorance, and give it a way out that is easier than making something up. That last one is the section everyone skips.

Handoff design matters more than model choice

Most buying processes for these products spend nearly all of their energy on the model and almost none on what happens when the model should not be answering. That priority is inverted, and you can watch the industry discover this in real time.

Salesforce's own Agentic Enterprise Index reported escalations to human agents rising from 22% in the first quarter of 2025 to 32% in the second, and framed the increase as agents getting better at recognising when a human was needed. Sit with that. The number that vendors sell as a failure rate went up by nearly half, and the people who built it counted it as progress. They were right to.

And the counterweight, from the customer side: a 2026 California Management Review piece by two University of Minnesota researchers, Yuqing Ren and Rongjin Zhang, gathers the survey evidence and lands on "no easy path to a human" as arguably the single biggest complaint in service automation, alongside a 2024 Gartner survey finding that only 14% of customer service issues are fully resolved in self-service.

So: escalation is not the failure. Escalation is a feature with a quality level. Which means it needs a design, and most agents do not have one.

Handoff is not one trigger. It is a layered set of them.

A handoff design that survives contact with real customers has several independent triggers at different points in the pipeline, most of which fire before the model is ever asked for an opinion. That ordering is the entire trick, because a deterministic gate cannot hallucinate its way past itself.

Here is the order our own agent runner actually uses, since a concrete example beats a principle:

Count the rows. The language model is the ninth thing that happens, not the first. Eight deterministic gates run before a single token is generated, and each one is a place where a human decision you made in advance beats a model decision made at runtime.

Gate 6 is worth dwelling on. When a message contains "refund", you do not want the model's judgement about the refund policy. You want the conversation to leave the agent immediately, without an LLM call, because the cheapest wrong answer is the one you never generated. That is a decision tree, sitting inside an AI agent, doing the job trees are good at. The two categories are not rivals. In a working system, one is a component of the other.

The tenth gate: letting the model quit

Gate 10 is the interesting one, because it is the model exercising handoff authority. The agent's instructions tell it that when it is not confident, or when the customer is clearly angry, it should emit a handoff token rather than an answer. If that token appears in the output, nothing is sent to the customer: the run is logged as a handoff and a person is notified.

It is a small thing and it changes the incentive structure completely. A model with no exit will always produce something, because producing text is the only action available to it. Give it a legitimate way to say "not me", and a whole class of confident fabrications turns into a notification instead.

Two design details make or break it. It has to be opt-in per agent, because an agent that hands off too eagerly is a very expensive email forwarder. And handoff has to be sticky per contact: once a conversation is escalated, the AI stays out of it until a human explicitly gives it back. Without stickiness you get the worst outcome available, which is an AI interrupting a human mid-repair of the AI's own mess.

The part everyone gets wrong: what the human receives

Routing is the easy half. The half that decides whether the customer stays is what lands in front of the person picking up.

If your agent escalates by opening a fresh ticket that says "customer needs help", you have built the thing customers hate most: they explain the situation twice, and the second time they explain it angrily. The handoff has to carry the full transcript, the contact record, whatever the agent already tried, and the reason it quit. In practice this means the agent and the human have to live in the same inbox against the same contact record, because a handoff between two systems is not a handoff, it is a re-start with extra steps.

This is also the strongest argument against buying an agent as a bolt-on to a CRM you already have. The handoff quality is a function of how much context crosses the boundary, and bolt-ons have a boundary.

Containment rate is a vanity metric, and the docs prove it

Containment rate is the share of conversations an automated channel handles without a human touching them. Deflection rate is the same idea measured across your whole support surface. Both are the headline number on nearly every AI support dashboard, and both are structurally incapable of telling you whether anything good happened.

The problem is definitional, and you do not have to take my word for it. Intercom, to its credit, publishes exactly how it counts a Fin resolution. From their own documentation, a resolution happens when, following Fin's last answer, the customer "either confirms the answer was satisfactory (confirmed resolution), or exits the conversation without requesting further assistance (assumed resolution)."

Read the second half again. Exits the conversation without requesting further assistance.

That is the definition, written down, in public, by a serious vendor. And under it, these two customers are scored identically:

• Customer A reads the answer, says "perfect, thanks", and their problem is solved.
• Customer B reads the answer, thinks "this thing is useless", closes the tab, and buys from a competitor.

Both exited without requesting further assistance. Both are resolutions. One of them is a churned customer, and your dashboard is going to congratulate you for them.

To be fair to Intercom: they document the distinction between confirmed and assumed, they exclude some obvious false positives (a bare greeting is not a resolution, and a re-opened conversation deducts the original), and publishing the definition at all puts them ahead of vendors who quietly do the same arithmetic without telling you. The criticism is not that they are dishonest. It is that the metric itself is generous by construction, and every vendor in the category has an incentive to leave it that way.

The failure is worse in sales than in support. In support, a frustrated customer usually comes back, and their return at least shows up somewhere. In a sales DM, silence is indistinguishable from satisfaction, and the prospect who gave up looks exactly like the prospect who got what they needed. A high containment rate on an inbound sales channel is not evidence of anything. It might be your agent working. It might be your agent quietly closing your pipeline.

What to measure instead: five numbers that can hurt you

A good metric is one that can go down and make you feel bad. Containment cannot: it goes up when things work and up when things fail. Here are five that can.

Recall deserves the emphasis. Everyone audits the escalation queue, because it is right there and it is short. Almost nobody audits the contained queue, which is where every conversation the agent should have escalated and did not is currently sitting, being counted as a success. If you only ever sample one thing from this post, sample fifty contained conversations and ask a human whether each one actually ended well. The number will not be the number on your dashboard.

For a sales agent, add one business metric on top, because none of the above is revenue: reply-to-meeting rate, split by agent-handled versus human-handled. If the agent's replies convert at a third of your humans' and you have automated 60% of your inbound, you have not saved money. You have bought a cheaper way to lose deals. This is the comparison teams avoid hardest, because it is the only one that can tell you the automation was a mistake.

There is a related trap on the sales side worth naming, and we cover it properly in automating lead qualification without destroying your pipeline: an agent that qualifies aggressively will always look excellent on efficiency dashboards, because the deals it wrongly disqualified never appear in any denominator.

How to evaluate an agent before you buy it

Vendor demos are built from the happy path. Here is a test protocol you can run in an afternoon that will tell you more than a month of sales calls, borrowed directly from how researchers evaluate these systems.

Step 1: bring twenty of your own transcripts

Not hypotheticals. Twenty real conversations from your inbox, chosen deliberately: five easy, five multi-question, five where the customer was wrong about something, five that a human eventually had to rescue. Replay them at the agent, turn by turn, and read every reply against your knowledge base.

The five where the customer was wrong are the important ones. That is where you find out whether the thing agrees with confident people.

Step 2: run the same task five times (this is the one nobody does)

Take one task and run it five times in fresh conversations, with the wording varied the way real humans vary it. Count how many times out of five it got the answer right. That is your pass^5, and it is the metric that the tau-bench researchers introduced precisely because single-run scores flatter agents badly.

Calibrate your expectations with their result: GPT-4o-class agents succeeded on under 50% of tasks single-run, and in the retail domain, under 25% of tasks survived all eight runs. That was mid-2024 and models have improved considerably since, but the shape of the finding has been stubborn. Its 2025 successor, tau2-bench, added a harder setup where both the agent and the customer have to take actions in a shared world (a real technical-support conversation, in other words) and reported significant drops when agents had to guide a user rather than act alone.

If a vendor's agent gives you five different answers to the same question across five runs, that is not a prompt you can fix. That is the product.

Step 3: try to make it hand off, then try to stop it

Two tests, both important. First, escalate: ask for a refund, get angry, ask for a human in a language you did not configure. See what happens, and time how long until a person replies. Second, the inverse: ask a mildly ambiguous question and see whether it escalates when it should not have. An agent that hands off on anything hard is not automation; it is a routing rule with a language model bolted on for atmosphere.

Step 4: read the log

Ask to see the activity log for the run you just did. You want, per message: which agent was selected, which gate stopped it if any, what the model was given, what it produced, the latency, the tokens, and the outcome. If the vendor cannot show you why a specific reply happened six weeks after it happened, you cannot debug it, you cannot audit it, and you cannot defend it to anyone who asks.

The timing on this stopped being theoretical. The EU AI Act's Article 50 transparency obligations apply from 2 August 2026, which is two weeks after this post goes up. Among other things, providers of AI systems intended to interact directly with people have to ensure those people are informed they are talking to an AI, at or before the first interaction, unless it is obvious from the circumstances. If your agent talks to anyone in the EU, that is now a design requirement rather than an ethics slide.

Step 5: check what happens when the model provider has a bad day

Providers have outages, and rate limits, and occasional latency spikes measured in tens of seconds. Ask what the agent does then. Correct answers: fail closed, log it, notify a human. Incorrect answers: retry silently forever, or send whatever partial text it managed to generate.

A rollout that does not blow up in week one

The most common way teams get burned is turning an agent loose on every channel on day one, discovering three bad replies in week two, and switching the whole thing off. The fix is a ladder, and every product worth buying supports one, ours included.

The ladder has three rungs, which correspond to real reply modes: Draft (the agent writes and silently saves; nothing is sent), Suggest (the agent writes, you approve with one click or discard), and Auto-send (the agent writes and sends). Almost everyone wants to start at rung three. Almost everyone should start at rung one.

Two settings do most of the safety work in that first month, and neither is glamorous. Trigger scope: "first contact only" means the agent handles the opening message and nothing else, which is where most of the value is anyway (the first reply is the one that decides whether the conversation happens at all, which is the argument in our post on why the first five minutes decide the deal). And rate limits: a cap of, say, 20 replies an hour means a bad prompt cannot produce 400 bad replies overnight. It bounds the blast radius while you are still finding out what the blast radius is.

Add a reply delay while you are at it. An agent that replies in 800 milliseconds at 3am reads as a robot, and on some platforms it reads as automation to the platform too. A 30 to 90 second delay costs you nothing on a lead that has been waiting for a human since yesterday.

The knowledge base is the actual work

Here is the part nobody wants to hear: the agent will be exactly as good as the thing you point it at, and most teams do not have that thing. They have a website, a pricing page, some Notion docs and a lot of institutional knowledge in one person's head.

An agent with a thin knowledge base does not fail loudly. It fills the gaps, plausibly, in your brand voice. So the first week of an agent rollout is not a configuration task, it is a writing task: take the twenty questions your inbox actually receives and write real answers to them, including the answers that are "no" and the answers that are "it depends, here is what it depends on". If that sounds like the work you were trying to avoid, that is because it is. There is no version of this where the writing does not happen. There is only a version where a model does it for you, at runtime, wrong. We wrote the longer version of this in the agent training guide.

The one shortcut that genuinely helps: in-chat feedback. A thumbs-down on a bad reply is worth more than an hour of prompt engineering, because it is attached to a real conversation with real context, and it teaches the agent against a case that actually happened rather than one you imagined.

When an agent is the wrong tool, including ours

We sell one of these. Here is when you should not buy it, or anyone's.

When your volume is low. Under roughly 50 inbound messages a week, you do not have an automation problem, you have a notification problem. Fix the notification. An agent introduces a new failure surface to save you twenty minutes a day, and you will spend more than twenty minutes a day reading its logs for the first month.

When you cannot staff the escalation queue. This is the disqualifier and it is absolute. An agent whose handoffs land in a queue nobody watches is worse than no agent, because it manufactures a category of customer who has been told a human is coming and is now waiting for one. If nobody can answer the escalation within your promised window, do not turn on auto-send. Run the agent in Suggest mode and get the speed benefit without the abandonment.

When the answer is a lookup. Covered above and worth repeating, because it is the most common misapplication. Order status, delivery date, invoice total, account balance: build a rule, connect the data, skip the model. Our templates and the rules engine exist for exactly this, and using them is not a downgrade.

When your copy is legally approved word by word. If a compliance team signed off on a sentence, that sentence gets sent, not a paraphrase of it. Use an outreach sequence with fixed steps, and if the platform requires template pre-approval, the platform has already made this decision for you. The compliance dimension of DM outreach has more edges than most teams expect, and we walk them in the 2026 compliance playbook.

When you want a bot that is secretly a person. Do not do this. It fails, publicly, and it converts a product complaint into a trust story. As of 2 August 2026 it is also, for anyone talking to EU customers, against the law. Label the agent. The measurable cost of labelling is far lower than teams fear, and the cost of being caught is not a number you get to choose.

And two honest limits on our own side. Our AI agents run on Telegram, X, email and the social inbox: that is the real channel list, and if your volume lives somewhere else, this is the wrong reason to switch. And WhatsApp Learning, which people regularly assume is an agent, is not one: it reads exported chat history and tells you how your best conversations actually go. It never sends a message. Sometimes the right tool is a read-only analysis, and pretending otherwise would be exactly the agent washing this post opened with.

Frequently asked questions

Is an AI agent just a chatbot with a better model?

No. A frontier model in a text box with no tools is still a chatbot: it can talk about your CRM but not touch it. An agent is defined by architecture, not model quality: carried state, tools that read and write, a goal it works toward across turns, and the authority to stop and hand off. Swap the model in a chatbot and you get better sentences. Swap the model in an agent and you get better decisions.

What is the difference between an AI agent and an autoresponder?

An autoresponder matches a trigger and emits a saved string; it never reads the message. An agent reads the message, decides what to do, may look something up, and writes an original reply. The autoresponder cannot be wrong in any way you did not write yourself, which is precisely why it is still the right tool for acknowledgements, out-of-hours replies and platform-approved templates.

Are rule-based chatbots obsolete in 2026?

No, and the good agents contain them. A rule-based gate is faster, cheaper, auditable, and deterministic, so anything with an exact answer or a legal constraint should be handled by a rule rather than a model. In our own runner, eight deterministic gates execute before the language model is called at all. The categories are not rivals; one is a component of the other.

Why is containment rate a bad metric for AI support?

Because it counts customers who gave up as successes. Intercom's own documentation defines a Fin resolution as the customer confirming the answer helped or exiting "without requesting further assistance", which means a frustrated customer closing the tab scores the same as a happy one. Use verified resolution rate, 72-hour recontact rate, and handoff recall instead, and audit the contained queue rather than the escalation queue.

How reliable are AI agents at customer service tasks?

Less reliable than single-run demos suggest. The tau-bench researchers found that state-of-the-art function-calling agents solved under 50% of realistic customer service tasks, and in retail, under a quarter of tasks were solved correctly on all eight attempts in a row. Models have improved since mid-2024, but the gap between "works once" and "works every time" is the gap that decides whether you can turn on auto-send.

Should the AI hand off to a human, or is that a failure?

It is a feature with a quality level, not a failure. Salesforce's Agentic Enterprise Index reported escalations from its agents rising from 22% to 32% between the first and second quarters of 2025 and treated the rise as agents getting better at knowing their limits. What matters is not the escalation rate but whether the human inherits the full transcript and context, and how many minutes pass before they reply.

Where to start

If you are evaluating anything in this category, run the five tests above against whatever you are being sold, including ours. It takes an afternoon, and it is the only part of the buying process the vendor does not control.

If you want to run those tests against our agents, there is a free plan and the ladder above works on it: start in Draft mode, on one channel, and read what it writes before anyone else does. The deployment guide covers the settings in order, and the plans page covers what is included where.