You already run a support chatbot, and it’s not doing the job. Resolution rates have plateaued, customers ask for a human within two exchanges, and your CSAT score hasn’t moved.
So you’re looking at agentic AI as the upgrade, and a dozen vendors are telling you their agent will fix everything the chatbot couldn’t.
It’s a hard buying decision, because most of these projects fail: Gartner expects over 40% of agentic AI projects to be cancelled by the end of 2027, citing rising costs, unclear value, and weak risk controls. The demo agent almost always works, until it’s time for deployment.
This guide gives you five questions to put to any vendor before you sign. It’s a condensed version of the full SCOPE framework, tuned for one job: choosing an agentic AI vendor for enterprise customer support when you own the CSAT, NPS, and CX numbers.
Pin down what the customer support agent does
Most failed support deployments fail at the same point, because nobody pinned down what the agent was for. The vendor pitched a system that “handles your entire customer service operation,” you signed, and six months later you’re chasing them because the agent does one workflow well and hallucinates on the rest.
There’s a principle behind this, drawn from AI philosophy: breadth and reliability pull against each other, so an agent can’t maximise both at once.
Picture a cardiologist who has spent twenty years on one organ system, where deep skill in one place means thinner skill everywhere else.
A CX agent built around one defined task, like resolving billing disputes under a set value, can reach high reliability on it, while an agent pitched as handling everything spreads its certainty too thin to hold anywhere.
For you, that thin certainty is the NPS score. A broad CX agent that runs at 60% accuracy means four in ten customers meet a wrong answer or a dead end, and those are the interactions that push your detractor count up.
The narrow agent is the better CX investment, because you can hold it to a standard you can measure.
The narrower the task, the more an agent behaves like a tool, returning a predictable result you can hold to account. Many vendors use tool and agent interchangeably, so the contract should state which one you’re buying.
Three questions to ask before the demo:
- What single task is this agent optimised for, and what does it deliberately leave alone?
- When a request falls outside that task, does the agent escalate to a human, decline, or attempt it anyway? That third behaviour is what produces wrong answers and runaway cost.
- How does accuracy change as the task widens? Ask for the number on the narrow core task and on the broad version, because the distance between those two figures is what you’re buying.
Match vendor claims to your KPIs
You own CSAT, NPS, operational efficiency, and customer lifetime value, and each one needs a different kind of proof from a vendor. Most vendor decks blur them together under a single “resolution rate,” which tells you almost nothing about the metric your board asks about.
Verify how the vendor defines success and how they measure it. “Resolved” can mean the agent marked the ticket resolved, or it can mean the customer’s problem was solved to their satisfaction, measured by a CSAT score above a threshold, and those are two different products at one price.
Pin the definition down in writing, with the measurement method attached, before you sign.
| Your KPI | What the vendor usually claims | What you need to verify it |
| CSAT | A high resolution rate | Post-interaction CSAT tied to agent-handled tickets, not ticket-close counts |
| NPS | Faster responses | Failure modes and escalation quality, since a confident wrong answer drives detractors |
| CX efficiency | Lower average handle time | Latency distributions rather than averages—customers bounce in the long tail |
| CLV | A case study from another sector | An attribution window written into the contract, on your data and your customer base |
A vendor confident in their numbers will hand these over without friction, because the same reporting discipline produced the product.
Ask compliance questions before the demo
Perhaps you work in finance, government, hospitality, or professional services, which means regulatory compliance is a precondition. Get the compliance questions answered before the evaluation goes any further.
An agent processing personal data falls under GDPR. An agent that makes or materially influences decisions about people can come within the scope of the EU AI Act if you operate in or serve the EU.
Sector regulators add their own requirements on top, and every market you scale into brings its own. Ask the vendor to state, in writing, which regimes they’ve designed for and which they leave to you.
Three things belong in that written answer: where customer data resides and how it’s processed; what the audit trail captures when the agent acts; and who carries the consequence when the agent gets a regulated decision wrong.
Your wider enterprise AI governance posture does the heavy lifting here, and the discipline behind a structured audit carries straight over to checking a vendor’s compliance claims. Certification against ISO 42001 tells you a vendor has built to a recognised standard.
Understand the CX agent pricing model before it scales
Pilot purgatory costs have a root cause: a pilot runs on a contained, predictable workload, so the cost looks manageable, but production doesn’t behave that way, so the bill that looked fine at 500 tickets a month can run out of control at 40,000.
You can’t manage a cost you can’t see. Ask your vendor to show you a live consumption dashboard before signing anything.
Agentic AI pricing has three layers:
- Compute cost, the tokens the agent reads and writes. A single interaction runs a few pence, but across enterprise volume those costs reach five and six figures, and they don’t rise in a straight line, because harder tasks make the agent reason more, and reasoning burns tokens faster than ticket volume alone.
- Runtime behaviour cost, the layer that surprises finance teams. Because the agent decides its own path at runtime, the same task can cost wildly different amounts on different runs, and a vague query like “can I get that thing from last week” forces the agent to reconstruct context, pulling transcripts, ranking them, and retrieving documents before it answers.
- Operational cost, which falls outside the model: the cloud bill, the human review the agent triggers, and the compliance overhead the deployment creates.
The one question to put to any usage-based or hybrid contract: what visibility and control do we get? A responsible vendor provides real-time consumption dashboards, spend forecasting, alerts before you reach plan limits, and soft caps with grace periods for short surges.
Where those tools are missing, you carry an uncapped liability, so put their presence on the checklist alongside the headline price.
Push back on “fast go-live, low risk”
We all want to go live quickly with little to no risk. But speed and low risk pull in opposite directions at launch (remember Floridi), because the fastest way to go live is to give the agent broad autonomy on day one, and that’s also the riskiest.
A responsible go-live commitment looks like a phased rollout. The agent starts with a narrow autonomous scope and clear escalation paths, a human reviews the decisions that carry cost or compliance weight, and the autonomous scope widens as the performance data comes in.
Human-in-the-loop design belongs in this conversation as much as the implementation one, so ask where the checkpoints are, what triggers them, and who owns the decision when the agent steps back.
A vendor who promises a fast launch with no risk and no phasing is selling you the demo. Press for the rollout plan, the baselines they’ll hold themselves to, and the point at which they’d recommend widening the agent’s remit.
Five questions for the vendor call
Take these five questions into the next vendor call. They run in under an hour and cover the dimensions where support deployments fail.
| The question | A good answer sounds like | A failing answer sounds like |
| What single task is this optimised for, and what does it exclude? | A defined task boundary with named exclusions | “It handles everything” |
| What’s the full cost stack, and what live visibility do we get? | Three cost layers and a real-time dashboard | A token quote and no dashboard |
| How is success defined, attributed, and verified, in writing? | A CSAT threshold with an attribution window | “Resolved means resolved” |
| Can you prove it at our scale, on our data, with failure modes? | Accuracy on a workload like yours, plus latency distributions | A single clean demo run |
| Who carries the financial, operational, reputational, and regulatory consequence? | Named liability terms and the regimes they’ve built for | Silence on liability and compliance |
A vendor building reliable agents answers all five without hesitation, because those same questions guided how they built the product. Where a vendor struggles, that hesitation is your data.
Before you shortlist a vendor for CX agentic AI
One more piece of context for the shortlist: Gartner estimates that only about 130 of the thousands of agentic AI vendors on the market offer real agentic capability, with the rest rebranding older chatbots and automation as agents.
The five questions above are built to surface that difference, because an “agent washed” chatbot can’t answer them.
For the full framework behind these questions, the SCOPE guide covers all five dimensions in depth, with an interactive tool for weighting them to your own risk profile.
If you’d rather run a live vendor evaluation with a second set of eyes, I offer a guided SCOPE assessment that maps each dimension against specific vendor proposals and contract terms. Book a free 30-minute call.