A practical checklist for evaluating any chatbot or automation vendor -- the concrete signs that separate a real, working system from a slide deck and a sales pitch. Use it on anyone you're considering, this site included.

---

"AI-powered" appears on more sales pages than it appears in working products. That's not necessarily dishonest -- a lot of it is genuine, if early, work -- but it means the phrase alone tells you nothing about whether what's being sold actually exists yet, or works the way the pitch implies. As a small business owner without an engineering team to vet the claim, you need a checklist that doesn't require you to know anything about AI. Here's one, and it applies to anyone you're evaluating -- including whoever you're reading this on.

Ask to See It Work on Something Real, Not a Demo Script

A canned demo proves someone can run a rehearsed script. It doesn't prove the system handles your actual questions, your actual edge cases, your actual messy real-world input. Ask to type something yourself, live, on the spot -- ideally something a little unusual, a little off the happy path. A vendor who's built a real system will let you drive. A vendor stalling, redirecting to "let's schedule a follow-up demo," or only showing pre-recorded footage is a real signal, not paranoia.

Ask What Happens When It Doesn't Know the Answer

This is the single best question in this list. A chatbot or agent that always has something to say is either connected to a much bigger, much riskier general-purpose model than your business needs, or it's going to eventually make something up and hand a customer a confidently wrong answer. The honest answer to "what happens when it doesn't know" is some version of "it says so, and hands off to a person." If the answer is vague, or sounds like it hasn't been considered, that's the tell.

Ask for a Real Timeline, Not a Range That's Really a Guess

"Two to four weeks" that turns out to mean "we haven't scoped it yet" is common enough to watch for. A vendor who's actually done this kind of build before can tell you roughly what's involved in getting from your first conversation to something live -- and can explain why it takes that long, not just quote a number. Ask what happens in week one specifically. A real answer sounds like a real answer.

Ask Who You're Actually Talking To

Is the person walking you through the demo the person who will build it, or a salesperson who hands you off to an implementation team the moment you sign? Neither is automatically dishonest -- plenty of legitimate agencies work this way -- but you should know which one you're getting, and factor the hand-off risk into your decision. Every hand-off is a place where context about your specific business gets lost or diluted.

Ask What a Real Client Would Say If You Called Them

Not a testimonial quote on the site -- an actual conversation. A vendor confident in their work will connect you with a past client, or at minimum point you at something live you can go inspect yourself right now, not a case study with the client's name redacted "for confidentiality." If every proof point is unverifiable by design, that's worth noticing.

The Pattern Behind All Five

Every one of these questions is really the same question asked five ways: can this be verified, right now, by me, without taking the vendor's word for it? A vendor with a real system wants you to verify it -- it's the fastest way to close the deal. A vendor without one needs you to trust the pitch instead, because the pitch is what they've actually got. Asking to see the thing itself, worked in front of you, on your own input, is the cheapest due diligence available -- and the one step a lot of buyers skip because the pitch was persuasive enough to feel like proof.