Testing Voice Agents Before You Put Them on a Client's Line

Every voice agent I have shipped has worked perfectly in the builder and then done something stupid on the first real call. That is not a criticism of the tools. It is the nature of the thing. A voice agent is a stack of independent systems (a carrier, a number, a media path, a model, a set of tools, a downstream CRM) and the builder only exercises the middle of it.

The single most useful sentence I can give you about testing voice agents is this: verify the wiring, not the logic. Your prompt is probably fine. What breaks is the connections between things.

Why the simulator is not enough

Every platform ships a simulator or web test widget. Use it, it is genuinely useful for iterating on the prompt fast. But understand what it does not cover.

It cannot inject system variables like caller ID. This is the big one. The simulator hands your agent a clean, populated context. A real inbound call may hand it something different, or nothing at all. I have had agents that behaved perfectly in the simulator and produced total silence on live calls, because the live path passed empty dynamic variables and the simulator did not. That failure looked exactly like a carrier problem and was not. The full story is in debugging dead air.

It does not test the phone path. Forwarding rules, carrier features that answer calls before you see them, voicemail races, caller ID substitution. None of that exists in a simulator. See phone forwarding for AI agents.

It does not test audio behavior. Barge-in, background noise, someone on a car speakerphone, a caller who talks over the greeting, DTMF from a caller punching a menu out of habit. Text simulators skip all of it. Even voice-enabled web widgets use a clean laptop mic on a good connection, which is not what your callers have.

It does not test the real transfer. A simulator will happily show a transfer step succeeding. The actual transfer might be rejected because the tool is not referencing the live call's control ID, and you will not find out until a real caller is stranded.

So the rule I work by: the simulator is for iterating on what the agent says. Real calls are for verifying that it works.

Test-call your own cell, every time

Before any rollout, I call the agent from my own phone. Actual cell, actual carrier, actual network. This is not optional and it is not a formality. It is the only test that runs the entire path a customer runs.

Same for SMS. If the agent texts, text it from your own phone and read the message on the phone screen, not in a log. Link formatting, message splitting, and opt-out language all look different on a real handset.

Do this from a number that is not on the client's account and not in any CRM, so you get the cold-caller experience rather than the recognized-customer one. Then do it again from a number that is in the CRM, so you test the recognized path too.

The pre-rollout checklist

Here is what I actually run through. It takes maybe twenty minutes and it has caught something real on most deployments.

Config checks (no calls needed)

1. Search the deployed instructions for {{. Every remaining token must correspond to a variable the platform actually populates. Unsubstituted placeholders silently break the prompt.
2. Confirm every dynamic variable has a sane default, so an empty payload does not produce silence.
3. Confirm a fallback model is configured. No fallback means dead air the day the primary model has an outage.
4. Open the transfer tool and confirm the call identifier is a runtime reference, not a hardcoded value left over from testing.
5. Confirm the number is pointed at the agent you think it is, and not at last week's clone.

Call checks (real phone)

6. Call and let the greeting play out. Time it. Anything over about two seconds of silence before the first word is a problem.
7. Call and interrupt the greeting immediately. Confirm barge-in works and the agent does not talk over you for ten seconds.
8. Call and say nothing at all. A silent caller should get a prompt, then a graceful exit, not an infinite wait.
9. Call and ask the three questions the business gets most. Then ask one thing the business does not do at all, and confirm the agent says it does not do that.
10. Run every transfer destination in the routing table, end to end, and confirm a human or an answering point actually picks up.
11. Call outside business hours and confirm the after-hours path.
12. Confirm the lead landed. Open the CRM, the inbox, the sheet, wherever the output is supposed to go, and look at the actual record.

That last step is the one people skip and it is the one that matters most.

Verify the wiring, not the logic

I learned this the expensive way. A system with green unit tests, all passing, shipped with a bug that cost a client eleven leads before anyone noticed. The logic was correct. The tests tested the logic. What was broken was the connection between the deployed system and the real data shape it received in production, which no test touched.

The fix, permanently, was to change what "tested" means. Now: replay real production records through the deployed system and look at what comes out the other end.

For a voice agent that means:

• Take actual call recordings or transcripts from the live agent and check that the extracted fields (name, number, reason for calling) landed correctly in the destination system. Not that the extraction step returned something. That the CRM row is right.
• Fire your webhooks with a payload copied from a real call, against the deployed endpoint, not a local one.
• After any change to the downstream integration, place one real test call and follow the record all the way through.

The general shape: test the deployed thing with real inputs, and inspect the far end of the pipeline. Anything less is testing your assumptions.

Test the things it should refuse to do

An agent that answers well is half the job. The other half is an agent that does not invent capabilities.

I had to fix a receptionist that hallucinated a caller-blocking ability. It told callers, with total confidence, that it had blocked a number for them. It had no such tool and had done nothing. The caller hung up satisfied, which is the worst possible outcome, because nobody found out until later.

So part of every test pass is deliberately asking for things the agent cannot do. Ask it to cancel an order, block a number, issue a refund, change an appointment, look up a balance. Whatever is plausible for that business but not actually wired up. If the agent agrees to any of it, add an explicit "abilities you do NOT have" list to the prompt. That negative list works far better than trying to constrain behavior positively. More on this in stopping chatbot hallucinations.

Also test the compliance boundaries. If the agent sends SMS, confirm it cannot text a number that has not opted in. I built an SMS assistant inside a client CRM where the consent gate is absolute: no opt-in, no message, no exceptions, no override. Test that gate directly by trying to make the agent break it. See TCPA compliance for AI SMS.

Testing after launch never stops

A voice agent is not a project that finishes. Things drift: a transfer destination gets disconnected, a model version changes behavior, a client adds a service the knowledge base does not know about.

What I keep running:

A silent-call detector. Query for calls that connected, lasted more than a few seconds, and produced zero agent turns. That number should be zero. If it is not, something is broken right now.

Weekly transcript review. Not all of them, a sample. Read fifteen real conversations and you will find things no test suite would produce. This is where you notice callers repeatedly asking about something the agent handles badly.

Transfer outcome logging. Log destination, result, and post-bridge duration. A "successful" transfer that lasted four seconds hit a voicemail beep. Details in call transfer flows.

A scheduled test call. A real call to every production agent on a recurring basis. It sounds excessive until the first time it catches a dead agent before the client does.

What good looks like

You are ready to hand a number over when: the agent answers in under two seconds, handles the three most common questions correctly, refuses clearly the things it cannot do, transfers successfully to every destination in its table, behaves correctly after hours, and the lead from your test call is sitting in the client's inbox with the right fields in it.

If any one of those is untested, you are going to find out about it from the client. That is a bad way to find out.

FAQ

Why does my agent work in the builder but fail on real calls?

Because the builder tests the middle of the stack and skips the ends. Simulators cannot inject system variables like caller ID, do not exercise the carrier and forwarding path, and do not reproduce real audio conditions. Always place a real call from a real phone before rollout.

How many test calls are enough before going live?

I aim for about a dozen deliberate ones covering the checklist above: greeting, barge-in, silent caller, the three most common questions, one out-of-scope request, each transfer destination, and one after-hours call. Then one final call where you follow the resulting lead all the way into the client's CRM.

What is the most commonly missed test?

Checking the far end of the pipeline. People confirm the agent said the right thing and stop there. Open the CRM and look at the record. Green tests once hid an integration bug that cost a client eleven leads, and the only thing that would have caught it was looking at real output from the deployed system.

Should I test with real customer calls?

Not as your first test, no. Do your own test calls first, then move to reviewing real transcripts once the agent is live. Reviewing real conversations weekly is the highest-value ongoing testing you can do, because real callers ask things you will never think of.

How do I test that the agent refuses things properly?

Deliberately ask it to do things it has no tools for: cancel, refund, block, reschedule, look up a balance. If it agrees to any of them, it is hallucinating a capability, and the fix is an explicit list of abilities it does not have written into the prompt.