Turn-Taking

Turn-taking is the mechanism that decides whose turn it is to speak. For a voice agent that means detecting when the caller has actually finished a thought, responding at the right moment, and yielding when the caller starts again. Humans do this without thinking. Software does it badly by default.

Why it matters when you are buying or building

The core problem is called endpointing: deciding that a pause means "done" rather than "thinking." Most systems use a silence timer. Set it short and the agent cuts people off mid-sentence, especially anyone reading a number off a screen or recalling an address. Set it long and every exchange has an awkward beat before the agent replies, which reads as slow even when the model is fast. See latency.

There is no correct value, only a correct value for your callers. A booking flow where people read out card details or addresses needs a longer threshold than a yes-or-no confirmation. Better platforms let you vary it per step, and if you are evaluating, ask specifically whether endpointing is tunable and whether it can differ by prompt.

Other turn-taking behaviors worth checking:

• Backchannels. Callers say "mhm" and "right" while listening. The agent should not treat those as a turn.
• Interruption recovery. When a caller cuts in, the agent must not assume the rest of its sentence was heard. That is barge-in handling done properly.
• Silence handling. If the caller goes quiet, the agent needs a graceful re-prompt after a set interval, then a clean exit. Not an infinite wait.
• Overlap. Both parties starting at once. The agent should yield, every time.

What I have seen in production

The pattern that fixed the most complaints for me was not a setting, it was writing. Short agent turns. One or two sentences, then a question, then stop. An agent that says three sentences and hands the turn back feels responsive even on a mediocre stack. An agent that delivers a paragraph feels like an obstacle regardless of how fast it is. That rule goes in the system prompt explicitly for every voice build I do, because models default to writing for the page.

Second: fill the gaps. When the agent has to make a lookup that takes a second or two, it should say something first. Silence during a tool call is indistinguishable from a broken call, and callers will say "hello?" into it, which then creates an unexpected turn to handle.

Third, and this applies to everything on this site: none of it can be validated in a simulator. Endpointing behavior depends on real audio, real network timing, and real people who pause in the middle of sentences. Test by calling the live number from a cell phone before any rollout. More in testing voice agents.