LLM (Large Language Model)

A large language model (LLM) is a program trained on huge amounts of text that predicts the next chunk of words given everything it has been shown so far. That is the whole trick. Everything a chatbot appears to "know" or "decide" comes out of that prediction loop, steered by the instructions and data you feed it.

Why it matters when you are buying or building

Most chatbot platforms are wrappers around someone else's LLM. Voiceflow, ElevenLabs Agents, Telnyx AI Assistants, Retell, Vapi: under the hood they are calling a model from OpenAI, Anthropic, Google, or an open-weights provider. When a vendor sells you "our AI", ask which model, whether you can change it, and whether you can set a fallback.

Three things about the model choice actually show up in your bill and your call quality:

• Speed. For voice, the model's time to first token drives how long the caller sits in silence. A smarter model that thinks for two seconds feels broken on a phone call. See latency.
• Instruction following. Cheaper models drift from the prompt faster and are more likely to invent capabilities. Bigger models hold a long rulebook better.
• Cost. You pay per token in and per token out. A bloated system prompt or a fat knowledge base result set gets re-sent on every single turn, so prompt size is a recurring cost, not a one-time one.

What I have seen in production

The single most expensive lesson I have had with models is not about quality, it is about availability. On a telecom client's voice agents I had assistants running with no fallback model configured. When the primary model had an outage, callers got answered and then heard nothing at all. Dead air. It looks exactly like a carrier or SIP fault, so you go debug the wrong layer for an hour. Now every agent I ship has a second model configured as a fallback, and I test the failover before handoff. More on that in voice agent dead air debugging.

The other thing worth internalizing: the model does not know your business. It knows language. Anything specific (hours, pricing, which tech covers which zip code) has to be injected, either in the prompt or through retrieval. If it is not injected, the model will produce something plausible-sounding instead of admitting ignorance. That failure mode has its own name, hallucination, and it is the thing that will embarrass you in front of a customer.

Do not over-index on picking the "best" model. Pick one that is fast enough, cheap enough, and follows instructions, then spend your effort on the prompt and the data.