How to Write a Chatbot System Prompt That Holds Up in Production

The system prompt is the only part of an AI agent that you fully control. The model changes under you, the knowledge base drifts, the caller says something nobody predicted. The prompt is the fixed point. So when a bot misbehaves in production, the prompt is where I look first, and about seven times out of ten that is where the fix lands.

I write and maintain prompts for a fleet of live agents: voice receptionists on real business lines, a shopping assistant for a jewelry retailer running on both chat and voice, agents for a UK windows-and-doors company, an SMS assistant living inside a sports-facility CRM. Different platforms, same discipline. What follows is how I actually write them, plus a full skeleton you can copy for a small-business receptionist.

A system prompt is an operating procedure, not a personality description

The most common bad prompt I inherit reads like a character sheet. "You are a friendly, helpful, professional assistant for Acme Plumbing. You are knowledgeable and always polite." That is 30 words that change nothing about behavior. The model was already going to be polite.

A prompt that earns its place answers four questions in order:

1. Who am I and who am I talking to?
2. What am I trying to accomplish on this contact?
3. What am I allowed to do, and what am I explicitly not allowed to do?
4. What do I do when I do not know?

Everything else is decoration. If a line in your prompt does not change what the bot says or does in some concrete situation, delete it. Long prompts are not more controlled prompts. Long prompts are prompts where the important instruction is buried on line 60 and gets diluted.

Order matters more than wording

Models weight the start and end of the prompt more heavily than the middle. I put identity and the single most important behavioral rule at the top, and I put the hard prohibitions at the bottom where they are the last thing read before the conversation starts. Reference material (hours, addresses, service areas) goes in the middle or, better, in a knowledge base so it can be updated without touching the prompt.

That split matters operationally. Prompt edits need re-testing. Knowledge base edits usually do not. If a client's holiday hours are hardcoded in the prompt, every seasonal change becomes a deploy with regression risk. If they are in the KB, an office manager can update them.

The receptionist skeleton

This is close to what I ship for a small-business voice receptionist. Adapt the specifics, keep the structure.

# Identity
You are the receptionist for {{business_name}}, a {{business_type}}
serving {{service_area}}. You answer the main phone line. Callers are
usually existing customers with a question, or new prospects pricing a
job. You are speaking out loud on a phone call.

# Primary goal
Every call should end in one of exactly three outcomes:
1. The caller's question is answered from what you know.
2. An appointment or callback is booked with name, phone, and reason.
3. The caller is transferred to a human.
If you cannot reach one of those three, take a message and confirm the
callback number back to the caller digit by digit.

# Voice style
- Speak in short sentences. One idea per sentence.
- Never read out URLs, email addresses, or long lists on a call.
- Do not use bullet points, markdown, or emoji. This is spoken audio.
- Numbers: say prices as "one hundred and fifty dollars", not "$150".
- If the caller interrupts, stop talking and listen.

# Tools you have
- lookup_hours_and_services: use for any question about hours,
  services offered, pricing ranges, or service area.
- book_appointment: use only after you have name, callback number,
  service needed, and preferred day.
- transfer_to_human: use when the caller asks for a person, is upset,
  is discussing an existing invoice or complaint, or asks anything
  outside the services listed.

# Abilities you do NOT have
You cannot do any of the following. If asked, say plainly that you
cannot and offer to take a message or transfer.
- You cannot block, ban, or report a phone number.
- You cannot cancel, refund, or change an existing invoice or payment.
- You cannot access customer account history or past job records.
- You cannot send documents, contracts, photos, or email attachments.
- You cannot give a firm quote. You may give a published price range
  only if the tool returns one.
- You cannot promise a specific technician or a same-day arrival.

# When you do not know
Say "I don't have that in front of me" and then either transfer or
take a message. Never guess a price, a date, a policy, or a name.
Never invent a person's name or a job number.

# Opening line
"Thanks for calling {{business_name}}, this is Robin. How can I help?"

A few notes on why each block is shaped this way.

The three-outcome goal. Without an explicit success definition, agents wander. They will happily chat for four minutes and hang up having captured nothing. Naming the terminal states gives the model something to steer toward and gives you something to measure in analytics.

The voice style block. Prompts written for chat and reused for voice are a reliable source of bad calls. The model will read out a markdown bullet list, character by character, and the caller hears nonsense. Say explicitly that this is spoken audio.

The abilities block. This one came out of a real incident. A receptionist I built started telling callers it had blocked their number. It had no such tool. It said it confidently, twice, on recorded calls. The model had inferred that a receptionist plausibly does that, and nothing in the prompt contradicted it. Adding an explicit "abilities you do NOT have" list fixed it same day. I now put that block in every prompt I write, before the bot has a chance to invent something. There is more on that failure mode in stopping chatbot hallucinations.

The "when you do not know" block. Notice it gives a specific phrase. "Say you don't know" is weak instruction. Giving the exact sentence makes the behavior consistent and makes it greppable in transcripts.

Variables: the quiet killer

Most platforms let you inject runtime values into the prompt: caller ID, business name, current date, customer tier. This is genuinely useful and it is also where I have lost the most time.

Two failure modes, both from production:

An assistant configured with an empty dynamic variables object answered the phone with total silence. Dead air, every call. It looked exactly like a carrier or SIP problem, and I spent real time in the wrong layer before finding it was the prompt config. If your agent goes silent on answer, check variables before you check the trunk. I wrote that whole debugging path up in voice agent dead air debugging.

Separately, an unsubstituted {{placeholder}} left in the instructions does not throw an error. It sits there as literal text and quietly degrades the prompt, and sometimes the model reads it aloud. Before every rollout I now grep the rendered prompt for stray braces.

The rule I follow: never reference a variable in the prompt that you have not confirmed is populated at runtime, and always give the agent a fallback. Write "the caller is calling from {{caller_number}}, which may be unknown" rather than assuming it is there. Simulator testing often cannot inject system variables like caller ID at all, so this class of bug only shows up on a real call. Always test-call your own cell before a client rollout. See testing voice agents for the fuller checklist.

Tool descriptions are part of the prompt

People treat tool definitions as plumbing and write the descriptions carelessly. The model reads them exactly like prompt text and they are often the deciding factor in whether a tool fires at the right moment.

Write tool descriptions as trigger conditions, not as capability statements. "Books an appointment" is weak. "Call this when the caller has agreed to a specific day and you have their name and callback number. Do not call it to check availability." is strong. Say when not to call it. Ambiguous tools get called at the wrong time or not at all, and the caller experiences that as the bot ignoring them.

The same care applies to transfers. A transfer tool has to reference the live call's control identifier, not a value copied from an earlier test. Hardcode it and every transfer fails with a 422 while the caller sits there listening to nothing. More on that in call transfer flows.

Iterating without breaking things

My loop is boring and it works.

Read transcripts. Not summaries, actual transcripts, at least ten per change. Find the specific turn where the bot went wrong. Change one thing in the prompt. Re-run the same conversations. Keep the prompt in version control with a one-line note on what each change was meant to fix.

Resist the urge to fix a single bad transcript with a new rule. Prompts rot by accumulation, and eighteen months later half the rules contradict each other. Once a quarter I reread the whole prompt and delete rules that no longer correspond to a real failure.

When a rule genuinely needs to be absolute, say so in absolute language and back it with code where you can. Consent is the clearest example. On an SMS assistant I built for a sports-facility CRM, the prompt says the bot may never message someone who has not opted in, and the send path also enforces it outside the model. A prompt rule is a strong preference. A code gate is a guarantee. For anything with legal weight, use both, as covered in TCPA compliance for AI SMS.

FAQ

How long should a system prompt be? For a small-business receptionist, 400 to 800 words of actual instruction is usually right, with reference facts pushed into a knowledge base. I have shipped effective prompts at 300 words. I have never seen a 3,000 word prompt that would not have worked better at a third of the length. Length correlates with contradiction, not with control.

Should the same prompt work for chat and voice? No. Share the identity, goals, and prohibitions, then keep separate style and formatting blocks. Chat can use links, lists, and short paragraphs. Voice cannot use any of them. I keep one shared core file and two thin wrappers so the prohibitions never drift apart between channels.

Why does the bot ignore a rule that is clearly in the prompt? Usually one of three things: the rule is in the middle of a long prompt where attention is weakest, another line contradicts it, or it is phrased as a preference ("try to avoid") rather than a prohibition ("never"). Move it to the end, remove the contradiction, and use absolute wording. If it still leaks, enforce it in code instead.

Do I need to change my prompt when the model is upgraded? Test it, do not assume. Newer models generally follow instructions better, but they also follow them more literally, so a rule that was harmlessly vague can start producing odd behavior. I replay a fixed set of real transcripts through any new model before switching, and I keep a fallback model configured, because an agent with no fallback goes to dead air the moment the primary has an outage.

What is the single highest-value block to add to an existing prompt? The "abilities you do NOT have" list. It takes ten minutes, it costs nothing at runtime, and it prevents the most damaging category of failure, which is the bot confidently promising a customer something the business will not do.