Your AI agent doesn't get expensive because it's clever. It gets expensive because it keeps rereading the chat.
Two ways to run the same booking
Take one ordinary WhatsApp booking. Eight turns: a question, a few details, a confirmation. Nothing exotic, and the customer gets the same answer either way.
The only difference is what the agent sends to the model on each turn.
Full history
- Turn 1 sends 1 message. Turn 8 sends 8.
- 68,400 tokens across the booking.
- Nothing is forgotten, so nothing is ever dropped.
- The bill grows faster than the conversation.
Lean context
- Every turn sends one compact pack.
- 8,320 tokens across the booking.
- Booking state, retrieved facts, latest message.
- Turn 50 costs about what turn 1 costs.
Eight turns, priced at R24.10 per million input tokens. Same booking, same answer, 8x the bill.
What full history actually sends
Every turn, the whole conversation goes back to the model. Turn 1 is one message. Turn 8 carries the first seven behind it, and the model reads every word again before it answers.
You are paying to re-read. Not to think.
- Cost per turn climbs with the length of the chat, so long conversations get expensive fast.
- A customer who types a lot, or a conversation with several confirmations, multiplies the bill.
- One change to the last message means every earlier message is re-sent anyway.
What lean context sends instead
Send only what the answer needs. Three things:
- A short booking state. Date, time, service, name. A few dozen tokens, not a transcript.
- The few facts retrieved from your knowledge base. The two or three passages that actually answer this question, pulled by search.
- The latest message. What the customer just said.
That is retrieval-augmented generation doing the cost work. The agent still knows the booking, because the state carries it, and it still knows your business, because search fetches the right passage. It just stops re-reading yesterday.
Does the answer get worse?
No, and this is the part that surprises people. A skilled agent working from three relevant passages usually answers better than one wading through eight turns of small talk, because the relevant words are no longer buried in the irrelevant ones. Accuracy and cost are not a trade-off here. You get both, or you are doing one of them wrong.
How to tell if your agent is doing this
- Log the input tokens on every turn. If they climb through the conversation, you are resending history.
- Check the cost per conversation, not the cost per token. Per-token pricing hides the growth.
- Ask what gets sent on turn 20 of a long chat. If the answer is "everything", that is the bill.
- Watch long conversations specifically. Short ones hide the problem almost completely.
How much is your AI agent really costing you per conversation?
This is the kind of thing we measure, then fix. If your agent is on WhatsApp, Facebook or a website, we can tell you what each conversation costs you today, and what it should cost.
AI Partner PTY LTD, Johannesburg South. 073 776 7138.