Reducing token usage should not mean making the output worse. It should mean cutting waste. Most teams pay too much because they keep sending repeated context, ask expensive models to do simple formatting work, or let agents re-run the same steps too many times.
What tokens really are
Tokens are the pieces of text the model processes on the way in and on the way out. Cost grows with both input and output, so the question is not only “how long was my prompt?” It is also “how often did I resend it, and what did the model generate back?”
Where usage explodes
- Huge system prompts repeated on every request
- Long chat histories carried forward even when they are no longer useful
- Agents that keep retrying with the same large context
- Verbose outputs when a compact result would do
- High-end reasoning models used for easy classification or formatting
The highest-leverage fixes
Prune context
Do not keep every prior message. Carry forward the current task state, the latest relevant facts, and a short summary of what happened.
Set output limits
If the job is to return a shortlist, an extraction, or a JSON object, ask for that directly. Many teams spend heavily on explanation they never read.
Use summarization handoffs
At stage boundaries, replace the full transcript with a compact state summary. This is one of the easiest ways to shrink context without losing the thread.
Route models by task
OpenAI’s model optimization guidance explicitly encourages using smaller, cheaper models where a larger one is not cost-effective. That usually means reserving top models for reasoning, judgment, or complex synthesis, not for rote cleanup.
Be disciplined about tool calls
If an agent keeps searching, re-reading, and reformatting the same information, you are not paying for intelligence. You are paying for loop noise.
How to measure cost per task
- Track input tokens, output tokens, and cache hits separately.
- Measure the full workflow, not just one model call.
- Compare cost against a useful business unit such as “one support resolution” or “one merged PR.”
- Watch retry counts and context size over time.
The goal is not to make prompts tiny. The goal is to stop paying for repetition, indecision, and unnecessary verbosity.
What Dera recommends
Start by removing waste before downgrading quality. Prune context, cache stable prefixes, cap retries, and route work to the cheapest model that can do it reliably. That usually cuts spend faster than rewriting every prompt from scratch.