Guide · Cost Optimization

Prompt Caching: Cut Repeated Input Costs by Up to 90%

If your app sends the same system prompt, documents or conversation history again and again, you are overpaying on every request. Prompt caching fixes that — here is how each provider implements it.

What is prompt caching?

LLM providers can remember the internal state of tokens they have already processed. When your next request starts with the exact same prefix — a system prompt, a codebase file, a long product catalog — the model skips recomputing it and bills those tokens at a steep discount. Same answer quality, fraction of the cost, faster time-to-first-token too.

The golden rule Caching matches by prefix, not by content anywhere in the prompt. Put stable content first (system instructions, reference docs) and variable content last (the user's actual question). One changed token early in the prompt invalidates everything after it.

Provider-by-provider

OpenAI (GPT-4o, o-series)

Automatic — no code changes. Requests longer than 1,024 tokens are cached when the prefix repeats; cached input costs 50% of base input ($1.25/M instead of $2.50/M on GPT-4o). Cache hits survive 5–10 minutes of idle traffic, longer during off-peak hours.

Anthropic (Claude)

Explicit — mark breakpoints with cache_control. Cache writes cost 25% extra once; cache reads cost about 10% of base input. Default TTL is 5 minutes, refreshable on each hit; a 1-hour TTL costs double to write. Ideal for long agent sessions and RAG pipelines.

Google (Gemini)

Explicit via the context caching API. You create a cached content object (minimum 4,096 tokens), pay a per-hour storage fee, then read it at roughly 25% of base input. Best for very large, stable corpora queried many times per hour.

DeepSeek

Fully automatic with a tiny 64-token minimum. Cache hits cost about one-tenth of base input, making DeepSeek one of the cheapest options for repetitive workloads even before its low base prices.

A worked example

Say your app sends a fixed 8,000-token system + document prompt with every user question, at 1,000 requests/day, on GPT-4o:

Run your own numbers with the cache-hit slider in the AITokenCalculator calculator — advanced options include cache hit rate, batch mode and reasoning effort.

Checklist to start saving today

  1. Move all static content to the front of your prompt.
  2. Keep prefixes byte-identical — no timestamps or random IDs inside them.
  3. Reuse one client/connection where possible; routing across pools can miss caches.
  4. Track the provider's cached-token metric in API responses to verify hits.
  5. Benchmark before/after with our cost-reduction playbook.

Prompt caching FAQ

How much does prompt caching save?

Cache reads cost 10–50% of base input price depending on provider: OpenAI cached input is 50% off, Anthropic cache reads are ~90% off, Gemini context caching is 75% off and DeepSeek automatically applies ~90% off for repeated prefixes.

Do cached tokens cost anything to write?

Anthropic charges a one-time 25% premium on cache writes; OpenAI charges nothing extra (caching is automatic); Gemini has a storage fee per hour; DeepSeek charges nothing. Write costs pay for themselves after one or two reuses.

How long does an LLM prompt cache live?

OpenAI caches evict after 5–10 minutes of inactivity (up to a few hours off-peak), Anthropic defaults to 5 minutes (refreshable to 1 hour), Gemini lets you set TTLs from minutes to hours, and DeepSeek caches for hours automatically.

What is the minimum prompt length for caching?

OpenAI requires 1,024 tokens, Anthropic 1,024 (2,048 for some models), Gemini 4,096 for explicit caching, and DeepSeek 64. Shorter prompts cannot be cached.