Prompt caching with the OpenAI, Claude and Gemini APIs: what is cached and what breaks it

A prompt caching API reuses the work a model already did on the unchanged start of your prompt, so repeated system prompts, tool definitions and documents cost less and start faster. OpenAI and Gemini cache automatically, Claude caches when you add cache_control, and all three need the cached part to be byte-for-byte identical and above a minimum length. Each provider reports hits in its usage fields, which is the only reliable way to know the cache worked.

What prompt caching does and when it pays off

A model turns input tokens into internal key-value states before it writes anything. Prompt caching saves those states for a prefix, the unchanged tokens at the start of a request. When the next request starts with the same prefix, the provider reads the saved states and only processes the new part.

Caching pays off when a large, stable block sits in front of many short, varying requests. Long system prompts, tool definitions, few-shot examples, reference documents and growing chat histories all fit. A short prompt, or one whose start changes each time, gains nothing.

OpenAI's guide gives the arithmetic for its newer models. A cache write costs 1.25× the normal input rate and a read 0.1×, so writing a prefix once and reusing it once costs 1.35× its ordinary input cost, against 2× without caching. Across ten requests, one write and nine reads cost 2.15× against 10×.

The three providers side by side

The figures below come from the OpenAI prompt caching guide (opens in a new tab), the Claude prompt caching docs (opens in a new tab) and the Gemini context caching docs (opens in a new tab), checked on October 12, 2026. Prices change, so confirm them on each pricing page, such as the Gemini API pricing page (opens in a new tab), before you budget.

Older OpenAI models behave differently. GPT-5.5 and earlier only cache implicitly and charge no cache-write fee. Their lifetime is set with prompt_cache_retention: in_memory (about 5 to 10 minutes of inactivity, up to one hour) or 24h, depending on the model.

OpenAI: automatic caching and how to check for a hit

OpenAI caches the full rendered context, including developer messages, tool definitions and conversation history with text, images and documents. You do not opt in. On GPT-5.6 and later you can choose implicit or explicit-only mode with prompt_cache_options.mode.

Explicit-only mode is useful when a static block is followed by content that changes on every call. Mark the end of the static block, and everything after it is billed at the plain input rate without a write charge:

{
  "model": "gpt-6.1-sol",
  "prompt_cache_options": { "mode": "explicit" },
  "input": [
    {
      "role": "developer",
      "content": [{
        "type": "input_text",
        "text": "Stable instructions and reference material...",
        "prompt_cache_breakpoint": { "mode": "explicit" }
      }]
    },
    { "role": "user", "content": "Today's question..." }
  ]
}

Each request can create up to four cache writes. Top-level instructions cannot hold a breakpoint, so put reusable instructions in an input_text block inside a developer message. To check a hit, read usage.input_tokens_details.cached_tokens on the response.

Claude: cache breakpoints and lifetimes

Claude caches the prefix in a fixed order: tools, then system, then messages. The quickest start is a single top-level "cache_control": {"type": "ephemeral"}, which moves the breakpoint forward to the last cacheable block as a conversation grows. For a prompt with a stable head and a changing tail, place the breakpoint yourself on the last block that stays the same:

import anthropic

client = anthropic.Anthropic()
response = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=1024,
    system=[{
        "type": "text",
        "text": LONG_STABLE_INSTRUCTIONS,
        "cache_control": {"type": "ephemeral"},
    }],
    messages=[{"role": "user", "content": question}],
)
print(response.usage.cache_read_input_tokens, response.usage.cache_creation_input_tokens)

Two lifetime details catch people out. The 5-minute clock starts when the request that writes or reads the entry starts, so a response that streams for 4 minutes leaves about 1 minute for the follow-up. For prompts reused less often than every 5 minutes but more often than hourly, the docs point to "ttl": "1h", priced at 2× base input for the write.

Reads look backward from each breakpoint for at most 20 blocks. A conversation that adds 20 or more blocks between cache writes loses its hit unless you add a second breakpoint closer to the earlier write.

Gemini: implicit and explicit caching

Gemini's implicit caching (opens in a new tab) is on by default for Gemini 2.5 and newer models, and Google passes on the saving automatically when a request hits. The docs offer two tips: put large, common content at the start of the prompt, and send requests with a similar prefix within a short time. Implicit hits carry no cost-saving guarantee.

Explicit caching works differently. You upload content once as a named cache object with a TTL, then reference it by name:

from google import genai
from google.genai import types

client = genai.Client()
cache = client.caches.create(
    model="models/gemini-3.8-flash",
    config=types.CreateCachedContentConfig(
        system_instruction="You answer questions about the attached manual.",
        contents=[manual_file],
        ttl="3600s",
    ),
)
response = client.models.generate_content(
    model="models/gemini-3.8-flash",
    contents="Which torque setting applies to part 14?",
    config=types.GenerateContentConfig(cached_content=cache.name),
)
print(response.usage_metadata)

Explicit caching is in Beta under v1beta and only works through the generateContent API. The Interactions API supports implicit caching alone. After creation you can change only the ttl or expire_time of a cache, and you pay for storage for as long as it lives.

How to order a prompt so the stable part comes first

All three providers match from the first token forward, so one changed token early in the prompt throws away everything after it. Order each request from least to most volatile:

1. Tool definitions, in a fixed order with fixed schemas.
2. System or developer instructions without dates, user names or request IDs.
3. Reference documents and few-shot examples.
4. Conversation history, appended and never rewritten.
5. Per-request content: the timestamp, the user's profile, the new question.

OpenAI's guide adds a tools rule: to stop the model calling a tool, set tool_choice to "none" or restrict calls with allowed_tools instead of deleting the definition. For a new instruction partway through a conversation, Claude's docs describe appending a {"role": "system"} message to messages on newer models, which leaves the cached system field untouched.

Mistakes that silently miss the cache

None of these return an error. The request succeeds, and you pay full price.

Caches are also scoped. OpenAI does not share caches across organizations or regional processing boundaries. Claude isolates caches per workspace on the Claude API, so the same prompt in two workspaces is cached twice.

How to measure the saving from the usage fields

Log the cache fields on every response, then compute the hit rate as cached tokens divided by total input tokens, grouped by endpoint or day. A tracing tool such as Langfuse can store these per request.

On OpenAI, input_tokens includes everything. Ordinary tokens are input_tokens - cached_tokens - cache_write_tokens, and the guide's cost formula weights cached tokens by the read multiplier and write tokens by the write multiplier.

On Claude, input_tokens counts only tokens after the last breakpoint. The total is cache_read_input_tokens + cache_creation_input_tokens + input_tokens. A request with 100,000 cached tokens and a 50-token question reports input_tokens: 50 for a total of 100,050.

def claude_input_cost(u, base_per_mtok, read_mult=0.1, write_mult=1.25):
    weighted = u.input_tokens + u.cache_read_input_tokens * read_mult \
        + u.cache_creation_input_tokens * write_mult
    return weighted * base_per_mtok / 1_000_000

Set read_mult to the multiplier for your model: 0.05 on Claude Opus 5.5 and Sonnet 5.5. For 1-hour writes, use 2.0, and read the split from the cache_creation object, which reports ephemeral_5m_input_tokens and ephemeral_1h_input_tokens separately.

OpenAI's guide also works through the case of a shared prefix just under the minimum. With a 1,024-token minimum, a 0.1× read and a 1.25× write, padding the prefix with useful stable material up to 1,024 tokens costs less across 10 requests once the original prefix is at least 221 tokens. A prefix of 102 tokens or fewer never benefits under those assumptions.