![\"Anatomy](\"/5304fcf54278b72694b271a9e6961b69/cache-anatomy.svg\")
Most of the tokens in an LLM request are the same tokens you sent last time. Take CloudCanvasAI, where Claude edits .docx and .pptx files inside a per-user sandbox. Every turn carries the same system prompt, the same tool descriptions, and a growing conversation history. The only genuinely new part is the last thing the user typed. Re-sending the rest on each request means paying to re-read a document that hasn’t changed.
Prompt caching fixes that. You mark a stable chunk of the request once, and Anthropic keeps its processed form around for a few minutes. The next request that starts with the same bytes skips that work.
\nThis post is for engineers already calling the Claude API who want to cut cost and time-to-first-token on repeated calls without changing what the model sees. I’ll cover what actually gets cached, the exact price of a hit versus a miss, and the handful of ways people break caching without noticing.
\nThe cache is a prefix match. Anthropic renders every request into one ordered sequence: tools first, then system, then messages. Caching keys off that sequence from the start up to a marker you place, and everything before the marker is the cached prefix. Change a single byte anywhere inside that prefix and the match fails from the point of the change onward, so the model reprocesses the rest.
That ordering is the whole game. Stable content has to come first. Volatile content (the new user question, a per-request id, anything with a timestamp) has to come after the last marker. Get the order wrong and you cache nothing.
\n
The marker is a cache_control entry of type ephemeral. It doesn’t cache a specific block in isolation. It caches the whole prefix up to and including that block.
The simplest form marks a stable system prompt. In the example below, the style guide and format rules are the same on every request, so they go in system with a cache_control marker. The user’s turn stays in messages, after the marker, where it belongs.
from anthropic import Anthropic\n\nclient = Anthropic()\n\nSYSTEM_DOCS = load_style_guide() # ~8k tokens, identical every request\n\nresp = client.messages.create(\n model=\"claude-opus-5\",\n max_tokens=4096,\n system=[\n {\n \"type\": \"text\",\n \"text\": SYSTEM_DOCS,\n \"cache_control\": {\"type\": \"ephemeral\"},\n }\n ],\n messages=[{\"role\": \"user\", \"content\": user_turn}],\n)The default cache lifetime, or TTL (time to live), is five minutes, and every hit inside that window resets the clock. If your traffic is bursty enough that requests land more than five minutes apart, you can ask for a one-hour lifetime by adding a ttl to the marker:
\"cache_control\": {\"type\": \"ephemeral\", \"ttl\": \"1h\"},One detail trips people up: there’s a minimum prefix length. Prompts shorter than the model’s floor won’t cache at all, and you get no error. The request just runs uncached.
\nThe floor depends on the model: 512 tokens on Claude Opus 5, 1,024 on Sonnet, and higher on some older models. The current table is in the prompt caching docs. If you’re caching a short prompt and seeing no savings, this is usually why.
\nAn agent loop is where caching earns the most, because the history only grows. The trick is to move the breakpoint forward each turn. Put it on the last message that won’t change again, so the cached prefix keeps extending as the conversation does.
\nYou get up to four breakpoints per request. A common layout uses:
\nThe snippet below sets that trailing breakpoint. It marks the last block of the existing history, then appends the new user turn after it:
\nmessages = build_history() # everything so far\nmessages[-1][\"content\"][-1][\"cache_control\"] = {\"type\": \"ephemeral\"}\nmessages.append({\"role\": \"user\", \"content\": new_turn}) # uncached tailBecause the new user turn sits after the marker, it never invalidates the cached history behind it.
\nThe numbers are what make this worth doing. Relative to the base input token price, the official multipliers are:
\nWork through the break-even. Without caching, two identical-prefix requests cost 1× + 1×. With caching, the first writes the prefix (1.25×) and the second reads it (0.1×), so the same two calls cost 1.25× + 0.1× = 1.35×.
A single reuse inside the window already comes out ahead, and the gap only widens from there. Ten reads is 1.25× + (10 × 0.1×) = 2.25× against 11× uncached. The extra quarter you pay on the write is bought back on the first hit.
The other half of the payoff is latency. A cache read skips the prefill work (the model’s first pass over the input) for that prefix, so time-to-first-token drops on long prompts. In a streaming UI like CloudCanvasAI’s split-panel view, that’s the difference between the response appearing to start immediately and a visible pause while the model re-reads context it already had.
\nIf you want the background on why prefill is the expensive part, I wrote about the KV cache and GPU memory separately. Prompt caching is essentially Anthropic holding that computed state for you between requests.
\nEvery failure mode here has the same symptom: the request works, the answer looks fine, and you’re paying full price because nothing hit the cache. None of these raise an error.
\nA timestamp in the prefix. The classic one. Something like f"Current time: {datetime.now()}" at the top of the system prompt changes the first bytes on every request, so the prefix never matches. If the model genuinely needs the time, put it after the last breakpoint, in the user turn.
Non-deterministic serialization. Say part of your prefix is JSON built from a dict. Python doesn’t guarantee the same key order across all the code paths that might construct it, and a reordered key is a different byte string. Serialize anything that lands in a cached block with json.dumps(obj, sort_keys=True).
A tool list that shifts. Because tools renders first, reordering tools or regenerating their descriptions invalidates everything after them, the system prompt and the whole conversation included. Build the tool array in a fixed order and keep the descriptions stable. I ran into a version of this with LLM DevMate. A tool set that’s stable across a session caches well. One rebuilt per request from a set or a dict throws the order away and quietly costs you the cache.
Letting the window lapse. On the five-minute cache, a gap longer than the TTL means the next request pays for the write again. That’s fine for chat, but wasteful for a batch job that pauses between items. Either keep requests flowing or reach for the one-hour TTL.
\nDon’t assume. The usage object on every response splits the input tokens into three buckets and tells you exactly what happened:
u = resp.usage\nprint(u.input_tokens) # uncached tokens, full price\nprint(u.cache_creation_input_tokens) # written to cache this request (1.25x)\nprint(u.cache_read_input_tokens) # served from cache (0.1x)The signal you want is cache_read_input_tokens climbing on repeated requests. If it stays at zero across calls you believe share a prefix, one of the invalidators above is at work. Start by diffing the exact bytes of two consecutive prefixes.
On the first request with a fresh prefix, you’ll see cache_creation_input_tokens populated and reads at zero. That’s expected, since something has to write the cache before anything can read it.
Caching isn’t free to reason about. Three things are worth keeping in mind:
\nephemeral.The mental model that keeps me out of trouble is to treat the prefix as immutable and design the request around it: freeze the system prompt, fix the tool order, sort any serialized structure, and shove every varying byte past the last breakpoint. Once the prefix is genuinely stable, caching is close to free money on any workload that repeats it. For the LLM tooling I’ve built around CloudCanvasAI and Archi, that’s nearly all of them.
\nIf you’re wiring this into a service, it pairs naturally with the retry and backoff patterns I covered earlier. A cached prefix makes a retried request cheaper too, since the retry reads the same cache the first attempt wrote.
","frontmatter":{"title":"Prompt Caching with the Claude API","date":"2026-08-30T00:00:00.000Z","description":"Prompt caching reuses a stable prompt prefix to cut Claude API cost and latency. How breakpoints, TTLs, and cache reads work, and what quietly breaks a hit.","thumbnail":{"childImageSharp":{"fluid":{"base64":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABQAAAALCAIAAADwazoUAAAACXBIWXMAAAsSAAALEgHS3X78AAABw0lEQVQoz1WP6XKjMBCE+RMbEBK3bonLgMGOTRInqayvXb//Q+1gJ1u11FeqVk83U3KGTo+dHlq1Hezt8vb1sbn82v8+Teev3eW4Px9Bv4ADVxCnuwnJoZ1bzoIUS1I+BRbnfb+71sNxPf1Zbc/1eOq219X20j1fWxCb8zDdms0JBKHrJ2yh5QSJAfzIkLzx49INjRcaOFEMpgYfxF1/mwBOiyCeWw6axwVTtSo6U7Y513FmmNAusTB+pFFkSWrSXIW0r4Z3075ldgqS0vFDyBkuzeeO3j7z6zvdDfx6YEmuXKJRrO/7Z/ZrLnUR8U0qh5j1sBXKyiMqzvSm59PIxo4dnikwDZQk0Ff3gITTGn7YZiRiCyQ8wsFxfCJdLDOqqoJbywvLq4KVlvc1fRlzFAoIRan1Q2nUbL6OOY7l/DsiHY8IF4so1VoJJX9QQgjZNbI2kmVVo/ssMYJL+NpKFlbAi/xQOB7my0AUmsfcBnaFZY15FajaC4QfzQtRKBeYS1qqvPRCeIWBfbAWilBmKORK24SZUFSJbjKzwpl2EYUR4GLmE6FEA+BIusG3DzhwARaILhF1UQ7cBX34/1g+Av+bfwHdJ0MNNm91MgAAAABJRU5ErkJggg==","aspectRatio":1.899441340782123,"src":"/static/ee39d6bd9f6285aa6710ac73b9b54026/40a76/hero.png","srcSet":"/static/ee39d6bd9f6285aa6710ac73b9b54026/c972b/hero.png 340w,\n/static/ee39d6bd9f6285aa6710ac73b9b54026/27625/hero.png 680w,\n/static/ee39d6bd9f6285aa6710ac73b9b54026/40a76/hero.png 1360w,\n/static/ee39d6bd9f6285aa6710ac73b9b54026/ed396/hero.png 2000w","sizes":"(max-width: 1360px) 100vw, 1360px"}}}}}},"pageContext":{"slug":"/2026-08-30-prompt-caching-claude-api/","previous":"blog/2026-08-26-scheduling-gpu-pods-kubernetes/","next":"blog/2026-08-29-circuit-breaker-llm-api-calls/"}},"staticQueryHashes":["32046230"]}