Prompt caching gives you 90% off tokens. Until you add a timestamp.

LLM & Model Guides · By Caleb Sakala · April 14, 2026

Teams implement prompt caching, check the API pricing docs, and expect their Claude bills to drop by 90%. Then the billing statement arrives. The line items look exactly the same.

This happens constantly. Anthropic’s own troubleshooting guide calls it a “common mistake.” The frustrating part is that the mistake is invisible: the API returns a successful response, no error is raised, and cache_read_input_tokens sits at zero with no explanation. The problem lives in placement.

What the 90% savings actually means

Claude Sonnet 4.6 charges $3.00 per million input tokens. Cache reads for the same model cost $0.30 per million, which is 10% of the base price, or 90% off.

OpenAI’s automatic caching discounts cached input tokens by roughly half. Both providers require a minimum number of tokens before caching activates: 1,024 for Claude Sonnet 4.5, 2,048 for Sonnet 4.6, 4,096 for Opus 4.6 and Haiku 4.5. Short prompts don’t cache at all, and neither provider returns an error when this happens. You only discover the problem if you’re watching cache_creation_input_tokens and cache_read_input_tokens in the response usage fields. Both sitting at zero means caching either wasn’t attempted or failed silently.

The savings are real. The issue is that most teams structure their prompts in a way that makes cache hits impossible.

The prompt caching placement requirement that nobody mentions first

The core rule: cache writes happen only at the breakpoint you set, and the breakpoint hash covers everything before it. If anything in that prefix changes between requests (any token, anywhere), the hash changes and the cache misses.

Anthropic’s docs document the exact failure mode as a “common mistake”:

Your prompt has a large static system context (blocks 1 through 5) followed by a per-request block containing a timestamp and the user message (block 6). You set cache_control on block 6. Request 1: cache write at block 6, hash includes the timestamp. Request 2: the timestamp differs, so the prefix hash at block 6 differs. The lookback walks through blocks 5, 4, 3, 2, and 1, but the system never wrote an entry at any of those positions. No cache hit. You pay for a fresh cache write on every request and never get a read.

Automatic caching hits the same trap. It places the breakpoint on the last cacheable block, which, if that block contains a timestamp or user message, changes every request.

The lookback does not find stable content behind your breakpoint and cache it retroactively. It finds entries that prior requests actually wrote. If those writes never happened because the breakpoint was on changing content, there is nothing to find.

Why automation workflows get prompt caching hit rates of zero

A concrete example. A customer support routing workflow makes 10,000 LLM calls per day. The system prompt is 3,000 tokens of classification instructions, static and identical across every call. The user message is the ticket text, which changes every time.

The setup looks reasonable. But the engineering team added a feature: the workflow prepends each ticket’s priority level and submission time to the system prompt before the instructions:

system_prompt = f"Priority: {ticket.priority}\nTime: {ticket.created_at}\n\n{STATIC_INSTRUCTIONS}"

The cache_control breakpoint is at the end of the system prompt. On every request, the hash of the dynamic prefix plus 3,000-token instructions is different because the timestamp changed. Zero cache hits. The team pays for 10,000 full system prompt reads per day.

The fix: move the dynamic fields after the breakpoint.

system_prompt = STATIC_INSTRUCTIONS  # 3,000 tokens, with cache_control breakpoint
user_message = f"Priority: {ticket.priority}\nTime: {ticket.created_at}\n\n{ticket.text}"

Now the system prompt is stable, the breakpoint hash matches on every subsequent call, and each request after the first reads from cache. At 3,000 tokens x 10,000 daily calls x $3.00/MTok, the uncached cost is $90/day. With 90% savings on cache reads, it drops to around $9.

For workflows built in Chase Agents, this structure emerges naturally. Each step’s system prompt is versioned independently of its runtime inputs: ticket metadata flows in as user content rather than as modifications to the instructions block. The boundary is structural by design, not something teams have to audit manually.

Three prompt architectures ranked by prompt caching efficiency

Dynamic data in the prefix produces a 0% hit rate. Any runtime-variable content (IDs, timestamps, per-user context) injected before or within your static instructions resets the cache hash on every call. This is the most common mistake, and it’s completely silent.

The baseline fix is moving dynamic data after the breakpoint. Put the breakpoint at the end of your static system instructions. Runtime data goes in the user message. Cache hit rate climbs to near 100% for any two calls within the 5-minute window. (The 1-hour TTL option costs 2x the write price, worth it for workflows where calls come less than every 5 minutes.)

Maximum efficiency comes from multiple breakpoints for content that changes at different frequencies. Anthropic supports up to 4 cache breakpoints per request. Tool definitions rarely change, so a breakpoint there makes sense. System instructions might change weekly but not between same-day calls, so another breakpoint there. For multi-turn conversations, a third breakpoint moves with the conversation history. Each breakpoint is independent, and adding more costs nothing if the content behind each one stays stable.

Parallel workflows have a specific timing trap: a cache entry is only available after the first response begins. If you fan out five simultaneous LLM calls with the same system prompt, all five may miss the cache because none waits for the first write to complete. If you need deterministic cache hits in a parallel Chase Agents workflow, run the first LLM step before fanning out to dependent branches; the cache entry exists by the time the subsequent steps fire.

Where prompt caching breaks down

Caching saves nothing for short prompts. The minimums are real limits, and the API won’t tell you when you’ve missed them. For workflows with system prompts under 2,000 tokens on Sonnet 4.6, caching simply doesn’t apply.

Low-volume automations hit the TTL problem. If an automation runs twice an hour, the 5-minute cache has already expired between runs. The 1-hour option helps here, but for workflows running a few times per hour, the math often doesn’t favor paying 2x for cache writes.

OpenAI’s automatic caching is convenient but inconsistent. The ngrok engineering team found roughly 50% hit rates in production testing even for identical repeated requests. For workloads where latency predictability matters, Anthropic’s explicit breakpoints produce deterministic results. The same test showed 100% hit rates when the prefix was genuinely identical.

The 5-minute TTL also creates a cold start problem for workflows that spike. A system prompt cached at noon is cold by 12:06. If your workflow has variable traffic, the first call after a quiet period always pays full price to re-warm the cache.

A prediction about where this is going

Right now, most teams discover the timestamp problem the same way: a billing surprise followed by an audit of the usage fields. The tooling to surface this automatically (a dashboard metric for cache hit rate per workflow step, or a warning when cache_creation_input_tokens never drops toward zero over a multi-day run) doesn’t exist yet in most platforms.

Within 12 months, expect this to be a first-class observability metric, the same way rate limit errors eventually became visible in dashboards. Until then, the fastest diagnostic is a one-line logging statement after every LLM call that prints cache_read_input_tokens. If it’s zero for calls you expect to hit, you’ve found your timestamp.