Caching strategies: where to cache, what to cache, and why invalidation goes wrong

01Cache at the right layer

Caching happens at several layers and each has a different reach and invalidation story. Caching the same data at multiple layers multiplies the invalidation problem, so choose deliberately rather than adding a layer whenever something feels slow.

The general principle is to cache as close to the consumer as the correctness requirements allow. A CDN edge cache eliminates the request entirely, which is unbeatable for performance and worst for invalidation control. An in-process cache is trivially invalidated but does not help other instances. The trade is always reach against control.

02Immutable content changes everything

The hardest invalidation problem disappears if content never changes. Content-hashed asset filenames mean the URL changes when the content changes, so the old URL can be cached forever and the new one is simply a different resource. No purging, no propagation delay, no stale content.

Apply this pattern beyond static assets wherever possible. Version the key rather than mutating the value - user profile data keyed by a version that increments on update, API responses keyed by a resource ETag. Invalidation becomes a write to a different key instead of a coordinated deletion, which is dramatically more reliable.

03The stampede that takes down your database

A popular cache entry expires. Two thousand concurrent requests find nothing in cache and all of them query the database simultaneously to recompute the same value. The database saturates, requests time out, and the cache never repopulates because every attempt fails. This is a cache stampede, and it typically strikes at peak traffic - precisely when the cache mattered most.

Three mitigations work and they compose. Lock so that exactly one request recomputes while others wait or serve stale. Serve stale content while refreshing in the background, so nobody ever waits on a cold recompute. And add jitter to TTLs so entries populated together do not expire together, which is how synchronised expiry happens in the first place.

• Add random jitter to every TTL - never expire a cohort simultaneously.
• Use a lock so one request recomputes and the rest do not pile on.
• Serve stale while revalidating in the background.
• Pre-warm known-hot keys after a deploy or a flush.

04Choosing an invalidation strategy

Time-based expiry is simple and always eventually correct, at the cost of serving stale data for up to the TTL. Event-based invalidation is precise and immediate, at the cost of needing every write path to know what it invalidates - and the failure mode is a write path someone added later that forgot.

In practice, combine them: event-based invalidation for correctness on the paths you know, plus a conservative TTL as a backstop for the paths you missed. The TTL is insurance against incomplete invalidation logic, and it will be needed, because invalidation coverage decays as the codebase grows.

05Never let the cache become required

A cache is an optimisation. If your application cannot serve requests when the cache is unavailable, it is not a cache - it is a database with no durability guarantees, and its failure is now your failure.

Test this deliberately. Take the cache offline in a staging environment under load and confirm the system degrades rather than collapses. Many teams discover their origin cannot handle even a fraction of full traffic, which means a cache restart is an outage. Knowing that in advance lets you plan capacity or add a warming procedure, rather than learning it during an incident.

06Measure hit rate per key pattern

An aggregate hit rate of 90% sounds healthy and can conceal that your most expensive query has a 20% hit rate while trivial lookups carry the average. Break the metric down by key pattern to find where caching is actually failing.

Watch for keys that are never hit twice - a sign the key includes something unnecessarily unique like a timestamp or a request identifier. That pattern consumes memory, adds latency, and returns nothing, and it is common enough to be worth checking for explicitly.