Caching Layers: From HTTP to the Database

Caching is not one optimization placed in front of a slow function. A production request may pass through a browser cache, a CDN, a reverse proxy, an application process, Redis, a query cache, and the database buffer pool. Each layer can remove work, but each also holds a partial copy of reality under a different key, lifetime, and invalidation mechanism.

The difficult question is therefore not “should this be cached?” It is “which layer owns freshness for this representation, and what happens when that layer is wrong or unavailable?” A useful design starts with the read path, gives every copy an explicit consistency contract, and treats invalidation, observability, security, and tests as part of the feature.

One request, many caches #

The layers serve different scopes. Putting the same object everywhere rarely helps: a browser cache is private and close to one user, while a distributed cache is shared by many application instances. The database buffer cache stores pages, not business objects, and may already eliminate physical I/O even when an SQL query still runs.

The upper layers avoid network and application work. The lower layers reduce computation and storage I/O. A CDN hit can remove the entire origin path; a buffer-pool hit still pays for a network round trip, query planning or lookup, locks, and row decoding. Measure saved work, not merely whether some layer reported a hit.

For a single layer, the basic ratio is:

But a 99% hit ratio can be disappointing if the missing 1% contains the most expensive requests, or dangerous if hits serve stale authorization data. Track latency and origin work by result type as well as the aggregate ratio.

HTTP caching is a protocol #

HTTP already defines freshness and validation. A versioned asset such as /assets/app.4f21c.js can be cached for a year because changing bytes produces a new URL:

HTTP/1.1 200 OK
Cache-Control: public, max-age=31536000, immutable
Content-Type: text/javascript; charset=utf-8

Mutable HTML needs a different policy. no-cache does not mean “do not store”; it permits storage but requires revalidation before reuse. An ETag lets a browser or CDN ask whether its copy is still current:

GET /articles/caching HTTP/1.1
If-None-Match: "article-42-v17"

HTTP/1.1 304 Not Modified
ETag: "article-42-v17"
Cache-Control: public, max-age=60, stale-while-revalidate=300
Vary: Accept-Encoding

max-age=60 makes the response fresh for 60 seconds. For the next 300 seconds, stale-while-revalidate allows a cache to return the old response immediately while refreshing it in the background. This changes the contract from “always current” to “normally within five minutes, with low tail latency.” It is excellent for public articles and often unacceptable for balances, permissions, or inventory checkout.

Vary makes selected request headers part of the HTTP cache key. Use it deliberately: Vary: Accept-Encoding has a small bounded cardinality, while Vary: User-Agent or an unrestricted locale header can fragment the cache. Responses containing user-specific data should generally be private; secrets and one-time responses may require no-store.

CDNs and reverse proxies need a trustworthy boundary. Do not cache a response merely because the path looks public. The origin should declare cacheability, and shared caches must not use Authorization or Cookie responses unless the policy and key explicitly isolate them. Surrogate keys such as article:42 can group several URLs for targeted purge without flushing the entire edge.

Keys, TTLs, and cache-aside #

A cache key should identify the exact representation and its schema. Include stable dimensions that change the result, normalize them, and version the key when serialization or meaning changes:

type ArticleView = {
  id: string;
  tenantId: string;
  locale: "en" | "vi";
  title: string;
  bodyHtml: string;
  version: number;
};

function articleKey(input: {
  tenantId: string;
  articleId: string;
  locale: ArticleView["locale"];
}): string {
  const tenant = encodeURIComponent(input.tenantId.toLowerCase());
  const article = encodeURIComponent(input.articleId);
  return `article-view:v3:${tenant}:${article}:${input.locale}`;
}

The v3 namespace makes a deployment safe when old JSON is incompatible. Avoid keys derived from raw SQL, unstable object serialization, or unbounded attacker-controlled text. Hash long canonical input and retain enough structured prefix for operations.

TTL should come from a freshness budget, not a round number. If source changes occur at rate change_rate per second and a copy lives for ttl_seconds, a rough probability of at least one change during that lifetime is:

The estimate assumes independent changes, but it forces a useful conversation: how often does the value change, and how long may readers observe the previous value? Add randomized jitter so thousands of keys do not expire simultaneously.

Cache-aside keeps the database authoritative: read cache, load on miss, then populate. The implementation also needs validation, bounded TTL, and safe failure behavior:

async function getArticle(input: {
  tenantId: string;
  articleId: string;
  locale: ArticleView["locale"];
}): Promise<ArticleView | null> {
  const key = articleKey(input);

  try {
    const cached = await redis.get(key);
    if (cached !== null) return parseArticleView(cached);
  } catch (error) {
    metrics.increment("cache.read_error");
    logger.warn({ error, keyPrefix: "article-view:v3" }, "cache read failed");
  }

  const article = await repository.findArticle(input);
  if (article === null) {
    await redis.set(key, JSON.stringify({ missing: true }), { EX: 15 });
    return null;
  }

  const view = renderArticle(article);
  const ttlSeconds = 300 + Math.floor(Math.random() * 60);
  await redis.set(key, JSON.stringify(view), { EX: ttlSeconds });
  return view;
}

The short-lived missing marker is negative caching. It protects the origin from repeated lookups for absent IDs, crawler mistakes, and some enumeration traffic. Its TTL should be short because a resource may soon be created. Use a tagged union in real code so a legitimate value can never be confused with the sentinel.

Invalidation, stampedes, and consistency #

TTL-only invalidation is simple but bounds staleness only by the TTL. Delete-on-write is fresher: commit the database transaction, then delete affected keys. Never delete before commit, because a concurrent reader can repopulate the old value before the write becomes visible. Even delete-after-commit has a failure gap between the database and cache operation.

Common strategies make different promises:

For strong requirements, avoid serving the cache on the critical read, or tie reads to a database version. For read-your-writes, return the committed object directly, carry a minimum version token, or temporarily bypass replicas and caches. “Eventually consistent” is incomplete unless it states a maximum expected delay and behavior during event backlog.

A hot key expiring can cause a cache stampede: hundreds of requests miss together and all query the database. Request coalescing allows one loader per key while others await it:

const inFlight = new Map<string, Promise<ArticleView | null>>();

async function coalescedLoad(
  key: string,
  load: () => Promise<ArticleView | null>,
): Promise<ArticleView | null> {
  const existing = inFlight.get(key);
  if (existing) return existing;

  const pending = load().finally(() => inFlight.delete(key));
  inFlight.set(key, pending);
  return pending;
}

This protects one process. Cross-process protection can use a short distributed lease, but lock expiry and owner failure make it more complex. Often stale-while-revalidate is safer: keep a soft expiry for refresh and a hard expiry for rejection, allow one worker to refresh, and let other callers receive stale data. Add TTL jitter, warm only proven hot keys, and cap refresh concurrency so recovery traffic cannot overwhelm the database.

Database caches, operations, and tests #

Application teams often add Redis before checking the query plan. A database buffer pool already keeps frequently accessed table and index pages in memory. A query that is slow with warm pages may need a better index, lower row count, or less decoding rather than another cache. Conversely, a query-result cache can help expensive stable aggregates, but broad invalidation may erase its value.

Observe every layer separately. Record hit, miss, stale, negative_hit, load_error, and refresh_error; latency for cache lookup and origin load; bytes and item count; eviction rate; key age; coalesced waiters; and database work avoided. High eviction with low memory utilization may indicate an item-count limit. A rising hit ratio alongside rising latency may mean oversized values or a hot-key network bottleneck.

Logs must not expose complete keys when they contain identifiers, and cached values need the same encryption, access control, retention, and deletion handling as their source. Prevent cache poisoning by validating upstream responses before storage. Bound key and value sizes, authenticate cache servers, isolate environments and tenants, and never treat a cache as the only copy of durable data.

Test behavior, not the presence of a Redis call:

it("coalesces misses and invalidates only after commit", async () => {
  repository.findArticle.mockResolvedValue(article);

  const [first, second] = await Promise.all([
    service.get(input),
    service.get(input),
  ]);

  expect(first).toEqual(second);
  expect(repository.findArticle).toHaveBeenCalledTimes(1);

  await service.update(input, { title: "New title" });
  expect(transaction.commit).toHaveBeenCalledBefore(redis.del);
  expect(redis.del).toHaveBeenCalledWith(articleKey(input));
});

Use a fake clock to test fresh, soft-expired, and hard-expired states. Add concurrent miss tests, cache timeout and malformed-value tests, negative-cache creation races, tenant-isolation cases, and an integration test against the real cache client because mock command signatures can hide production errors. Load testing should include a cold cache, mass expiry, a hot key, and cache failure while origin limits are active.

Takeaways #

• Place data at the highest layer that can safely represent its scope; higher hits remove more downstream work.
• Treat HTTP directives, key dimensions, TTLs, and invalidation order as a consistency contract.
• Use versioned, bounded, tenant-aware keys and short negative caching for genuinely absent values.
• Prevent stampedes with jitter, request coalescing, stale-while-revalidate, and strict origin limits.
• Remember that query caches and database buffer pools solve different problems; inspect plans and warm behavior first.
• Measure freshness and work avoided, protect cached data like source data, and test expiry, concurrency, failure, and isolation.

A cache is successful when it makes the system cheaper and faster while preserving an explicit correctness boundary. Without that boundary, every new layer increases speed in the happy path and uncertainty everywhere else.