Skip to content
build-3000
backend

Caching Layers

Answering repeat questions from memory - and knowing when memory starts lying.

Before this: Databases and CRUD

Objective

Add a cache to an expensive endpoint correctly: choose keys and TTLs, invalidate on writes, stay aware of stampedes, and prove the fast path without breaking correctness.

Vocabulary

The words to use when directing your agent.

cache
A fast temporary store of answers so expensive work is not repeated.
cache hit
The answer was found in the cache - no slow source touched.
cache miss
The cache did not have it; the real source was queried and the result stored.
TTL
Time to live - how long a cached answer may be served before it expires.
invalidation
Deliberately removing cached entries when the underlying data changes.
cache stampede
Many requests missing at once all rush the database for the same answer.

Understand it

Caches live in layers

Browsers cache assets, CDNs cache pages, your app can cache in process memory or a shared store like Redis. Each layer answers before asking the next. For this curriculum the interesting layer is application-level caching of expensive computations, because you control its keys and invalidation rules directly.

Invalidation and stampedes

Two classic failure shapes. First, forgotten invalidation: a write lands but old cached answers keep serving - users see ghost data. Second, the stampede: one hot key expires while a thousand requests are in flight, and all of them hammer the database for the same rebuild. The cures are deliberate: invalidate on write, and let only ONE request rebuild a missing key while others wait briefly.

Core concepts

Fast path, true source

The database stays the source of truth; the cache is a shortcut for repeated questions. Every cached read is a small bet that the answer has not changed. Correctness comes from choosing how long that bet is allowed to stand.

Staleness is a decision, not an accident

A TTL says: this answer may be up to sixty seconds old and that is acceptable. Deciding staleness budgets per kind of data - money totals tight, marketing copy loose - IS the engineering work. Undecided staleness becomes user-reported bugs.

Examples

The monthly totals endpoint

Aggregating a year of expenses on every page load wastes work: totals change only when expenses change. Cache the summary under a per-user key with a TTL measured in minutes, and invalidate it whenever that user writes an expense. Reads get fast; writes stay truthful.

Serving stale safely

Some systems prefer availability over perfect freshness: when the rebuild fails, serve the slightly-stale value with a visible 'as of' note instead of an error. Choosing this tradeoff explicitly is production thinking; stumbling into it accidentally is an outage report.

Common mistakes

Common mistake

Building cache keys without the user id, letting one user's private summary serve another.

Fix

Every key must carry the full identity of the question: user, resource, parameters. Test cross-user reads explicitly.

Common mistake

Caching without any expiry 'because nothing changes much'.

Fix

Every entry gets a TTL even a generous one. Forever-fresh caches are time bombs with long fuses.

Working with your AI agent

Before any cache code, make your agent fill out this table in writing: what is cached, exact key shape, TTL, invalidation trigger, and behavior on miss. Reviewing that table takes thirty seconds; reviewing tangled cache bugs takes evenings. Then demand log lines proving hits versus misses.

Build it

The module’s real project. Your agent builds; you direct and inspect.

Project: Fast Path

Give the Private Ledger a proper caching layer: the monthly-summary endpoint becomes dramatically cheaper on repeats, with per-user keys, explicit invalidation on writes, stampede-aware rebuilding, and logs that prove every claim.

Requirements

  • Monthly-summary endpoint served from cache keyed by user and month range with a configured TTL
  • Any expense write by the user invalidates their cached summaries immediately
  • On miss, exactly one rebuild runs; concurrent requests wait instead of piling onto the store
  • Log lines record HIT or MISS per request - counts derivable from logs
  • Cache behavior configurable via environment variables (TTL, enabled flag)

Context for your agent

Add an application-level cache layer to my Expense Ledger behind one small interface (get/set/invalidate) so the store implementation can swap later. Target endpoint: monthly expense summaries. Per-user keys including month parameters; TTL from env; write operations invalidate affected keys synchronously. Implement single-flight rebuilding so a cold key triggers exactly one recompute. Log HIT/MISS with user id and key shape but no payload contents. Show me the cache table (key/TTL/invalidation) BEFORE coding.

Verify it - twice

Check every item yourself first. Then hand your agent the prompt and compare verdicts.

Self-verified0/11
Functional
Architecture
Security
Testing
Production

Verification prompt - paste into your AI agent

Perform an adversarial audit of my Fast Path caching layer - treat cached answers as guilty until proven fresh. Verify PASS or FAIL with evidence: (1) the key includes every input that changes the answer - name any missing dimension, (2) a write genuinely invalidates affected entries - attempt to observe stale data after a write and report the maximum staleness window you can produce, (3) single-flight actually holds - describe how concurrent cold misses behave, (4) no cross-user leakage is possible through key manipulation - try it if you can run the code. Then answer: which money-total could a user legitimately see wrong, for how long, and would you ship it?

Run this against “Caching Layers” after your own checklist pass. Where the agent and you disagree, investigate before moving on.

Review questions

Answer out loud. If you cannot, revisit before moving on.

  1. State your cache table from memory: keys, TTLs, invalidation triggers.

  2. Why does a forever-TTL feel free until the day it is not?

  3. Walk through what a stampede is and the mechanism standing in its way here.

You can now

  • Add caching with explicit keys, TTLs and invalidation instead of hope
  • Reason about staleness budgets per kind of data
  • Prove cache behavior with logs rather than vibes

Up next

Rate Limiting and Abuse Control