Caching, from browser to database
Six layers of cache stand between a click and a disk: the browser, the edge, the gateway, the app, Redis and the buffer pool — plus the part nobody solves, invalidation.
Your browser
The fastest request is the one that never leaves your machine. Every layer you are about to see exists for one reason: to answer a question without disturbing the layer behind it. Caching is the whole reason the web feels instant.
Memory and disk cache
Your browser keeps images, scripts and fonts on disk, and the most recent ones in memory. A cache control header from the server says how long each file may be trusted, which is why a second visit can load with almost no network at all.
Service worker cache
A service worker goes further. It is a small script sitting between the page and the network that can answer from its own store, which is how a web app opens instantly on a train with no signal and no server in sight.
DNS resolver cache
Even the address lookup is cached. Your machine, your router and your provider each remember which numeric address a name maps to, for as long as the record's time to live allows. Move a domain and the old answer can linger for hours.
CDN edge cache
A content delivery network keeps copies of your files in hundreds of cities. The request stops at whichever one is nearest, so a visitor in Sydney is served from Sydney, turning a round trip of two hundred milliseconds into about ten.
API gateway
At the gateway the cheap tricks run out. This is the first machine that knows who you are, and a reply built for one account can never be handed to another. Gateways still cache the boring parts: a token checked once can be trusted for a minute rather than re-verified on every single call.
Application server
The application server does the real work. Rendering one page might mean twenty separate lookups, and with no caching each of those becomes a database query for every single visitor, multiplied by everyone who arrives in the same second.
In process cache
So the process keeps a small cache inside its own memory. Reads take nanoseconds because nothing is serialised or sent anywhere. The catch is that each server holds its own copy, so ten servers can cheerfully hold ten different answers.
Redis shared cache
Redis fixes that by giving every server one shared memory store. A lookup now crosses the network but still returns in well under a millisecond, and a single Redis node will comfortably serve over a hundred thousand reads a second.
Database
Only now does anything reach the database, and even here it may never reach a disk. A database is itself an elaborate stack of caches, which is worth remembering the next time somebody describes a cache miss as expensive.
Database buffer pool
The buffer pool is a slab of memory holding recently used pages. On a healthy server ninety nine reads in every hundred are answered from it without touching storage. Giving a database more memory is often the cheapest tuning there is.
Disk and page cache
Beneath that sit the operating system's own page cache and finally the drive itself. A solid state disk answers in roughly a hundred microseconds. The spinning disks it replaced took ten milliseconds, about a hundred times longer.
The write path
Writes are where the design gets interesting. Write through updates the cache and the database together and stays correct but slow. Write behind answers immediately and saves later, which is faster and occasionally loses your data.
Invalidation
Now the hard part. A price changes, and out there sit copies of the old one in a browser, an edge node in Frankfurt, two application servers and Redis. Knowing exactly which copies are now wrong, and reaching them, is the unsolved half of caching.
Stampede protection
Expiry has a nasty edge too. When a popular key dies, a thousand requests miss in the same instant and stampede the database together. The fix is a lock so one request refills while the others wait, or refreshing quietly just before expiry.
Hit ratio and evictions
So you measure two numbers above all: the hit ratio and the eviction rate. Going from ninety hits in a hundred to ninety nine sounds like a small nine point gain, but the traffic actually reaching your database drops to a tenth. That is the entire game.
Watch it explain itself
Every step above is narrated aloud. Play it, or open it in the editor and make it yours — no account needed.