Series: HTTP lecture 4/6 (intermediate). For people who can already read a curl -v exchange and set headers, and now need to decide how long browsers and CDNs may keep a response. By the end you will set Cache-Control, ETag and Last-Modified correctly, answer conditional requests with 304 Not Modified, and spot the three bugs that make caching either useless or dangerously stale.

This lecture is for the developer who gets one of two tickets: "the app downloads the same 180 KB catalog on every launch" or "users still see yesterday's file after the deploy". Both are caching problems, and both come down to a handful of response headers. Everything below was run for real on an Apple Silicon Mac (M1 Pro) with curl 8.7.1, PHP 8.5.6 (php -S) and Python 3.9.6 (http.server), all on 127.0.0.1, plus a few read-only requests to example.com and to this site.

The two questions a cache asks

A cache is any copy of a response kept for later: the browser's disk cache, a CDN edge, a reverse proxy, or your app's own HTTP client. Before reusing a stored response it asks two questions.

  1. Is it still fresh? The server says how long a response may be reused without asking, mostly with Cache-Control: max-age=<seconds>. While the copy is younger than that, the cache answers on its own and the server never sees a request.
  2. If it is stale, has it changed? A stale copy is not thrown away. The cache sends a conditional request: the same GET plus If-None-Match: <the ETag it stored> or If-Modified-Since: <the Last-Modified it stored>. If nothing changed, the server answers 304 Not Modified with no body, and the cache reuses its copy.

The values the server hands out for that second question are called validators. An ETag ("entity tag") is an opaque string in double quotes, such as "41455fbfcdb6b7a0", that changes whenever the content changes. Last-Modified is a date with one-second resolution. Keep that last detail in mind; it causes one of the bugs below.

[[fig:http-cache-flow]]

What a plain static server sends

Start with the servers you already have. We generated a 179,500-byte JSON file (1,500 products, random.seed(42)) and served it with Python's built-in server:

cd ~/demo/cache/site python3 -m http.server 18150 --bind 127.0.0.1 curl -sI http://127.0.0.1:18150/products.json 
HTTP/1.0 200 OK Server: SimpleHTTP/0.6 Python/3.9.6 Date: Thu, 12 Nov 2026 14:28:41 GMT Content-type: application/json Content-Length: 179500 Last-Modified: Thu, 12 Nov 2026 14:28:40 GMT 

There is a Last-Modified but no ETag and no Cache-Control. Python's server does answer If-Modified-Since, so a revalidation works:

$ curl -s -o /dev/null -w '%{http_code} %{size_download}\n' \ -H "If-Modified-Since: Thu, 12 Nov 2026 14:28:40 GMT" http://127.0.0.1:18150/products.json 304 0 $ curl -s -o /dev/null -w '%{http_code} %{size_download}\n' \ -H 'If-None-Match: "abc"' http://127.0.0.1:18150/products.json 200 179500 

PHP's built-in server sends even less for a static file: only Content-Type and Content-Length, no validator at all, so nothing can ever be revalidated.

Without Cache-Control, a cache is allowed to guess. RFC 9111 calls this heuristic freshness and suggests, as a typical value, 10% of the time since Last-Modified. A file last changed ten days ago may therefore be reused for about a day without asking. That is why "we never set any cache header" does not mean "nothing is cached". (Browser behaviour is described here, not run: curl itself has no cache.)

ETag and If-None-Match with a php -S router

Now we write the server side ourselves. Save this as router.php next to the site folder and start it with cd site && php -S 127.0.0.1:18151 ../router.php:

<?php // php -S 127.0.0.1:18151 router.php $path = parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH); $file = __DIR__ . '/site/products.json'; if ($path === '/good.json') { $body = file_get_contents($file); $etag = '"' . substr(hash('sha256', $body), 0, 16) . '"'; $lastMod = gmdate('D, d M Y H:i:s', filemtime($file)) . ' GMT'; header('Content-Type: application/json'); header('Cache-Control: no-cache'); // may store, must revalidate header("ETag: $etag"); header("Last-Modified: $lastMod"); $inm = $_SERVER['HTTP_IF_NONE_MATCH'] ?? ''; if ($inm !== '' && in_array($etag, array_map('trim', explode(',', $inm)), true)) { http_response_code(304); // no body exit; } echo $body; exit; } if ($path === '/bad.json') { // common mistake: ETag changes every time $body = file_get_contents($file); header('Content-Type: application/json'); header('ETag: "' . md5($body . microtime()) . '"'); $inm = $_SERVER['HTTP_IF_NONE_MATCH'] ?? ''; if ($inm === md5($body)) { http_response_code(304); exit; } echo $body; exit; } return false; // let php -S serve other files as-is 

The ETag is a hash of the body, so it changes exactly when the content changes and is identical on every server that holds the same file. The first request gets the full response:

HTTP/1.1 200 OK Host: 127.0.0.1:18151 Date: Thu, 12 Nov 2026 14:28:53 GMT Connection: close X-Powered-By: PHP/8.5.6 Content-Type: application/json Cache-Control: no-cache ETag: "41455fbfcdb6b7a0" Last-Modified: Thu, 12 Nov 2026 14:28:40 GMT 

Send the ETag back, quotes included, and the server answers with headers only:

$ curl -si -H 'If-None-Match: "41455fbfcdb6b7a0"' http://127.0.0.1:18151/good.json HTTP/1.1 304 Not Modified Host: 127.0.0.1:18151 Date: Thu, 12 Nov 2026 14:28:54 GMT Connection: close X-Powered-By: PHP/8.5.6 Content-Type: application/json Cache-Control: no-cache ETag: "41455fbfcdb6b7a0" Last-Modified: Thu, 12 Nov 2026 14:28:40 GMT 

Send it without the quotes (If-None-Match: 41455fbfcdb6b7a0) and you get 200 179500: the quotes are part of the value.

Note what no-cache does here. It does not forbid storing. It says "store it, but ask me every time before you use it". Combined with an ETag this is the safest setting for data that can change at any moment, because each reuse costs one small round trip instead of a full download.

Choosing Cache-Control: a decision table

Content Header What happens
Passwords, bank pages, one-time tokens Cache-Control: no-store Nothing is written to any cache.
API data that may change any time Cache-Control: no-cache + ETag Stored, revalidated on each use; 304 when unchanged.
HTML pages of a blog Cache-Control: public, max-age=600 + ETag Reused for 10 minutes, then revalidated.
Per-user responses (profile, cart) Cache-Control: private, no-cache Browser may keep it; shared caches (CDN) must not.
CSS/JS with a hash in the file name (app.3f9c1e.css) Cache-Control: public, max-age=31536000, immutable Reused for a year; a new deploy uses a new name.
Same file name that changes on deploy (app.css) short max-age or no-cache + ETag Long max-age here is the "users see old CSS" bug.
Different rule for the CDN than for browsers add s-maxage=<seconds> Shared caches use s-maxage, browsers use max-age.

[[fig:http-cache-decide]]

Measured: what a 304 saves

We timed the good.json endpoint with one curl process per request: 50 full downloads, then 50 revalidations with the right If-None-Match, repeated for three rounds. Then we did 10 + 10 requests with curl's --limit-rate 1M to imitate a slow mobile link (two rounds).

Case Status Body bytes per request Median time
Loopback, full download (3 × 50) 200 179,500 1.57 / 1.43 / 1.47 ms
Loopback, revalidation (3 × 50) 304 0 1.38 / 1.40 / 1.38 ms
1 MB/s limit, full download (2 × 10) 200 179,500 173.29 / 173.01 ms
1 MB/s limit, revalidation (2 × 10) 304 0 1.43 / 1.46 ms

The response headers were 252 bytes for the 200 and 262 bytes for the 304 (the 304 is not smaller in headers). Fifty full downloads moved 8,975,000 body bytes; fifty revalidations moved none.

Two honest readings of this table. First, on a fast link a 304 saves almost no time, because the round trip and process start dominate; the win is bandwidth and the user's data plan. Second, the server still did all the work: our router read the file and hashed it for every 304. If producing the body is expensive (a database query), compute the ETag from something cheaper, such as a version column or the file's content hash stored at deploy time.

[[fig:http-cache-304-time]]

Common mistakes: what actually happens, and the fix

1. An ETag that changes on every response. bad.json mixes microtime() into the hash. We fed each response's ETag into the next request, ten times:

bad.json x10: 200 200 200 200 200 200 200 200 200 200 

Every revalidation is a full download, and the caching headers are pure overhead. The same thing happens in real life with ETags built from inode numbers or timestamps when two servers behind a load balancer hold the same file. Fix: derive the ETag from the content (a hash) or from a version number you control.

2. Last-Modified has one-second resolution. We wrote a file, read its Last-Modified, wrote a different version within the same second, and revalidated against Python's server:

Last-Modified seen: Thu, 12 Nov 2026 14:29:52 GMT -> 304 actual file: {"version": 2} 

The server told us "not modified" while serving a changed file; the client keeps version 1. Build scripts that write and then quickly rewrite a file hit exactly this. Fix: send an ETag; when both are present, If-None-Match takes precedence over If-Modified-Since (RFC 9110).

3. Comparing ETags as plain strings. Our good.json compares exactly, so a client sending W/"41455fbfcdb6b7a0" would get a 200. W/ marks a weak ETag: "semantically the same, maybe not byte-identical", typically after compression. If-None-Match uses weak comparison, so a correct server ignores the W/ prefix. Exercise 1 below fixes the router.

4. Reading no-cache as "do not cache". Teams set no-cache on a login page believing nothing is stored. The response is stored. Use no-store for secrets.

Real headers: example.com and this site

Two read-only requests on the day of writing. example.com is served through a CDN:

HTTP/2 200 content-type: text/html; charset=utf-8 server: cloudflare last-modified: Thu, 05 Nov 2026 21:17:41 GMT age: 8757 cf-cache-status: HIT 

There is no Cache-Control and no ETag. age: 8757 says the edge had held its copy for about 2.4 hours. A revalidation with If-Modified-Since returned 304 with a 0-byte body. With only Last-Modified, the heuristic applies: the page was 580,303 seconds old, so 10% is about 16 hours of possible reuse.

This site (provider-specific headers removed):

HTTP/2 200 content-type: text/html; charset=UTF-8 vary: Accept-Encoding last-modified: Thu, 12 Nov 2026 04:13:31 GMT etag: W/"22d0-6af53ceb-1d219362a7cb89c4;gz" cache-control: public, max-age=600 

Interesting detail: without Accept-Encoding the ETag was weak (W/"…;gz"); with curl --compressed the same page came back gzip-encoded with a strong "…;gz". Sending the weak value back as If-None-Match still produced 304 with a 0-byte body, which is the weak comparison working as specified.

CDN pitfalls

A CDN is a shared cache, so everything above applies to it, plus three traps:

  • It caches what you did not mean to cache. Redirects and API responses can be stored at the edge, and a fix on the origin is invisible until the copy expires. We hit this with 301s; the full story is in Your Host's CDN May Be Caching API Responses and 301s.
  • Vary decides the cache key. A response that differs by Accept-Encoding or Origin must say so in Vary, or one client's version is served to another.
  • Purge is not instant everywhere. Prefer versioned file names for assets; then you never need to purge them.

In production: a cached 403 and a cached redirect loop

Two incidents on our own servers in November 2026 showed that the CDN in front of a shared host follows the headers you send, including the ones you forget to send.

A cached error. On 16 November, during a deploy, a post folder existed for a few seconds without its index.html. The server answered 403 in that window, and the CDN stored the 403 like any other response. Readers kept getting 403 for a correct page until the entry expired. The fix was one rule: never let a CDN or browser keep an error response.

Header always set Cache-Control "no-store, max-age=0" "expr=%{REQUEST_STATUS} >= 400" 

A cached redirect that became a loop. On 19 November we replaced a folder's index.php, which answered 301 → local_ko.html, with an index.html, and made local_ko.html redirect back to the folder. The old 301 had been sent with no Cache-Control at all, and the CDN was still serving it from cache: x-hcdn-cache-status: HIT, age: 855, and later age: 1899. Folder → file → folder: a redirect loop for anyone who hit the cached copy. Requests with a query string (?nocache=…) bypassed the cache and looked fine, which is exactly how this kind of bug survives a quick check. What we did:

  1. Checked x-hcdn-cache-status and age on the old URL before adding the reverse redirect.
  2. Kept both URLs serving the page, with no redirect, until the cached 301 was gone.
  3. Purged the CDN. The first purge did not clear that entry; a second purge of the right domain did.
  4. Marked redirects as not cacheable from then on:
Header always set Cache-Control "no-store, max-age=0" "expr=%{REQUEST_STATUS} -ge 300 && %{REQUEST_STATUS} -lt 400" 

Caching on purpose. The same week we put a 60-second server-side cache in front of the busiest JSON endpoints of a lottery API, because the host allows only 20 new database connections per second. Identical requests in a burst are answered from a file: one request rebuilds it while the others get the previous copy. In a local test, 8 simultaneous requests for an expired entry cost 1 database connection instead of 8. Each response carries X-Tlabo-Cache: HIT|STALE|WAIT|MISS, so one curl -I shows which path answered.

The lesson for this lecture: decide the caching of error responses and redirects explicitly. If you send no Cache-Control, someone else decides for you. The CDN side of the story is in When a CDN caches API responses and redirects.

Honest limits / version notes

  • Run: curl 8.7.1, PHP 8.5.6 built-in server, Python 3.9.6 http.server, on 127.0.0.1 only, Apple M1 Pro. Loopback times include starting one curl process per request. --limit-rate imitates bandwidth, not latency or packet loss.
  • Not run: any browser cache, service workers, stale-while-revalidate handling in a CDN. Browser behaviour is described from the specifications.
  • Specs: HTTP caching is RFC 9111 (June 2022), which replaced RFC 7234; ETag, Last-Modified and conditional requests are in RFC 9110. immutable is RFC 8246, stale-while-revalidate/stale-if-error are RFC 5861. The 10% heuristic is a suggestion in RFC 9111, not a rule every cache follows.
  • The two live-site observations are a snapshot from one day; header values change with configuration.

Exercises

1. Change good.json so that If-None-Match uses weak comparison: W/"x" must match "x", a list such as "old", "x" must match, and * must match.

2. A file was last modified ten days ago and is served with Last-Modified but no Cache-Control. Using RFC 9111's typical 10% heuristic, how long may a cache reuse it without asking, and what single header removes the guesswork?

Answers

1. Replace the in_array test with this function:

function etag_matches(string $etag, string $header): bool { if (trim($header) === '*') return true; $strip = fn(string $t): string => preg_replace('#^W/#', '', trim($t)); foreach (explode(',', $header) as $t) { if ($strip($t) === $strip($etag)) return true; // weak comparison } return false; } 

We ran it with five header values:

"41455fbfcdb6b7a0" => 304 W/"41455fbfcdb6b7a0" => 304 "old", "41455fbfcdb6b7a0" => 304 41455fbfcdb6b7a0 => 200 * => 304 

The unquoted value still gets 200, which is correct: it is not a valid entity tag.

2. About one day (10% of ten days). Send an explicit Cache-Control, for example max-age=600 or no-cache, and the cache no longer guesses.

Next

Originally published at tlaboratory.net. Every output in this post was copied from a real terminal run.