The cache control header on 27 homepages: 13 send no ETag and no Last-Modified

FIELD TEST · 2026-08-24 · 27 homepages · single crawl · response headers only

Sample and method: the same 30-site panel in use since 2026-08-15, each homepage fetched once on 2026-08-24 with a browser user agent. We recorded the response headers of the HTML document itself — Cache-Control, ETag, Last-Modified, Age, Vary — and nothing about subresources.

Twenty-three of 27 homepages send a cache control header. Thirteen send neither an ETag nor a Last-Modified, which means a crawler's conditional request has nothing to compare against and can never come back as a 304. Eight declare max-age=0. One sends its directives separated by a semicolon.

How we measured it

One HTTP request per site, no rendering, no retries. Inclusion was fixed before the run: status 200, body at least 10,000 bytes, body containing <body. Three of the 30 failed — stackoverflow.com and medium.com returned 403, www.reddit.com returned an 8,393-byte shell — leaving 27, the same three exclusions as previous batches.

We read what the edge returned to us, at one moment, from one location. A CDN can and does answer differently to a crawler in another region, and we cannot tell from a single fetch whether a header comes from the origin or from the cache in front of it. We did not send a second, conditional request to see who actually answers 304.

Google asks for one specific behaviour, and 13 sites cannot produce it

Google's crawl budget documentation, read on 2026-08-24, puts it as an instruction: "Use HTTP caching: Support 304 (Not Modified) HTTP status codes. If a page hasn't changed since Google last crawled it, returning a 304 code tells Google to reuse the cached version, saving your server bandwidth and resources."

A 304 needs a validator. The crawler has to be able to send back an If-None-Match with your ETag, or an If-Modified-Since with your Last-Modified. Without either header on the way out, there is nothing to send back.

The three that send both are developer.mozilla.org, www.framer.com and www.wikipedia.org. The 13 with neither include stripe.com, linear.app, www.notion.com, www.cloudflare.com, railway.com, substack.com, arstechnica.com, techcrunch.com and news.ycombinator.com.

A cache header describes how long something stays fresh. A validator is what lets a crawler ask whether it still is. Sites ship the first and forget the second.

Four homepages send no cache control header at all

stripe.com, slack.com, webflow.com and arstechnica.com return the HTML with no Cache-Control in the response. That is not the same as forbidding caching — it hands the decision to heuristics — but it does mean nobody downstream has been told anything.

webflow.com is the interesting one of the four: no Cache-Control, but it does send Last-Modified, and an Age of 72,331 seconds. Something in front of it has been holding that response for about 20 hours, on its own initiative.

max-age=0 is the most common value in the panel

Eight sites declare it: nextjs.org, vercel.com, www.notion.com, supabase.com, www.framer.com, www.netlify.com, github.com and news.ycombinator.com. Six of those pair it with must-revalidate, which is the standard framework default for a page that is rendered per request but cached at the edge.

Six sites separate the browser from the edge with s-maxage, and five add stale-while-revalidate. railway.com is the most explicit in the panel: public, max-age=86400, s-maxage=86400, stale-while-revalidate=604800. Two sites go the other way and send no-store: www.canva.com and www.theverge.com.

The one with a semicolon

news.ycombinator.com returns cache-control: private; max-age=0. We checked it twice on 2026-08-24, with two different clients, and got the same string both times.

MDN's Cache-Control reference, read the same day, states the rule plainly: "Multiple directives are permitted and must be comma-separated (e.g., Cache-control: max-age=180, public)." A semicolon is not a separator here. What each individual cache does with the malformed second half is not something we tested, and the practical stakes are low — the page is already private — but it is a good reminder that response headers are strings nobody validates for you.

What this means for you

The check takes one command and no tooling. Run it against your own page and read three lines.

curl -sI https://example.com/your-page | grep -i 'cache-control\|etag\|last-modified'

• Send a validator. An ETag alone is enough, and it is what makes the 304 Google asks for possible.
• Keep max-age=0, must-revalidate if the page is dynamic — it is honest, and it still allows conditional revalidation.
• Don't send no-store on a public page you want crawled often. It removes the cheap path for everyone.
• Don't assume your framework sent a validator. Five of the eight max-age=0 sites here send an ETag; three send none.

Caching only matters for pages a crawler reaches at all. If you are not sure yours are reachable, check whether AI crawlers can read your page before tuning headers, and see how to get a page indexed faster for the step that comes after a change.

Common questions

How did you measure this?

One request per homepage on 2026-08-24, browser user agent, and we kept the response headers of the HTML document. No conditional request, no second fetch, no subresources. Every value in this piece is a string returned by an edge server at one moment.

Does the cache control header affect rankings?

Not as a ranking input anyone documents. It affects crawl efficiency, which Google discusses in the context of large sites and crawl budget. For a 40-page site this is close to irrelevant, and we would rather say that than sell a header as an optimisation.

Should I set a long max-age on my HTML?

Only if the page really does not change, and even then a validator serves you better. One site in this panel declares 86,400 seconds on its homepage. That is a deliberate choice by someone who knows their release cadence, not a default to copy.

What is the difference between no-cache and no-store?

no-cache allows storage but requires revalidation before reuse. no-store forbids storing the response at all. Four sites here send the first and two send the second, which suggests the distinction is understood by the people shipping them. For the neighbouring question of who still sends a modification date, see the last modified header across the same panel.