304 Not Modified: make a re-crawl cost one header instead of a whole page

A 304 Not Modified is your server telling a crawler "nothing changed, keep what you have" in a response with no body at all. Send one and the re-crawl costs a header exchange instead of a full page render. Google supports this through two header pairs, recommends one of them, and publishes a number that says almost nobody is using it: 0.017% of its fetches are cacheable.

Before you start

This chapter sits at the end of a run about crawl volume. The chapters before it deal with where addresses come from — filter combinations, and page sequences in pagination SEO. This one leaves the count alone and works on the other half: what each repeat visit to an address you already have costs you. If you have not yet decided whether crawl volume is your problem at all, is crawl budget your problem is the prior question, and for many sites the answer is no.

What a 304 Not Modified actually saves

Two costs, and Google names both in the same breath. When a crawler holds a validator from a previous visit, it sends that value back on the next request. If the value still matches, your server answers with a status line and headers and stops there — no body, no render, no database read behind it.

The documented benefits are put in plain commercial terms: "your server doesn't have to spend compute resources on actually generating content; that is, you save money" and "your server doesn't have to transfer the HTTP body; that is, you save money" (Crawling December: HTTP caching, published 2024-12-09, read 2026-09-08).

The supported mechanism is narrow and worth stating exactly. Google's crawling infrastructure supports heuristic HTTP caching "specifically through the ETag response- and If-None-Match request header, and the Last-Modified response- and If-Modified-Since request header". Two pairs. Nothing else in the caching family stands in for them.

A 304 is the only response where doing less work is the correct answer.

The scale of the missed opportunity is in the same post, and it is the number worth carrying around: "10 years ago about 0.026% of the total fetches were cacheable, which is already not that impressive; today that number is 0.017%." That is Google's own share across the whole web, not a measurement of your site. It went down.

What 27 large homepages actually send

We read the response headers of 27 large homepages on 2026-09-08, one request each, desktop Chrome user agent. Only three of them send both validators — the configuration the documentation asks for.

Eleven of 27 send nothing a crawler could hand back on the next visit. Those are not broken sites — a homepage that changes several times an hour has a fair reason to skip validators entirely. The point is that the decision is usually not being made; the headers are whatever the framework emitted.

One more count, reported without a verdict attached. Eight of the 13 ETags on the panel are weak validators, sent with a W/ prefix. Google's documentation says to set the ETag "to any arbitrary ASCII string" and does not discuss the weak form at all, so we are not going to tell you it matters. It is a thing you will see in your own headers and wonder about. The fuller header picture from that panel is in the cache control header survey.

Do it: four steps

The first step is a measurement, not a change, and it decides whether the rest is needed. Each step has a signal you can check yourself.

1. Find out whether your server already answers 304. Fetch a page, keep the validator, ask again with it. Done when you have seen either a 304 status line or a full 200 for the second request — both are real answers, and the second one tells you there is work to do.
2. Add an ETag if there is no validator. Google's preference is explicit: "We strongly recommend using ETag because it's less prone to errors and mistakes (the value is not structured unlike the Last-Modified value)." A content hash or a build version both qualify. Done when a fresh fetch returns an ETag header and the same page returns the same value twice in a row.
3. Decide what counts as a change. This is a judgement Google explicitly hands back to you, with a worked example: "Our recommendation is that you require a cache refresh on significant changes to your content; if you only updated the copyright date at the bottom of your page, that's probably not significant." An ETag derived from the whole rendered page will change when a footer year changes, and that is the common failure. Done when you can name what goes into the hash.
4. If you also send Last-Modified, format the date exactly. The documentation gives both the rule and the shape: the date "must be formatted according to the HTTP standard", and to avoid parsing issues Google recommends "Weekday, DD Mon YYYY HH:MM:SS Timezone", giving Fri, 4 Sep 1998 19:15:56 GMT as the example. Done when your header matches that shape character for character.

What to ship: the check and the decision table

The whole verification is two requests. Run it against a page that does not change often, because a page that changes every minute is supposed to answer 200.

# 1. Get the validator
curl -sI https://example.com/some-stable-page \
  | grep -iE '^(etag|last-modified|cache-control):'

# 2. Hand it back and read the status line
curl -sI https://example.com/some-stable-page \
  -H 'If-None-Match: "PASTE-THE-ETAG-HERE"' | head -1

# Expect: HTTP/2 304
# If you get 200 with a full body, no conditional handling is in place

One optional extra, and the documentation is careful to mark it optional: "While not required, consider also setting the max-age field of the Cache-Control header to help crawlers determine when to recrawl the specific URL." The value is meant to be how long you expect the content to stay unchanged.

Three ways this goes wrong

All three produce headers that look correct from outside, which is why the two-request check is worth more than reading a config file. Two of them waste effort. The third quietly stops your updates from being picked up, and it is the one to check for first if a page you changed is still showing the old version in search.

1. Sending a validator without honouring it. The most common one, and the reason step 1 is a measurement. A framework emits an ETag, the conditional request arrives, and the application rebuilds and returns 200 anyway. From the outside the headers look right. Nothing is saved.
2. A validator that changes on every request. If the ETag is derived from anything that varies — a timestamp, a nonce, a randomised asset name — it will never match, and every conditional request becomes a full response plus wasted round trip. The way to catch this is to fetch the same unchanged page twice and compare the values.
3. A validator that never changes. The opposite failure and the worse one. Google's documentation notes that a 304 "will happen to every subsequent request until the preconditions fail to validate", so a frozen ETag means an updated page keeps answering "nothing changed". You lose updates rather than bandwidth.

Where this stops

Google publishes the mechanism and one global statistic. It does not publish what any of this is worth on a particular site, and neither can we.

1. How much crawl this buys you. There is no number, and the documentation's own framing is conditional: caching "may help your site be crawled more efficiently", especially on large sites with rarely-changing content. It is not stated as a promise, and we are not going to upgrade it into one.
2. Whether it affects rankings. Nothing in the documentation connects conditional responses to ranking, we have run no experiment, and on a live site the variable cannot be isolated. The saving is on your server bill and your crawl load.
3. What your CDN does with all of this. A cache in front of your origin can generate its own validator, strip yours, or answer the conditional request itself. Our measurement read what arrives at a client and cannot tell which layer produced it. If your headers change when you bypass the edge, that is the layer to go and read.
4. Whether other crawlers behave the same way. The documented behaviour above is Google's. We did not test what any AI crawler does with a conditional request, and none of them publish it in the same detail.

Frequently asked questions

What does 304 not modified mean?

It is a response saying the version you already have is still current, sent with headers and no body. A crawler that gets one keeps its stored copy, and your server skips generating the page.

Should I use ETag or Last-Modified?

ETag, on Google's stated preference, because its value is unstructured and therefore harder to get wrong than a formatted date. The same post adds that setting both is better if you have the option. On our panel only three of 27 homepages do.

Will a 304 make Google crawl my site more often?

Not something anyone has documented. The stated benefit is efficiency on your side — no page generation and no body transfer. Treat any promise of a higher crawl rate as unsupported.

My site sends an ETag but never returns 304. What is wrong?

Almost always the application, not the header. Emitting an ETag and comparing an incoming If-None-Match are two separate pieces of work, and many stacks do the first by default and the second never. The two-request check above distinguishes them in about ten seconds.

Does any of this help if crawlers cannot reach the page at all?

No, and that is the layer to check first. Conditional requests are an optimisation on top of access, so if a crawler is being refused before it ever sees a header, none of this applies. The AI crawler access check answers that question.