Soft 404: what 27 homepages return for a URL that does not exist

FIELD TEST · 2026-08-18 · 30 sites · single snapshot

Sample and method: the same 30 sites we used for the robots.txt, sitemap and homepage structure surveys. One request each to a path we invented, browser user agent, 2026-08-18, from an exit in Japan. Three sites refused us at the homepage, so counts are out of 27. Same crawl as four other posts in this batch.

A soft 404 is a page that does not exist but answers with a success code anyway. We asked 27 homepages for a path nobody has ever linked to. Twenty-three returned a real 404, two returned 200, one returned 401. So the status code is usually right. The payload is not: the median 404 body was 68,678 bytes, and one site sent 1,320,341 bytes to say a page was missing.

How we measured it

Fixed before the crawl, unchanged after.

1. Follow redirects from http://<site> to whatever the homepage finally is, and take the scheme and host from there.
2. Request /querywin-probe-20260818-this-path-does-not-exist once, with a desktop browser user agent, following redirects.
3. Record the final status code, the byte length of the body, the title, and whether the body carries a noindex tag.

One path per site, chosen so it could not collide with anything real. We did not test deep paths, and we did not test paths that used to exist and were removed, which is the common case in the wild.

What 27 homepages returned for a path that does not exist

Four different answers to the same request. Only one of them is the boring correct one.

Notion's 401 is defensible. It ships a noindex tag with it, so nothing about that response invites indexing, and an authenticated app returning 401 for an unknown path is honest behaviour.

The two sites that answered 200

Vercel and Linear both return a success code for a path that does not exist, and they do it in different ways.

Vercel redirects the unknown path into its login flow, which settles at 200 with a page titled "Login – Vercel" and a body of 476,961 bytes. Linear answers 200 directly with a 25,045-byte shell titled "Linear". Neither response says the page is missing in a way a crawler can read.

That is the definition Google works from. Its documentation on HTTP status codes says: "If the content suggests an error for Google Search, an empty page or an error message, Search Console will show a soft 404 error." The judgement is made on the content, because the status code has stopped being informative.

A correct 404 can still be expensive

Returning the right code is one decision. What you attach to it is another, and the sample is much worse at the second one.

Figma is worth following step by step, because the shape is common. Our request got a 308 that added a trailing slash, and the slashed URL then returned 404 with 1,320,341 bytes. Two round trips and 1.3 MB to communicate one fact: nothing here.

None of this is an indexing problem. It is a cost problem, and it lands on the crawler as well as on you.

Why this matters for crawl budget

Google's crawl budget documentation names soft 404s directly: "Eliminate soft 404 errors. soft 404 pages will continue to be crawled, and waste your budget." The same page makes the sibling point about redirects: "Avoid long redirect chains, which have a negative effect on crawling."

Both sentences were read on 2026-08-18 at the crawl budget guide on developers.google.com/crawling/docs/crawl-budget. The mechanism is the same in each case. A crawler has a finite number of fetches it will spend on you, and every fetch that resolves to nothing is one it does not spend on a page you wanted read.

A 404 tells a crawler to stop asking. A 200 on a missing page tells it to keep coming back forever.

What this crawl cannot tell you

We invented one path per site. A made-up URL may not take the same code path as a real link that broke — a removed article, a renamed product page, a stale sitemap entry. Those are where the problem usually starts, and we did not test them.

We also do not know how any of these sites appear in their own Search Console. That label is applied after Google looks at the content, and we have no access to that verdict for someone else's property. What we measured is the raw response, which is the input to that judgement, not the judgement itself.

What to do about soft 404s on your own site

Two checks worth running today, and two habits worth dropping.

• Request a path you know does not exist and read the status code with curl -I. If it is not 4xx, you have found the problem without opening any tool.
• Weigh your error page. If your 404 ships the full site shell, strip it down — the response has one job.
• Do not answer a missing page with a redirect to the homepage. That is a soft 404 with extra steps, and it costs a hop.
• Do not rely on a JavaScript router to render the error state. The status code is set before your code runs, and that code is what a crawler reads.

The same crawl produced four other measurements from the same 30 sites. How many hops it takes to reach these homepages at all is in the redirect chain survey, and whether their servers send any freshness signal is in the last modified header survey. Checking status codes across every page of a site, rather than one path on the homepage, is what QueryWin is being built to do.

Common questions

How did you measure this?

One HTTP request per site to an invented path, browser user agent, redirects followed, on 2026-08-18. We recorded the final status code and the byte length of the body. No rendering, no JavaScript, no second attempt.

What is a soft 404, exactly?

A URL that has no content but returns a success status code instead of 404 or 410. Google decides the label by reading the page, not by reading the header, because the header has already told it the wrong thing.

Is a 200 on a missing page actually harmful?

It wastes crawling that would otherwise go to your real pages, and Google's own guidance says such pages keep getting crawled. Whether it costs you rankings is a separate claim, and not one this crawl tested.

Should a deleted page return 404 or 410?

Either is read as gone. 410 is the stronger statement, 404 is the safer default if you might restore the page later. Both beat a 200.