Mobile first indexing: how to check the version Google actually keeps

Mobile first indexing means Google indexes and ranks the mobile version of your page, so the copy worth inspecting is the one a phone receives, not the one on your monitor. This chapter is a two-fetch procedure: request the same address twice, once as Googlebot Smartphone and once as Googlebot Desktop, then compare the four things Google's own checklist names. One page takes about a minute.

What you need before this chapter

You need a page address, a terminal with curl, and Python 3 for the comparison step. If you would rather see what a set of crawlers get back without opening a terminal, the AI crawler check does the requesting for you. One question decides whether this chapter applies to your site at all: if the page assembles its main content in the browser, delivered HTML is not the right thing to compare, and that case is handled in can AI crawl JavaScript.

What changed, and the date it finished

The migration is over. Google's blog post of Monday, June 3, 2024 says "the small set of sites we've still been crawling with desktop Googlebot will be crawled with mobile Googlebot after July 5, 2024", and that "After July 5, 2024, we'll crawl and index these sites with only Googlebot Smartphone" (Mobile-indexing-vLast-final-final.doc, read 2026-09-06). The same post gives the consequence without hedging: "If your site's content is not accessible at all with a mobile device, it will no longer be indexable."

Two details keep confusing people afterwards. The first is that Googlebot Desktop did not disappear: "You may still find Googlebot Desktop in your server logs and reporting. For example, among a few other Search features, Googlebot Desktop is sometimes used when crawling for product listings and Google for Jobs." The second is that you cannot steer the two crawlers apart. Google's crawler documentation is explicit — "both crawler types obey the same product token (user agent token) in robots.txt, and so you cannot selectively target either Googlebot Smartphone or Googlebot Desktop using robots.txt" (Googlebot, read 2026-09-06).

Why mobile first indexing turns a desktop-only element into a missing one

Google keeps one copy of the page, and for most sites that copy comes from the smartphone crawler: "Google uses the mobile version of a site's content, crawled with the smartphone agent, for indexing and ranking" (Mobile-first indexing best practices, read 2026-09-06). Whatever your desktop template renders and your mobile template does not is therefore not a quieter version of that content. It never arrived.

The page Google keeps is the page the phone got. A block that exists only on desktop is not a weaker signal — it is an absent one.

That reframes a sentence teams say casually. "Our mobile template is a little lighter" is a description of everything Google has, not of a trimmed secondary edition sitting next to the real one. The same goes for structured data dropped from the mobile head to save weight, alt attributes emptied by an image component that only runs on small screens, and a noindex that one of the two templates emits and the other does not.

The seven things Google asks you to keep identical

Google publishes the list, and it is short enough to work through by hand. Each row below pairs one published instruction with the thing you compare between your two fetches, and with the shape a failure takes when you find one.

Three of those rows deserve their published wording. The image row reads "Provide high quality images. Don't use images that are too small or have a low resolution." The lazy-load row is narrower than people remember — the instruction is "Don't lazy-load primary content upon user interaction", so an image that loads as it scrolls into view is not what that sentence is about. And the last row, "Let Google crawl your resources", is the one that does not live inside the page at all; you check it by reading robots.txt, not by diffing HTML.

How to run a mobile first indexing check

Seven steps, each with something you can look at to know it worked. Do one page end to end before you scale it, then repeat on one page per template rather than on every page you own.

1. Pick a page that earns something — a product page, a documentation page, a post that already ranks. It should return HTTP 200 in an ordinary browser before you start.
2. Make an empty directory and work inside it. Both files will be large, and you will want them all in one place to delete afterwards.
3. Fetch the address twice with the two documented user agent strings, following redirects. You are done with this step when two non-empty files sit on disk.
4. Compare visible text volume first. It is the coarsest measure and it catches the largest failures; the two numbers should land within a few percent of each other.
5. Count the structured data blocks in both files. Identical counts pass, including a count of zero on both sides.
6. Read the robots meta tag out of both. Identical strings pass, and so does the tag being absent from both.
7. Count the images carrying a non-empty alt attribute in both. The two ratios should match, and the total on each side should match too.

The check itself, in one block

Copy the whole block. It saves two files and prints four rows, and the last column of each row says same or DIFFERENT. DIFFERENT is where you start reading.

# 1. The two documented user agent strings, verbatim from Google.
#    W.X.Y.Z is Google's own placeholder for the Chrome version.
UA_M='Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)'
UA_D='Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36'
URL='https://example.com/the-page-you-care-about'

# 2. Fetch the same address twice. -L follows redirects, so a hop to a
#    separate mobile host is followed instead of silently saved.
curl -sSL -A "$UA_M" "$URL" -o mobile.html
curl -sSL -A "$UA_D" "$URL" -o desktop.html

# 3. Compare the four things Google's checklist names.
python3 - <<'PY'
import re
def read(p): return open(p, encoding='utf-8', errors='replace').read()
def text(h):
    h = re.sub(r'(?is)<(script|style|noscript|template)\b.*?</\1>', ' ', h)
    h = re.sub(r'(?s)<!--.*?-->', ' ', h)
    return len(re.sub(r'\s+', ' ', re.sub(r'(?s)<[^>]+>', ' ', h)).strip())
def ldjson(h): return len(re.findall(r'(?i)application/ld\+json', h))
def robots(h):
    m = re.search(r'(?is)<meta[^>]+name=.robots.[^>]*content=.([^"\']*)', h)
    return m.group(1).strip() if m else '(none)'
def alts(h):
    t = re.findall(r'(?is)<img\b[^>]*>', h)
    return '%d of %d' % (sum(1 for x in t if re.search(r'(?is)alt=.[^"\'>]', x)), len(t))
m, d = read('mobile.html'), read('desktop.html')
for name, fn in [('visible text chars', text), ('ld+json blocks', ldjson),
                 ('robots meta', robots), ('img with alt', alts)]:
    a, b = fn(m), fn(d)
    print('%-19s %-26s %-26s %s' % (name, a, b, 'same' if a == b else 'DIFFERENT'))
PY

# What a passing run looks like. astro.build, 2026-09-06:
# visible text chars  6761                       6761                       same
# ld+json blocks      0                          0                          same
# robots meta         max-image-preview:large    max-image-preview:large    same
# img with alt        34 of 43                   34 of 43                   same

Those four lines at the end are one real run against one site on one day, and that is the whole of what they prove: the commands work and a passing result looks like this. They are not a survey, and nothing about them describes anybody else's site. Run the block yourself and the numbers you get are the ones that matter.

What this check cannot tell you

Three limits. The third is the one that gets misread as a diagnosis.

1. Sending the Googlebot Smartphone user agent string is not the same as being crawled by Googlebot. It shows what your server hands to that string, which is a fact about your server, not about Google's renderer. Anyone can send that string. Proving that a request really came from Google runs the other direction and is a separate procedure — how to verify Googlebot.
2. The commands do not execute JavaScript, so they compare delivered HTML rather than the rendered page. If your content is assembled in the browser, both fetches under-report by the same amount and the parity result stays green while telling you nothing; you need a rendering check instead of this one.
3. A difference found here is not the reason a page is not ranking. The check tells you a difference exists between two responses. Whether that difference costs you anything is a question this procedure does not answer, and treating a red row as the explanation for a traffic drop will send you rewriting a template that was never the problem.

Three ways the result lies to you

All three produce a clean-looking table that is not describing what you think it is describing.

1. You compared the redirect, not the page. Drop -L and curl saves the redirect body — often a few dozen characters — instead of the document. The text volume row then reports a difference that has nothing to do with your content, or agreement between two empty files.
2. An edge filter answered instead of your site. Some configurations return a challenge page to anything announcing itself as a crawler. Both files come back small and similar, the table reports parity, and the parity is between two pages that are not yours. Open one file and look at it before you trust any row.
3. The page is larger than what gets fetched. Google states that "Googlebot crawls the first 2MB of a supported file type". Two templates can agree perfectly past a boundary that Google has told you it stops at, so on a very heavy page a matching pair of numbers is a weaker result than it looks.

Common questions

Is there a mobile first indexing checker?

The block above is one, and it runs against any page without an account. A tool that returns a status is less useful here than a difference, because the answer you want is not pass or fail — it is which of the seven rows moved, so that you know which template to open.

Does Google still crawl with Googlebot Desktop?

Yes, in a minority of cases. Google says you may still find it in server logs, and names product listings and Google for Jobs as examples, alongside a few other Search features. Seeing it in your logs does not mean your site was left out of the migration.

Can I allow Googlebot Smartphone and block Googlebot Desktop?

No. Both obey the same product token in robots.txt, so a rule written for one applies to the other. If you want different behaviour for the two, robots.txt is not the file that will give it to you.

My mobile page shows less text on purpose. Is that a problem?

It is the exact case Google's instruction covers: "Make sure that your mobile site contains the same content as your desktop site." Collapsing text behind an accordion is a layout decision and the text is still in the HTML. Rendering a shorter string on small screens is a content decision, and it is the shorter string that gets indexed.

What do the mobile first indexing best practices come down to?

Mobile desktop content parity, plus two things that are not content: the same robots meta tags, and resources Google is allowed to crawl. The seven-row table above is the published list in the order it is easiest to check.