Google-Extended and Applebot-Extended
Neither of these is a crawler. Nothing visits your site as Google-Extended or Applebot-Extended. They are names you write in robots.txt, and Google’s and Apple’s search crawlers read them to decide whether the pages they fetch may also train Gemini or Apple’s models. Closing them keeps you in Google Search and in Spotlight, Siri and Safari.
Google-Extended: Gemini, not Search
Google calls Google-Extended a standalone product token with no HTTP user agent of its own: the crawling is done by Google’s existing crawlers, and the token only says what may happen to what they fetched. It controls whether your content may be used to train future Gemini models, the ones behind the Gemini apps and Vertex AI, and for grounding, which Google describes as giving the model content from the Search index when it answers.
Google says Google-Extended does not affect whether your site is included in Google Search, and is not used as a ranking signal. To keep your pages out of Gemini’s training:
User-agent: Google-Extended Disallow: /
The token takes ordinary rules, so it can close part of a site and leave the rest, as in Google’s own example, which opens one archive page and closes the archive around it.
Applebot-Extended: Apple's models, not Apple's search
Apple says Applebot-Extended does not crawl webpages. Applebot does, for Apple’s search features, and Apple says what it crawls may also help train its foundation models. Applebot-Extended is how you opt out of that training, which covers the generative features of Apple Intelligence. Your pages stay findable in Spotlight, Siri and Safari.
User-agent: Applebot-Extended Disallow: /
Three things that catch people out
- Blocking the crawler instead of the token. User-agent: Googlebot with Disallow: / takes you out of Google Search. To refuse only Gemini’s training, the line is for Google-Extended. The same goes for Applebot and Applebot-Extended.
- Looking for them in your logs. You will not find them: Google says the token has no user agent of its own, and Apple that Applebot-Extended does not crawl. The requests come from Googlebot and Applebot.
- Applebot reads Googlebot’s rules. Apple says that when robots.txt does not mention Applebot but does mention Googlebot, Applebot follows Googlebot’s instructions. A file that closes Googlebot and says nothing of Applebot closes Apple’s search too.
Is it really Applebot?
Apple says Applebot’s requests come from hosts under applebot.apple.com, which a reverse DNS lookup shows, and it publishes the address ranges in applebot.json. Google-Extended has nothing to check: no request is ever made under that name.
Check what your robots.txt tells Google-Extended
Paste your file into the robots.txt validator, opened with Google-Extended filled in. It runs in your browser and shows, for Google-Extended and every other search and AI bot, whether it may come in and the exact line that decides. The list of AI crawlers has the others.
Other AI crawlers
Sources
Read on 28 October 2026; operators change these pages, so we read them again every month.
Last changed 28 October 2026.