AnswerManiac

Getting things ready

AnswerManiac

Getting things ready

Crawlers & control files

Google-Extended

Written by Faizan Babar·COO, AnswerManiac·Updated November 2026

A robots.txt token governing Gemini's use of your content, with no effect on Search.

Google-Extended is a robots.txt token that controls whether Google may use your content to train and ground its Gemini models. Crucially, it does not affect whether your pages appear in Google Search or in AI Overviews — those are fed by Googlebot, which is governed separately. Disallowing Google-Extended limits Gemini's use of your content without touching your search visibility.

This separation lets you opt out of Gemini training while staying fully present in Google Search and its AI Overviews. Teams that want to limit training exposure can disallow Google-Extended without harming their search or AI-Overview visibility.

As always, the key is not to conflate the training token with search access — blocking Google-Extended is safe for visibility; blocking Googlebot is not.

Key takeaways

  • Google-Extended is a robots.txt token, not a crawler, and it has no separate user-agent or IP of its own.
  • It controls whether content Google already crawled may be used for Gemini model training and grounding, including grounding in Vertex AI.
  • It does not affect Google Search, and AI Overviews are fed by Googlebot, so blocking Google-Extended won't remove you from Search or AI Overviews.

A control token, not a bot

Google-Extended is a standalone product token you place in robots.txt. There is no crawler called Google-Extended visiting your site and no distinct IP range. Google still crawls with Googlebot; the token only governs how the crawled content may be used. Specifically, it controls whether your content helps train Gemini models and is used for grounding in Gemini apps and Vertex AI.

This is the point people miss: Google-Extended does not affect your inclusion or ranking in Google Search. And AI Overviews and AI Mode are features inside Google Search that rely on Googlebot, not Google-Extended. So blocking Google-Extended keeps you out of Gemini training and grounding while leaving Search and AI Overviews completely intact. The only way to pull yourself out of Search (and the AI features built on it) is to block Googlebot, which almost no B2B site should do.

So the decision is narrow and low-stakes for visibility: do you want your content used to train Gemini and ground its answers, or not? Either way, your Google Search presence is unaffected.

  • User-agent: Google-Extended
  • Disallow: /

Google-Extended vs Googlebot

Google-ExtendedGooglebot
Is it a crawler?No (robots.txt token)Yes
ControlsGemini training + groundingGoogle Search crawling
Affects Search / AI Overviews?NoYes (feeds both)

Example

A client assumed that adding `User-agent: Google-Extended` / `Disallow: /` would hide them from Google's AI Overviews. It wouldn't. AI Overviews draw on Googlebot's index, so the rule changed nothing there. It only opted their content out of Gemini training and grounding. If they'd actually wanted out of AI Overviews, the only lever is Googlebot itself, and that would also nuke their organic Search traffic.

Frequently asked questions

Does blocking Google-Extended remove me from AI Overviews?

No. AI Overviews are part of Google Search and rely on Googlebot's index. Google-Extended governs Gemini training and grounding only.

Will Google-Extended hurt my search rankings?

No. Google states Google-Extended does not affect inclusion in Search. Your rankings depend on Googlebot, which the token doesn't touch.

Is there a Google-Extended user-agent in my logs?

No. It's a control token with no crawler of its own. Google fetches with Googlebot and applies the Google-Extended preference to how the content is used.

Want to know where you stand in AI answers?

We’ll show you which engines cite you, and which don’t.

Get a free AI citation audit