Google-Extended

Google-Extended is a robots.txt product token, not a separate crawler that identifies itself in HTTP requests. Google fetches pages with existing crawler user agents, then uses the Google-Extended group as a control over whether the crawled content may be used by specified generative AI products.

Google says the token covers training future generations of Gemini models that power Gemini Apps and the Vertex AI API for Gemini. It also controls grounding with content from the Google Search index in Gemini Apps and Grounding with Google Search on Vertex AI. A disallowed page is outside those Google-Extended uses.

This setting is independent of ordinary Google Search. Google states that a Google-Extended rule does not affect inclusion in Search and is not a Search ranking signal. Publishers can therefore keep pages available to Googlebot for search indexing while declining the uses covered by Google-Extended.

There is no Google-Extended user-agent string to find in an access log. Google's common crawler documentation says requests are made with existing Google user agents. A request from Googlebot cannot, by itself, reveal which later Google-Extended use might apply to the fetched content.

Google's common crawlers automatically obey robots.txt, and the directory records Google-Extended as respecting it. The control can cover the entire site or selected paths. It is a use preference for content Google crawls, not a firewall rule that blocks every Google request to those URLs.

Because Google-Extended is not an HTTP identity, a dedicated IP check or signature for that token would not make sense. Site owners who need to verify an underlying Google crawler should use Google's published crawler-verification methods, but should keep that identity question separate from the Google-Extended content-use policy.