Promptwatch Logo

Google Feedfetcher

Feedfetcher is used for crawling RSS or Atom feeds for Google News and PubSubHubbub.
Feed Fetcher

What is Google Feedfetcher?

A person or product feature asks Google Feedfetcher to retrieve an RSS or Atom feed. Google uses it for Google News and WebSub, and it stores and refreshes feeds requested through apps or services. This is a user-triggered feed grabber, not the general Googlebot crawl of a website.

Its published HTTP user agent is FeedFetcher-Google; (+http://www.google.com/feedfetcher.html). Google also publishes IP data for user-triggered fetchers, which can help distinguish its traffic from a copied header. Requests should normally concentrate on feed URLs rather than move through unrelated pages.

Google explicitly says Feedfetcher ignores robots.txt because it acts on a user's request. That conflicts with the respectsRobotsTxt value preserved in this bot. A robots group is still a readable statement of preference, but it is not an effective denial for this fetcher.

Feedfetcher can keep feed data current in the Google products that requested it. Google does not assign it a generative search or model training role, and it is not a substitute for Googlebot. One of these requests does not establish that the linked article was indexed for Search or used by an AI system.

Not relevant for AI search

Is Google Feedfetcher relevant for AI search?

No. Google Feedfetcher is not part of AI search or training, so allowing or blocking it does not change your AI visibility.

Google Feedfetcher polls RSS, Atom, or podcast feeds to deliver your content to subscribers. It does not affect search or AI answers; it only serves the people who subscribed to your feed on that platform.

How to handle Google Feedfetcher

Use FeedFetcher-Google for log searches and any client-specific handling. The following robots.txt group is correctly addressed but Google says the fetcher will ignore it:

User-agent: FeedFetcher-Google
Disallow: /

When the feed should remain in Google News or a WebSub workflow, reduce cost through conditional responses, caching, or a simpler feed endpoint. Denying the client can leave a requested feed stale in those products.

For a real stop, Google recommends returning an error such as 404 or 410 to this user agent. Feed authorization or a server and CDN rule can also enforce access. Validate high-impact network rules against Google's current user-triggered fetcher ranges, since the header can be forged.

Examples

  • A news feed receives a new entry. Feedfetcher later refreshes that RSS URL for a Google product that requested the feed.
  • A site owner adds a disallow and the requests continue. After checking Google's documentation and source ranges, the owner returns 410 for `FeedFetcher-Google` at the server.

Frequently asked questions about Google Feedfetcher

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

A user action in an app or service requests the RSS or Atom feed. Google then stores and periodically refreshes common requested feeds.

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard