What is Adsense?
Google's AdSense ads crawler visits pages that request ads from Google's systems. It processes and indexes the page content so AdSense can choose relevant advertising. Google says only URLs carrying its ad tags should generate these crawl requests.
That index supports ad delivery, not Google Search. Blocking the AdSense crawler can stop Google from understanding a page well enough to serve contextual ads, but it does not remove the page from organic search. There is also no documented connection between this crawler and AI answers or foundation-model training.
AdSense normally reads the page available to an unauthenticated visitor. Publishers who serve ads behind a login can configure crawler credentials in their AdSense account and verify the site through Search Console. Those credentials should be scoped to the intended pages rather than used to expose a private site generally.
The catalog stores Adsense as its matching token, but Google's official user agent for this crawler is Mediapartners-Google. That official token is the one to use in robots.txt. Google says the crawler honors specific robots rules, and a wildcard rule alone may not prevent it from visiting pages that contain ad tags.
