Promptwatch Logo

MagnetmeBot

For Magnet. me customers, MagnetmeBot is part of the job-sync process.
Magnet.meMagnet.me-web
Aggregator

What is MagnetmeBot?

For Magnet.me customers, MagnetmeBot is part of the job-sync process. It discovers career pages so vacancies, recruitment events, and employer information on Magnet.me can be updated from the customer's own site.

The crawler follows links and scrolls, but it does not fill in forms or click buttons. It may load JavaScript, although Magnet.me warns that this is not guaranteed. The operator recommends canonical URLs, XML sitemaps, hreflang annotations, and structured data such as JobPosting so the crawler can identify the intended content.

The collected pages support Magnet.me's employment platform. The operator does not describe MagnetmeBot as a source for generative AI search or model training, so its traffic has no documented connection to AI citation visibility.

Logs show the HTTP user agent as Magnet.me-web/XX.XX, with the number changing by version. Robots rules use a different name, MagnetmeBot. Magnet.me documents support for Allow, Disallow, and Crawl-delay, and says the crawler adjusts its speed to avoid overloading a site.

Indirectly relevant

Is MagnetmeBot relevant for AI search?

Indirectly. MagnetmeBot has no AI product of its own, but its output can end up in the systems that AI answers draw on.

MagnetmeBot collects content from many sites and redistributes it through its own product. Aggregated copies can end up in datasets that AI systems later learn from, and some aggregators are themselves sources that AI answer engines draw on.

How to handle MagnetmeBot

Customers that want Magnet.me to keep their career content current should leave the relevant public pages crawlable. If load is the concern, set a longer Crawl-delay; if the crawler is wandering into unrelated areas, use path-specific rules.

Target the documented robots product token, not the versioned string in the HTTP header:

User-agent: MagnetmeBot
Disallow: /

This rule applies to the web crawler described on Magnet.me's bot page. A separate user agent, Magnet.me API, calls known API URLs and does not follow robots.txt. Check which agent appears in the request before assuming the same control covers both.

Examples

  • An employer publishes a vacancy with `JobPosting` data and sees `Magnet.me-web/1.0` fetch the canonical job page for synchronization.
  • A recruitment event is hidden behind a button, so the site exposes the details on a linkable page that MagnetmeBot can reach without clicking.

Frequently asked questions about MagnetmeBot

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

That is the crawler's versioned HTTP user agent. Magnet.me uses `MagnetmeBot` as the product token for robots rules.

Related bots

AirsoftdbBot

Airsoftdb is a search engine for airsoft prod.

Aggregator

Artsdata Crawler

Web crawler that collects publicly available LOD for arts and culture in Canada.

Aggregator

BestChange Bot

The BestChange bot downloads exchange rate information from 600 websites every 5 seconds.

Aggregator

BoardGamePrices Bot

Price comparison site for board games. Need to crawl store pages for participating stores. All stores give permission to be crawled.

Aggregator

Capital One Bot

Capital One Bot crawls dealer websites for getting the usage information for Capital One lead navigator button.

Aggregator

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard