Promptwatch Logo

MagiBot

MagiBot is a Peak Labs crawler associated with research into information extraction and retrieval from natural-language material.
Peak LabsMagiBot
Search Engine Crawler

What is MagiBot?

MagiBot is a Peak Labs crawler associated with research into information extraction and retrieval from natural-language material. The record classifies it as a search engine crawler, but it does not name a public search product or describe what appears in that product.

The recorded user agent is Mozilla/5.0 (compatible; MagiBot/1.0.0; Matarael; +https://magi.com/bots). Requests carrying that string can fetch web pages for Peak Labs' retrieval work. No crawl schedule, preferred content type, or update policy is documented in the available record.

Identification stops at the user-agent token. MagiBot does not publish a signature endpoint here, and the record has no verified IP range. A request matching MagiBot is therefore a useful log classification, not proof that Peak Labs sent it.

Work on natural-language extraction can involve machine learning, but that fact alone does not make MagiBot a model-training crawler. There is no supported claim that allowing it improves placement in AI answers, or that blocking it opts a site out of general-purpose AI training.

Relevant for AI search

Is MagiBot relevant for AI search?

Yes. MagiBot collects pages for an AI product, so what it can crawl influences how AI systems describe your brand.

MagiBot feeds a search index that also powers that engine's AI answer features and overviews. One crawl can serve a classic results page and a generated summary, so blocking it costs you both the rankings you would expect and a growing share of AI answers built on the same index.

How to handle MagiBot

Decide based on the traffic you actually see. If MagiBot requests a small set of public pages at an acceptable rate, allowing it exposes no more than those public responses. If the crawl has no clear value to you, exclude it and measure whether the requests stop.

Use this robots rule for the recorded token:

User-agent: MagiBot
Disallow: /

Robots.txt compliance is not confirmed for MagiBot. Treat the rule as a crawl preference, then check subsequent requests. Rate limits or server-side access rules are needed when you require enforcement.

Examples

  • A research publisher sees the `MagiBot/1.0.0` signature fetching abstracts and checks whether the rate is acceptable before choosing a policy.
  • A documentation site blocks MagiBot from licensed manuals while leaving its public release notes available.
  • An analyst reports MagiBot crawl volume without converting those visits into an unsupported AI citation metric.

Frequently asked questions about MagiBot

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

The bot record identifies Peak Labs as its operator.

Related bots

AdIdxBot

An AdIdxBot visit usually starts with a Bing Ads workflow rather than an organic search crawl.

Search Engine Crawler

Adsense

The AdSense crawler visits participating sites in order to provide them with relevant ads.

Search Engine Crawler

adsnaver

adsnaver is tied to paid search advertising, not Naver's general web index.

Search Engine Crawler

Algolia

The Algolia Crawler extracts content from your site and makes it searchable.

Search Engine Crawler

Amazon AdBot

Amazon AdBot scans pages that request ads from Amazon's advertising systems.

Search Engine Crawler

Be the brand AI recommends

Monitor your brand's visibility across ChatGPT, Claude, Perplexity, and Gemini. Get actionable insights and create content that gets cited by AI search engines.

Promptwatch Dashboard