What does Nutch-1.16 in the user agent mean?

It identifies Apache Nutch as the crawling software. SMTnetPMBot is SMTnet's configured deployment of that general-purpose crawler.

Does SMTnet PM Bot respect robots.txt?

Yes, according to the bot record. Apache Nutch also documents support for robots.txt and robots meta directives.

Can I verify that SMTnet sent the request?

this bot provides no signature directory or verified IP range. The user agent identifies the claimed crawler but can be spoofed.

Is this bot collecting data for AI training?

No such use is documented. Its stated destination is SMTnet's own site-search index.

SMTnet PM Bot: AI Crawler Guide & robots.txt

What is SMTnet PM Bot?

SMTnet PM Bot visits partner company websites so their content can appear in SMTnet's on-site search. SMTnet is focused on surface-mount and PCB electronics manufacturing, so this crawl has a narrower purpose than a general web index.

The recorded user agent, SMTnetPMBot/Nutch-1.16, shows that the crawler is built with Apache Nutch. In practice, it requests public partner pages and supplies an index used inside SMTnet. The available record does not specify a crawl interval or promise that every fetched page will be listed.

Nutch supports robots.txt, and this bot marks SMTnet PM Bot as compliant. There is no published Web Bot Auth endpoint or verified IP range here, so the custom user-agent name helps classify traffic but does not authenticate its sender.

No AI answer or model-training use is documented. For a partner, blocking the bot can make company information stale or absent in SMTnet search. For everyone else, allowing it has no established effect on broader search or AI visibility.

Relevant for AI search

Is SMTnet PM Bot relevant for AI search?

Yes. SMTnet PM Bot collects pages for an AI product, so what it can crawl influences how AI systems describe your brand.

SMTnet PM Bot feeds a search index that also powers that engine's AI answer features and overviews. One crawl can serve a classic results page and a generated summary, so blocking it costs you both the rankings you would expect and a growing share of AI answers built on the same index.

How to handle SMTnet PM Bot

An SMTnet partner should let SMTnetPMBot reach the public company and product pages intended for the industry search. Exclude account routes, quote forms, or generated URL spaces that do not belong in that index. If the partnership has ended, ask the account owner whether the crawl is still expected.

To stop only SMTnet's deployment, use its specific token:

User-agent: SMTnetPMBot
Disallow: /

The bot record and the underlying Nutch software indicate robots.txt support. Avoid blocking the generic Nutch token unless you intend to cover unrelated crawlers built with the same software.

Examples

An electronics supplier lets SMTnet PM Bot refresh public equipment pages included in its SMTnet partner profile.
A manufacturer disallows a parameterized quote calculator while leaving product specification pages crawlable.
A former partner blocks `SMTnetPMBot` specifically without affecting other services that happen to use Apache Nutch.

Frequently asked questions about SMTnet PM Bot

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

Its stated purpose is to index partner company websites for search within SMTnet. Check with the team that manages your SMTnet listing or partnership.