How can a publisher control Instapaper's parsed article?

Instapaper's publisher guide recommends Open Graph markup for parser output. Save links can also pass a URL, title, and optional description.

Will Instapaper obey robots.txt?

Cloudflare's bot directory marks the bot as not following robots.txt. A rule for `Instapaper` expresses a preference, but it is not assured enforcement and does not match the recorded `Instaparser/1.0` string.

How can a publisher opt out of Instapaper parsing?

The legal owner or copyright agent can email support@instapaper.com with the domains and contact information that Instapaper can use for identity verification.

Can Instapaper traffic be authenticated from its user agent?

No. The record includes `Instapaper/4.0` and `Instaparser/1.0`, but no verified IP ranges or Web Bot Auth keys. Those strings are useful labels, not proof of origin.

Does Instapaper affect AI search or model training?

Neither use is documented. Its published purpose is parsing articles that people save to read later.

Instapaper: AI Crawler Guide & robots.txt

What is Instapaper?

Instapaper fetches articles for its read-later app. A reader can save a page through Instapaper or use a publisher-provided save link, after which the service parses the article for reading in the app. This is a reader-driven content fetch, not a crawl meant to discover an entire site.

The Instapaper publisher guide lets sites shape that parsed result with Open Graph metadata. Publishers can provide a title and description in a save link as well. These controls concern how an article is presented to the person who saved it.

Instapaper also offers a formal parser opt-out. The site's legal owner or copyright agent can contact [email protected] with the affected domains and contact details for identity verification. That operator-level route is more specific to text parsing than a generic bot block.

Cloudflare's bot directory records Instapaper/4.0 and Instaparser/1.0 as user agents, with patterns for Instapaper and Instapaper/4.0. The directory's stable token is Instapaper, so it can classify the main client. The separate Instaparser string is worth retaining in log searches when investigating parser traffic.

Robots behavior is not listed as confirmed in the bot's listing, while Cloudflare's bot directory marks the bot as not following robots.txt. A rule for Instapaper can publish a preference, but it may neither stop the service nor cover the Instaparser/1.0 form. Use the publisher opt-out or enforced access controls when the result must be definitive.

Despite the AI Assistant category in the bot directory, the documented workflow is saving articles for later reading. The supplied Instapaper material does not connect these requests to public AI search answers or model training. Blocking the parser affects the saved-reading experience, not a documented AI visibility or training pipeline.

Relevant for AI search

Is Instapaper relevant for AI search?

Yes. Instapaper collects pages for an AI product, so what it can crawl influences how AI systems describe your brand.

Instapaper crawls and indexes pages so the AI search or assistant behind it can retrieve them at answer time. A page it has never fetched cannot be quoted, summarized, or linked in that product's answers, so most sites keep it allowed to stay citable. Blocking it removes your pages from that AI surface and hands those citations to competitors.

How to handle Instapaper

Publishers that want readers to save articles can leave Instapaper's parser accessible and use Open Graph markup to improve the extracted title and text. A site that wants to state a crawl preference can keep this token-specific rule:

User-agent: Instapaper
Disallow: /

Do not treat the rule as complete enforcement. Cloudflare's bot directory marks Instapaper as not following robots.txt, and a recorded parser client uses Instaparser/1.0 rather than the Instapaper token.

For an Instapaper-specific opt-out, the operator tells a legal owner or copyright agent to email [email protected] with the domains and identity-verification contact details. Private or licensed material should still be protected by the site's own authentication.

No signed-agent directory or verified IP range is supplied. Use Instapaper/4.0 and Instaparser/1.0 to classify likely traffic, while remembering that either header can be copied.

Examples

A reader clicks a publisher's Save to Instapaper link, and Instapaper fetches the article so it can be read later.
An editor adjusts Open Graph metadata after checking how long-form stories appear in Instapaper's parser.
A copyright owner who does not permit text parsing uses Instapaper's publisher contact process to request a domain opt-out.

Frequently asked questions about Instapaper

Learn about AI visibility monitoring and how Promptwatch helps your brand succeed in AI search.

The documented workflow begins when a reader saves an article, including through a publisher's Instapaper link or button.

Related bots

Amazon Kendra

Amazon Kendra is a managed information retrieval and intelligent search service that uses natural language processing and advanced deep learning model.

AI Assistant