What is Jina Reader?
Jina Reader is the service behind r.jina.ai. It takes a single URL, loads the page, extracts the main content and returns it as markdown that a language model can read. The JinaReader bot page says it fetches a page only when a person or an application asks for that specific URL. It does not discover URLs, follow links or crawl sites.
Depending on the page and the request, Reader loads the URL in a headless browser or with a plain HTTP request. The requester decides what happens to the result, which in practice usually means feeding it to an LLM app, an agent or a RAG pipeline. Reader is open source, and the code lives at github.com/jina-ai/reader.
Identification works through Web Bot Auth rather than a fixed user-agent string. Signed requests carry Signature-Agent: agent="https://r.jina.ai" with Signature-Input and Signature headers, and the keys are published at https://r.jina.ai/.well-known/http-message-signatures-directory. Jina says requests without these headers do not claim to be JinaReader.
robots.txt handling depends on that signature. For signed requests, Reader fetches /robots.txt first and applies the JinaReader group, falling back to *. An authenticated account can switch the check off; anonymous callers cannot. Unsigned requests are only checked against robots.txt if the caller asks for it, which is why compliance is listed as partial here.
