NLWeb

NLWeb is a set of open protocols and an open-source reference implementation for giving a website a natural language interface. Instead of an AI agent crawling pages and guessing at their structure, it can send a question to the site's NLWeb endpoint and get back structured results. Microsoft announced NLWeb at its Build conference in May 2025. It was designed by R.V. Guha, who earlier created RSS and Schema.org, and the project describes its goal as a semantic layer for the agentic web.

The protocol is small. A site exposes two endpoints, /ask and /mcp, that take the same arguments. The only required one is the natural language query; optional ones include earlier queries in the conversation, a site filter, streaming (on by default) and a mode. In 'list' mode the endpoint returns the most relevant items, 'summarize' adds a summary, and 'generate' works like conventional retrieval-augmented generation. Each result carries a URL, name, score, a generated description and the underlying item as a Schema.org object. The /mcp endpoint returns the same answers in a format MCP clients understand, so every NLWeb instance also acts as a Model Context Protocol server, and the project says support for the Agent2Agent protocol is planned.

The reference implementation, now maintained at github.com/nlweb-ai/NLWeb, is written in Python. It ingests a site's existing structured data and RSS feeds into a vector store (Qdrant, Snowflake, Milvus, Azure AI Search, Elasticsearch, Postgres or Cloudflare's service) and works with models from OpenAI, Anthropic, Google, DeepSeek and others. The project points out that Schema.org and formats like RSS are already used by more than 100 million websites, which is what NLWeb builds on. Sites with thorough schema markup therefore have most of the raw material in place.

The easiest deployment path runs through Cloudflare. Its AI Search product, formerly AutoRAG, offers an 'NLWeb Website' option that crawls and indexes a site and deploys a Cloudflare Worker exposing the /ask and /mcp endpoints (Cloudflare docs). Work on the standard itself moved partly to the W3C, where an NLWeb Community Group opened in October 2025; as of October 2026 it lists 15 participants from nine organizations, including Microsoft, Cloudflare, Wix and Elastic, and has not yet chosen a chair (W3C). There is no public count of NLWeb deployments, because endpoints have no fixed URL that can be crawled and counted, and adoption appears limited outside early partners.

NLWeb fits a line of structured content formats. Schema.org tells crawlers what an entity is, llms.txt points AI systems to important pages, and NLWeb lets a site answer questions about its own content. For GEO teams the appeal is control and accuracy: when an agent queries the site directly, the site decides what is returned and can keep prices, stock and schedules current, avoiding misreadings of rendered pages. The limits are equally real. Few AI assistants are known to query NLWeb endpoints today, and answer engines like ChatGPT, Gemini and Perplexity still build answers from their own crawling and search indexes. NLWeb is worth piloting for sites with large, structured and fast-changing data, such as retail catalogs, event listings, recipes or documentation, but it adds to crawlability, clean structured data and citable pages rather than replacing them, as part of broader agent experience optimization.