On 10 August 2026, Jeremy Howard published the first major revision to the llms.txt proposal, nearly two years after the original. The spec lives at llmstxt.org.
What llms.txt is
A markdown file that gives AI agents a curated map of your site: an H1 with your project name, a blockquote summary, and H2 sections listing links with short descriptions. Alongside it, you serve clean markdown versions of your pages. The file stays small enough to fit in context; the detail lives behind the links and gets fetched only when needed.
The file format is unchanged in v2. What changed is how agents find it, where it can live, and what they do with it.
llms.txt v1 vs v2: the differences
| v1 (Sept 2024) | v2 (Aug 2026) | |
|---|---|---|
| Discovery | None — agents had to guess URLs | rel="alternate" type="text/markdown" for the markdown page, rel="describedby" for the covering llms.txt, via <link> tags or HTTP Link: header |
| Markdown URLs | page.html.md only | page.html.md or page.md |
| Subpaths | Allowed, undefined | A file covers everything under its path; most specific file wins |
| Consumption | Unspecified; shipped llms_txt2ctx to expand files into context | Agents read or search the file, then follow links |
| File structure | H1, blockquote, prose, H2 link lists | Unchanged |
The four changes
1. Agents can finally discover your files
This was the most-requested change and the most consequential one.
Under v1, an agent that landed on one of your pages had no reliable way to work out whether a clean markdown version existed, or which llms.txt described that page. It had to guess at URL patterns and hope. In practice, that meant the file only got used by agents that already knew to look for it at the site root.
v2 fixes this with two standard link relations already in the HTML spec:
rel="alternate" type="text/markdown"— points to the markdown version of the current pagerel="describedby"— points to thellms.txtfile that covers it
You can ship these as <link> elements in your document head, or as an HTTP Link: response header:
Link: </docs/page.html.md>; rel="alternate"; type="text/markdown", </docs/llms.txt>; rel="describedby"
The header form is the sleeper feature. It works for resources that aren't HTML at all including the markdown files themselves and it can be configured once at the web server or CDN layer without editing a single page template. For a large site, that's the difference between a migration project and an afternoon of config.
2. Both markdown URL conventions are now valid
v1 specified exactly one form: append .md to the full page URL, producing page.html.md. The logic was that it never collides with an existing path.
But a lot of publishing tools ignored that and replaced the extension instead, giving page.md. Rather than declare most of the ecosystem non-compliant, v2 accepts both. For URLs without a filename, you append index.html.md or index.md.
If your docs platform was already doing it the "wrong" way, you have nothing to fix.
3. Subpath files have defined semantics
v1 permitted llms.txt files in subdirectories but never said what one meant. v2 spells it out: a file covers every page beneath its path, and where more than one file could apply, the most specific one wins. So /docs/llms.txt governs everything under /docs/, and a root-level file handles the rest.
Two things follow. First, you can scope your files sensibly — a documentation set, a product area, and a company overview can each have their own entry point instead of being crammed into one root file that serves none of them well. Second, anyone who controls only a path can participate: a GitHub Pages project site, or a team that owns one section of a shared corporate domain, no root access required.
That second point is also the proposal's answer to the obvious objection, why not use /.well-known/, the RFC 8615 location reserved for exactly this kind of metadata? Because well-known URIs exist only at the origin root, so they can't express "this describes the pages under here," and they're unavailable to anyone who doesn't control the whole host.
4. The consumption model is stated, and the tooling is gone
v1 said very little about how these files should actually be read, and shipped a utility called llms_txt2ctx that expanded a file and everything it linked to into one large LLM context.
v2 drops the tooling and states the expectation directly: an agent views or searches your llms.txt, decides what's relevant, and follows those links on demand. Nothing gets bulk-expanded.
The practical consequence is that your links have to point at LLM-friendly content. A tidy llms.txt whose links all lead to JavaScript-heavy HTML pages defeats the purpose, the agent still ends up scraping chrome out of markup.
One casualty: the Optional section no longer carries mechanical meaning. It existed to tell expansion tools what they could safely skip. It survives as a convention for secondary links an agent can ignore when context is tight, but nothing enforces it now.
What to do
- Add the
rel="alternate"andrel="describedby"links, ideally via CDN headers - Check your
llms.txtlinks point to markdown versions, not HTML - Consider scoped files (
/docs/llms.txt) over one root file covering everything - Test it: hand an agent nothing but your
llms.txtand ask it questions about your product
Adoption is real, but publishing a file isn't the same as being read, ship your update, then measure whether your pages actually start appearing as citations in AI answers.
