llms.txt

Websites were built for people. The top navigation, the sidebar of related articles, the cookie banner, the contact form stapled to the bottom of every help page — all of it exists so a human can find their way around a screen. A person picks the actual content out of that mess without thinking about it.

A model can't. To a model a page arrives as one run of text with menus, body copy, and promotional blocks mixed together, and it has to work out on its own where the useful part starts and stops. It also has a hard ceiling on how much text it can hold at once, so reading an entire site and then deciding is not an option. On a site with 300 help documents, a model reads a handful of whatever crawling brought back and answers from those. Crawling is the process by which a bot walks links from page to page collecting what it finds, and the order links happen to be in has nothing to do with which pages matter.

Pagedeck ran into this directly. Ravi asked a chatbot what Pagedeck's team plan costs and got a price table that had been retired two years earlier. The old announcement was still sitting on the site, and that was the page the model happened to pick.

llms.txt is a proposal to put up a signpost. It first appeared in 2024, and the idea is that the site itself says, in advance, "here is what we make, and here are the twelve documents to start from."

It solves the problem of a model not knowing where to begin, by letting the site owner choose the reading list and write it down.

The file lives at /llms.txt on the site's root domain. It is ordinary markdown, and it usually runs in this order.

1. A title line, naming the site or the product.
2. A one-sentence summary. "Pagedeck is a collaborative document tool for teams" is enough.
3. An optional short paragraph. This is where you clear up whatever models get wrong about you.
4. The document list. Each entry is a link plus one line, like - [Team plan pricing](url): per-seat prices and billing cycles. That one line is what a model uses to decide whether the document is worth opening.
5. A section for things that can wait, so anything short on room can be skipped.

The work is in what gets left out of item 4. List all 300 documents and the file becomes a second copy of the site, which puts the model back where it started. A list worth having runs somewhere between ten and twenty entries. One way to get there is to take the ten questions that come in most often and keep only the documents that answer them.

Three other approaches address the same problem, and none of them covers the same ground as this file.

1. An XML sitemap — a file listing every page address on the site for search engines. It points the opposite way. A sitemap is trying not to miss anything, so 3,000 entries is fine; llms.txt is trying to narrow things down, so fifteen entries is the point. Having both is normal.
2. Structured data — markup inside a page that tells a machine which block is a question and which line is a price. It works at a different scale. llms.txt picks which documents to read across a whole site; structured data labels meaning inside a document you already opened.
3. Rewriting the documents themselves — the work of raising answer extractability and citability. You rewrite paragraphs that only make sense after the paragraph above them so each one stands alone, and you attach an as-of date to every number. This has the most reliable effect and costs the most, because the effort scales with the number of documents.

Of the three, rewriting is the real work and llms.txt is the cheapest thing on the list. If fixing the documentation is a project measured in months, publishing one file in the meantime is a reasonable call. The order still matters, though. Publish the file and leave the documents alone and the model reads exactly what you pointed it at and still finds nothing worth quoting.

"We published llms.txt, so why is it still quoting the old pricing?"

Publishing a file and having models read it are two separate things, and that hasn't landed yet.

"Should this doc go in llms.txt? It's the third most common support question."

Entries are being chosen by support volume.

"That's a robots.txt thing, not an llms.txt thing."

Correcting someone who has mixed up the file that guides with the file that blocks.

At Pagedeck this file belongs to Ravi. Once a quarter he pulls the ten questions support saw most, checks whether the documents answering them are on the list, adds the ones that are missing, and drops entries for pages that no longer exist. For documents that change often, like pricing and the security policy, he puts a marker such as "as of March 2026" in the description line next to the link.

Grace asks what the work produced, and Ravi has a hard time answering. He counts how often Pagedeck shows up in an AI overview, but he can't separate any movement in that count from the documentation rewrites happening at the same time.

The most common expectation is that publishing the file gets your site quoted. A file at the root of a domain feels like something everyone reads, the way robots.txt is, so the assumption makes sense. But this is a proposed format rather than an agreed standard, and few major search or model providers have publicly confirmed that they read it. Report it as work with a guaranteed payoff and, when the citation count hasn't moved in six months, the whole generative engine optimization effort gets written off along with it. Treating it as cheap insurance holds up better.

The other mistake is assuming more entries are better. It's a signpost, so more information sounds helpful — but the value of this file comes from how little is on it. An llms.txt with all 300 documents is a copy of the site rather than a list, and the model is back to choosing for itself. A file like that performs the same as no file at all, with one more thing to maintain every quarter.