Does llms.txt actually do anything? An honest verdict

Mostly no, with one narrow exception. As of mid-2026, no major AI search engine reads llms.txt at retrieval time: Google, OpenAI, Anthropic, and Perplexity have not committed to it, and Ahrefs found 97% of published files receive zero requests. The one real use case is coding agents like Claude Code and Cursor pulling docs. So for ranking and AI citations, it does close to nothing today.

That is the honest verdict, and it is going to annoy a lot of people who spent an afternoon generating one. The pitch behind llms.txt was seductive: a clean Markdown file at yoursite.com/llms.txt that hands large language models a tidy, ad-free map of your best content so they cite you instead of your competitors. Proposed by Jeremy Howard of Answer.AI in September 2024, it spread fast because it felt like the next robots.txt or sitemap. But feeling like an established standard and being adopted by the engines that matter are two very different things. Let me walk through what the file actually is, who reads it, who explicitly does not, and what you should do instead if your real goal is getting cited in AI answers.

What llms.txt actually is (and is not)

At a technical level, llms.txt is a plain Markdown file you place at your site root. It is meant to give an LLM a curated summary and a list of links to your most important pages, stripped of the HTML, JavaScript, ads, and navigation chrome that bloat a normal page and burn context window tokens. There is also a companion llms-full.txt format, co-developed by Mintlify and Anthropic, that inlines the full text of your docs into one file.

Here is the crucial distinction people skip. llms.txt is not a permissions file. It does not block or allow crawling the way robots.txt does. It is purely advisory content meant to be consumed, not obeyed. There is no enforcement, no official registry, and no W3C or IETF standardization behind it. It is a proposal that gained momentum, nothing more. If you want to control what AI crawlers can access, that still happens in robots.txt, not here.

Who actually reads llms.txt in 2026

The most important data point in this whole debate comes from Ahrefs. In their May 2026 study of 137,210 domains, 28% published an llms.txt file, and 97% of those files received zero requests. Of the tiny slice that did get fetched, the requests came overwhelmingly from SEO audit tools (21.7%), unidentified bots, general crawlers, and tech-profiling scanners, not from the AI answer engines you actually care about. Perplexity's search bot barely registered at 1.1%.

So who does fetch it? Coding agents. The study found Claude-Code outfetched most AI retrieval bots, and GPTBot dominated as a training crawler. This lines up with how Mintlify describes real usage: when a developer asks Claude Code, Cursor, or Windsurf to implement a feature using your library, the assistant may pull your llms.txt to understand the API surface. That is a genuine, measurable use case. It is just not the SEO or AI-search use case most marketers were sold.

Why Google explicitly says no

Google has been unusually blunt here. At Search Central Live APAC in July 2025, Gary Illyes stated flatly that Google does not support llms.txt and is not planning to, and that ranking in AI Overviews just needs normal SEO. John Mueller went further, comparing the file to the keywords meta tag, the tag search engines have ignored for over a decade precisely because it is controlled by the site operator and therefore trivially gamed.

The logic is sound. Why would an AI system trust a self-authored marketing file as ground truth when it can read the actual page? Any signal a publisher fully controls and can stuff with claims is, by definition, a weak signal. This is the same reason self-declared signals lose to earned ones across all of search. It is also why earned links still carry weight that on-page declarations never will.

There was a brief moment of false hope. On December 3, 2025, an llms.txt file briefly appeared in Google's own Developer Docs and was pulled the same day. Mueller clarified it showed up only because Google's internal CMS had added the capability and some teams had not removed it. It was a CMS default, not an endorsement. Search Engine Land summed up the consensus cleanly in its piece arguing llms.txt is not the new meta keywords, but the more common verdict across the industry is closer to Kai Spriestersbach's, who called it a dud.

The adoption paradox

Here is what makes this confusing. Adoption is huge. BuiltWith tracked over 844,000 websites with the file by late October 2025, and big names like Cloudflare, Stripe, Vercel, Next.js, Zapier, and Supabase all publish one. So why is something nobody reads everywhere?

Because tooling defaulted it on. Mintlify enabled automatic llms.txt generation for every documentation site it hosts in November 2024, instantly minting files for Anthropic, Cursor, Pinecone, and Windsurf. GitBook added support in January 2025. Yoast SEO shipped auto-generation in version 25.3 in June 2025. Most of those 844,000 files exist because a platform created them automatically, not because the publisher made a deliberate, evidence-based bet. High adoption here measures tooling defaults, not effectiveness. Treat the install-base number with the same skepticism you would treat any vanity metric in our link building statistics.

What to do instead if you want AI citations

If your real goal is getting surfaced and cited in AI answers, redirect that energy. The mechanics of AI citation are now reasonably well understood, and none of them involve llms.txt. Start with the fundamentals in our guide to generative engine optimization, which covers how answer engines actually select and quote sources.

1. Earn citations and brand mentions on pages the engines already trust. ChatGPT, Perplexity, and Google AI Overviews lean heavily on sources that are linked to and referenced elsewhere. See how to get cited by ChatGPT for the citation patterns that repeat.
2. Win the underlying organic rankings. AI Overviews still draw from the link-driven results, so classic ranking strength feeds AI visibility directly. There is no shortcut around it.
3. Structure content so it is extractable. Clear headings, direct answers near the top, and clean HTML do more for machine readability than a parallel Markdown file no bot fetches. Strong internal linking also helps crawlers map your topical authority.
4. Build genuine authority signals. Digital PR and editorial coverage create the off-site mentions and links that AI models weight, the exact opposite of a self-authored file.

In other words, the things that earn AI citations are the same things that earned search rankings for the last decade: relevance, authority, and links from sources the engines respect. The delivery mechanism changed; the trust model did not. If you want to benchmark what authoritative placements cost, our link pricing index is a more useful afternoon than generating an llms.txt.

Should you publish one anyway?

Yes, conditionally, and with zero expectations of SEO benefit. If you run a developer product, an llms.txt pointing to clean docs helps coding agents use your API correctly, which is a real adoption lever. If your platform (Mintlify, GitBook, Yoast) generates one automatically, leave it; it costs nothing and does no harm. But if you are a non-technical brand hoping it lifts your AI-search visibility, your hours are far better spent on links, digital PR, and extractable content.

The one thing not to do is treat it as done-and-dusted insurance. A stale llms.txt pointing at dead URLs is worse than none, because the rare agent that does fetch it gets bad data. If you publish, keep it in sync with your real content, or generate it dynamically.

The verdict, in one line

llms.txt is a well-intentioned proposal that solved a real problem (token-efficient docs for coding agents) and got marketed as a fix for a different problem (AI search visibility) that it does not touch. The engines that decide who gets cited have publicly declined to read it, and the data confirms they do not. Use it for developer tooling if that is your business. For everything else, invest in the durable signals: authoritative links acquired safely, earned coverage, and content built to be quoted. That is what AI answers actually reward in 2026.