Does llms.txt Actually Work? What the Evidence Shows
Short answer: no - not for AI search visibility, not yet. The file is real, the proposal is thoughtful, and adding one is nearly free. But the best measurement we have shows AI systems almost never read it, and the one company everyone hopes reads it calls it “purely speculative.”
Here is the evidence, dated and sourced, and what to spend the effort on instead.
What llms.txt is (and what it was actually for)
llms.txt was proposed on September 3, 2024 by Jeremy Howard of Answer.AI: a Markdown file at your site root that gives language models “information to help LLMs use a website at inference time” - a curated map of your site, built for small context windows, with links to clean Markdown versions of key pages.
Worth noticing before the hype: the proposal’s home turf is developer documentation. Its flagship adopters are FastHTML, nbdev, and Answer.AI’s own projects, and its tooling is docs plugins and CLI utilities. The idea that llms.txt would function as “SEO for ChatGPT” - publish the file, get cited - was the marketing industry’s extrapolation, not the proposal’s claim.
The adoption data: published often, read almost never
The strongest evidence to date is Ahrefs’ study (published June 15, 2026) of 137,210 domains with server-side analytics, measuring May 2026 traffic:
| Finding | Number |
|---|---|
| Domains publishing a valid llms.txt | 28% |
| Published files receiving zero requests in the month | 97% |
| Of requests that did arrive, share from bots | 96% |
| Share of bot requests actually from AI tools | 19.5% |
| AI bots probing for a file that was not published | 0 |
Read that middle row again: near-total silence. And the tail is stranger than the headline - of the little traffic llms.txt files do get, a meaningful slice is GEO/AEO audit tools checking whether you have one (12% of requests), which means part of the llms.txt economy is tools measuring compliance with a standard nobody consumes. Ahrefs’ summary is blunt: “AI retrieval bots barely fetch these files, and no AI system goes looking for one you haven’t published.” Their one genuine bright spot: coding agents - Claude Code fetched llms.txt more than any AI retrieval bot in the dataset, which fits the proposal’s docs-first origin story.
What Google says
Google’s John Mueller, June 2, 2026 (via Search Engine Journal): “I don’t think anyone knows - it’s purely speculative for now (the file has existed for years, yet none of the AI systems use it - what does it mean?).”
His practical test is the most useful sentence in this debate: “When an AI platform that brings you clients complains that it needs the file for your site, then I’d recommend taking the time to create one.” In other words: demand-driven, not faith-driven. No major AI provider - OpenAI, Anthropic, Google, or Perplexity - has publicly committed to using llms.txt for search or answers as of this writing. Independent reviews reach the same place: Wix’s AI Search Lab reviewed over 1,400 llms.txt files in late 2025 with a six-month follow-up, framing most claimed benefits as myths.
Why doesn’t anyone use it? The systems’ incentives explain it. AI search engines already crawl and index full pages the way search engines always have - and a self-authored file describing how great your site is carries the same trust problem as the old keywords meta tag: it is testimony, not evidence. Engines rank what they can verify on real pages, yours and others’.
The honest position: cheap hedge, not a strategy
Putting the evidence together:
- What the evidence supports: adding llms.txt costs minutes, carries no known penalty, and may help coding agents and future tools read your docs. If you sell to developers, that alone can justify it.
- What it does not support: any expectation of AI citations, rankings, or traffic from publishing one. There is no measured correlation, no confirmed consumer, and near-zero fetch traffic.
- The tell to watch: the day a major assistant announces it reads llms.txt, that changes - Mueller’s demand-driven test handles this automatically. Until then, effort spent polishing the file is effort taken from things that measurably move citations.
What actually moves AI visibility
The unglamorous fundamentals, which we cover in depth elsewhere:
- Be crawlable by AI bots. Most basic agent optimization, per Mueller, is simply not blocking the crawlers - check your robots.txt and CDN rules before any exotic tactic.
- Publish clear answers to real buyer questions. AI engines cite pages that answer a question cleanly, with structure and specifics. That is the core of getting your SaaS recommended by ChatGPT.
- Earn third-party mentions. Assistants lean heavily on comparison pages, reviews, and communities - the pages about you matter as much as your own (the full AEO playbook).
- Measure what AI actually sends you. AI referral traffic is real and growing even as AI Overviews squeeze classic organic clicks - track it in GA4 so decisions come from your data, not vendor decks.
And when a buyer does arrive from an AI answer, they arrive mid-conversation - they asked a question somewhere else and clicked through for the specifics. A page that answers questions in the moment finishes what the AI answer started; a static page with a form does not.
Frequently asked questions
Does llms.txt work?
Not for AI search visibility, per current evidence: 97% of published files got zero requests in Ahrefs’ 137,210-domain study, and no major AI system has confirmed using the file.
Does ChatGPT or Google use it?
No provider has confirmed it. Mueller (June 2026): the file has existed for years, yet none of the AI systems use it. AI bots in Ahrefs’ data barely fetch it and never probe for missing ones.
Should I add one anyway?
Fine as a minutes-long hedge - no known penalty, plausible value for coding agents and docs. Mueller’s test: create one when an AI platform that brings you customers asks for it. Expect no citations from it.
What improves AI visibility instead?
Crawlability, pages that answer buyer questions cleanly, third-party mentions AI systems cite, and measuring your AI referral traffic. Fundamentals, not files.
The bottom line
llms.txt is a reasonable developer-docs proposal that got drafted into a job it never applied for. The measured reality in 2026: a quarter of the web publishes it, almost nobody reads it, and no AI system has promised to. Add one if you like - it is the cheapest hedge in SEO - but the citations you want come from crawlable pages that answer real questions and the third-party consensus around them. Optimize the evidence, not the testimony.
See the after-the-click half live: when AI answers send buyers your way, Marqeable’s AI website chat answers their next question in the moment, captures the lead, follows up through automations, and shows which engines send buyers that become revenue with attribution.
Marqeable runs your campaigns, answers every visitor, text, and email in seconds, and turns them into booked jobs and meetings - even at 9pm on a Saturday. We’re in private beta with a small early cohort. Get early access
