Does llms.txt Matter for SEO? Official Pages, Studies and Our Check

For Google rankings, no: Google says its Search and AI features do not use llms.txt, and no AI company we checked documents it as a ranking or citation signal. Its use is narrower. OpenAI, Anthropic, Perplexity and Google's Gemini API publish one on their developer docs, where coding agents use such files.

Published · Updated

We searched this question on 20 November 2026. This post puts the official pages of Google, OpenAI, Anthropic and Perplexity next to a dated check of who publishes the file, and keeps the limits of each piece of evidence next to it.

What is llms.txt?

llms.txt is a proposed Markdown file that gives AI models and agents a short summary of a site and links to the pages worth reading. It is voluntary, and none of the search engine or AI company pages we read documents it as an input for rankings or citations.

The proposal was first published in September 2024 and reached version 2 in August 2026, according to its changes page. Version 2 lets the file sit at the root or under any path, where it covers the pages below it, and asks sites to offer clean Markdown copies of pages at the same URL, with .md appended or swapped in for the extension. It says the heaviest use is in software documentation, where coding agents rely on the file to reach API references and tutorials.

The file does not control crawling. SE Ranking notes that adding one “won't override restrictions in robots.txt,” and access for AI crawlers is still set there. Our glossary has the one-paragraph definition.

Does Google use llms.txt for rankings or AI Overviews?

No. Google's guide to generative AI search says Google Search does not use llms.txt and that you do not need it, or any other AI text file, to appear in Search, AI Overviews and AI Mode included.

The optimization guide (last updated 10 July 2026) lists “LLMS.txt files and other ‘special’ markup” among the things you can ignore: “You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them.” It adds that Google may crawl and index many file types, and that this “doesn't mean that the file is treated in a special way.” Its summary repeats the point: you can ignore “creating unnecessary AI text files (like llms.txt).”

Google's second page, AI features and your website (last updated 10 December 2025), does not name llms.txt. It says there are no additional requirements to appear in AI Overviews or AI Mode, that a page must be indexed and eligible to show with a snippet, and that “you don't need to create new machine readable files, AI text files, or markup to appear in these features.”

Google's John Mueller said much the same in a Reddit thread, as Search Engine Journal reported on 17 April 2025: “AFAIK none of the AI services have said they're using LLMs.TXT.” He compared the file to the keywords meta tag, a site owner's own claim about what the site is about, which a system can check against the site itself.

What do OpenAI, Anthropic and Perplexity say about llms.txt?

Their crawler pages for site owners do not present it as something sites should publish. Each one explains its bots and the robots.txt controls for them, and none asks sites to publish an llms.txt or says the file affects search results or citations.

What official pages say about llms.txt (all read 20 November 2026)
SourceWhat it says about llms.txtWhat it covers insteadDate on the page
Google: optimization guideNames it. Google Search “doesn't use” AI text files; you can ignore creating them.Crawlable, people-first content; the Generative AI performance report in Search Console.Last updated 10 July 2026
Google: AI features and your websiteDoes not name it. No new “AI text files” are needed.Indexing, snippet eligibility, robots.txt and preview controls.Last updated 10 December 2025
Chrome Lighthouse auditChecks for the file. A missing file makes the audit Not Applicable; a server error gets the page flagged.Agentic browsing checks, not Search.Last updated 5 May 2026
OpenAI crawlersDoes not mention it as something sites should publish. The page itself opens with a link to the llms.txt index of OpenAI's own docs.OAI-SearchBot for ChatGPT search, GPTBot for training, ChatGPT-User for user-triggered visits.No date shown
Anthropic crawlersDoes not mention it.ClaudeBot for training, Claude-User for user requests, Claude-SearchBot for search, Crawl-delay, and from 30 October 2026 a “no AI training” preference signal based on an IETF draft.“Updated over 2 weeks ago”
Perplexity crawlersDoes not mention it as something sites should publish. The docs page carries a pointer to Perplexity's own docs index at /llms.txt.PerplexityBot for search results, Perplexity-User for user requests, published IP ranges.No date shown

These pages are silent on llms.txt for third-party sites. That tells you only that the companies have not documented a use for it there; it does not show what their systems fetch. For that, see the log study below.

Who actually publishes an llms.txt?

OpenAI, Anthropic and Perplexity publish llms.txt on their developer documentation. Their main domains returned 404, except openai.com, which blocked our request. On 20 November 2026 at 05:03 UTC we requested /llms.txt on 34 domains: 22 returned a file, 11 returned 404 and one blocked the request.

Method: one curl -L request per domain, following redirects, from one location outside the US and the EU, recording the status code, content type and final URL. We picked the domains by hand in four groups (AI providers, search engines, well-known SaaS and developer-tool sites, SEO tools and publishers) plus llmstxt.org and our own site. It is a small sample chosen by us, not a measure of adoption across the web.

AI providers and search engines, checked 20 November 2026 (05:03 UTC run; one follow-up request at 05:11 UTC)
DomainURL checkedHTTP resultDate
openai.comopenai.com/llms.txt403, blocked our request; not measured2026-11-20
developers.openai.comdevelopers.openai.com/llms.txt200, text/plain2026-11-20
platform.openai.complatform.openai.com/llms.txt4042026-11-20
anthropic.comanthropic.com/llms.txt (redirected to www)4042026-11-20
docs.anthropic.comdocs.anthropic.com/llms.txt200, redirected to platform.claude.com/llms.txt2026-11-20
docs.claude.comdocs.claude.com/llms.txt200, same file at platform.claude.com2026-11-20
perplexity.aiperplexity.ai/llms.txt (redirected to www)4042026-11-20
docs.perplexity.aidocs.perplexity.ai/llms.txt200, text/plain2026-11-20
google.comgoogle.com/llms.txt4042026-11-20
developers.google.comdevelopers.google.com/llms.txt4042026-11-20
ai.google.devai.google.dev/llms.txt4042026-11-20
ai.google.dev (follow-up)ai.google.dev/gemini-api/docs/llms.txt200, redirected to aistudio.google.com/llms.txt2026-11-20
bing.combing.com/llms.txt (redirected to www)4042026-11-20
learn.microsoft.comlearn.microsoft.com/llms.txt (redirected to /en-us/)4042026-11-20

The follow-up row is not in the 34-domain count. Google's root domains return 404, but the Gemini API documentation has a file under a deeper path, which llmstxt.org links to. A check that only asks for the root file misses files placed under a path, which the proposal allows.

The other 21 domains in the same run (20 November 2026, 05:03 UTC)
GroupReturned a file (200)Returned 404
SaaS and developer toolsstripe.com, docs.stripe.com, vercel.com, supabase.com, cloudflare.com, developers.cloudflare.com, github.com, docs.github.com, hubspot.com, notion.com, linear.app, zapier.com, mintlify.com (13)shopify.dev, which redirected to /llms.md and returned 404 (1)
SEO tools and publisherssemrush.com, searchengineland.com, searchenginejournal.com (3)ahrefs.com, moz.com (2)
Otherllmstxt.org, agentagency.si (our own site) (2)None

A file that exists is not a file anyone reads. We did not measure who requests these files, and we have not checked our own server logs for AI crawler requests to /llms.txt, so this check says nothing about use.

Is there evidence llms.txt improves AI citations?

We found no published study that shows it does. The two large studies we could open point the other way: one found no link between having the file and a domain's citation frequency in LLM answers, the other found that almost nothing requests the file.

  • SE Ranking, about 300,000 domains. Only 10.13% had an llms.txt. Statistical analysis, including Spearman correlation, found no link with how often a domain was cited in LLM answers, and an XGBoost model predicted citations better with the llms.txt factor removed (SE Ranking; reported by Search Engine Journal on 20 November 2025, while SE Ranking's post now carries the date 15 October 2026). Limits: it is a correlation across domains, the post does not name the LLMs or the period it sampled, and SE Ranking says its results are context-dependent.
  • Ahrefs, 137,210 domains. The study covered sites in Ahrefs Web Analytics with traffic in May 2026 and was published on 15 June 2026 by Louise Linehan and Xibeijia Guan. 28% had a valid file, which Ahrefs treats as an upper bound because its customers skew technical. 97% of those files got no requests at all in May 2026. Of the requests that did arrive, 96% came from bots, 19.5% from named AI bots, and 1.1% from AI retrieval bots such as OAI-SearchBot and PerplexityBot. Ahrefs' summary names GPTBot as the AI bot that fetched the files most, with Anthropic's coding agent Claude-Code second, and no AI bot asked for a file that did not exist (Ahrefs). Limits: one month, one vendor's customers, and, as Ahrefs writes, “fetched” does not mean “read.”

A third study that ranks for this query, Search Engine Land's tracking of 10 sites, refused our requests on 20 November 2026 (HTTP 403), so we leave its numbers out.

When is an llms.txt worth adding?

Add one when developers point AI coding tools at your documentation, or when a tool you care about reads it. Do not add one expecting better rankings or more citations.

  • Developer documentation and APIs. This is where the proposal says use is heaviest, where OpenAI, Anthropic, Perplexity and the Gemini API publish their own, and where Ahrefs saw a coding agent among the top fetchers.
  • Sites whose text needs JavaScript. The proposal's case is that HTML wraps information in navigation, ads and JavaScript that are hard to turn back into clean text. A plain summary with links to your key pages gives an agent a shorter path.
  • Point to the file. Ahrefs found that AI tools fetch the file only after something points them to it, such as a link, an index or a user's instruction. Version 2 of the proposal suggests a rel="describedby" link from each page to the file that covers it.
  • Keep it the same as your pages. Mueller's keywords-tag comparison rests on trust: a file that claims something the pages do not show gives a system no reason to believe it. Ahrefs also logged a research crawler named prompt-injection-survey fetching llms.txt files, and advises keeping the file to plain links and descriptions, under version control.

The cost is one small text file you have to keep current. Ahrefs warns that “a stale or compromised file misleads every agent that reads it.”

What should you do instead for SEO and AI answers?

Spend the time on what the engines document: crawler access, indexing, and pages that answer real questions in plain text. These steps help in Google and in the AI assistants at the same time.

  1. Let the search crawlers in. Allow Googlebot, OAI-SearchBot, Claude-SearchBot and PerplexityBot. OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, and Anthropic says disabling Claude-SearchBot may reduce your visibility in user search results. Our guide to AI crawlers and robots.txt has an example file that keeps training crawlers separate.
  2. Get indexed. For Google's AI features a page must be indexed and eligible to show with a snippet. Check the pages that matter in Search Console.
  3. Put the facts in text on the page. Google's AI features page lists “making sure that important content is available in textual form” among its SEO basics. Price, who the product is for and what it does should be readable without a script.
  4. Earn real mentions. Google's guide says that seeking inauthentic mentions “isn't as helpful as it might seem.” Our guide to getting your brand mentioned in AI answers covers the honest version.
  5. Measure. Use the Generative AI performance report in Search Console for Google, and a fixed prompt log for ChatGPT and Perplexity, as in how to track brand mentions in ChatGPT and Perplexity. For where llms.txt sits among other GEO tactics, see GEO vs SEO for startups.

Agent Agency, which publishes this blog, makes AI community agents for branding, SEO and GEO, and its product reader is one of the tools that reads llms.txt. When you paste a link, the reader starts with your llms.txt, then the home page, then up to four same-site links from the file, to draft the Product DNA you correct and approve. The details are on how it works.

Questions

Does llms.txt help SEO?

Not for Google. Google's guide to generative AI search says Google Search does not use llms.txt or other AI text files, and you do not need them to appear in Search, AI Overviews or AI Mode. None of the OpenAI, Anthropic or Perplexity crawler pages we read on 20 November 2026 documents it as a ranking signal.

Does ChatGPT read llms.txt?

OpenAI's crawler page does not say so. In Ahrefs' May 2026 log data, GPTBot, OpenAI's training crawler, fetched llms.txt files more than any other AI bot, while AI retrieval bots such as OAI-SearchBot made 1.1% of requests. A fetch does not show the content was used.

Does llms.txt replace robots.txt?

No. llms.txt describes content and blocks nothing. robots.txt is still where crawl rules live, and only bots that honor it follow them. SE Ranking notes that adding an llms.txt will not override restrictions there.

Can an llms.txt file hurt my rankings?

Google says Search does not use the file, and that crawling or indexing a file does not mean it gets special treatment. The risks are elsewhere: a file that says something your pages do not, and, as Ahrefs reports, prompt injection if the file is edited by someone you do not control.

How do I check whether anything reads my llms.txt?

Look at your server or CDN logs for requests to /llms.txt and group them by user agent. Compare the names with the bots OpenAI, Anthropic and Perplexity publish, such as GPTBot, OAI-SearchBot, ClaudeBot, Claude-SearchBot and PerplexityBot. Ahrefs found that 97% of the files it studied got no requests in May 2026.

Join the waitlist

Sources