llms.txt Explained: Does Your Website Need One in 2026?

llms.txt is a plain text file, written in Markdown, that sits at the root of your website (yoursite.com/llms.txt) and gives large language models a short, curated map of your most important pages. It was proposed in September 2024 by Jeremy Howard of Answer.AI as a way to help AI tools find the useful parts of a site without wading through navigation, scripts and cookie banners.

Publishing one costs almost nothing, but on its own it will not get you cited by ChatGPT, Gemini or Google's AI Overviews. We know this partly from what the search engines have said and partly from our own server logs, which we walk through below.

What an llms.txt file contains

The proposal, published at llmstxt.org, defines a simple structure:

1. An H1 with the name of the site or company.
2. A one or two sentence summary in a blockquote.
3. H2 sections that group links to the main pages, each with a short description.

Here is a trimmed version of the one we run on ruler.agency:

# Ruler Digital Agency
> Boutique, senior-led digital agency in Athens, Greece. We build brand,
> marketing and custom digital products for premium brands.
## What we do
- Web & Digital Products (https://www.ruler.agency/what-we-do/web): Custom websites, apps, SaaS and AI tools.
- Marketing (https://www.ruler.agency/what-we-do/marketing): Social, content, performance, SEO and GEO.
## Contact
Ruler Digital Agency, Kallithea, Athens, Greece. This email address is being protected from spambots. You need JavaScript enabled to view it.

The proposal also suggests two optional extras. One is an llms-full.txt file that puts the full text of your main pages in a single document. The other is a clean Markdown copy of each page at the same URL with .md added on the end. Both make a lot of sense for software documentation. For a typical business website they are a maintenance burden with little to show for it.

Is llms.txt an official standard?

No. It is a community proposal and has none of the standing of robots.txt or sitemaps. Nobody is obliged to read it, and the major AI companies have not committed to using it in their search or answer products.

Google has been the most direct about this. Gary Illyes said at a Search Central Live event in 2025 that Google does not support llms.txt and has no plans to, and that ordinary SEO is what gets content into AI Overviews. John Mueller compared it to the old meta keywords tag. Google's own guidance on AI features now says the file is not needed to appear in Search.

OpenAI, Anthropic and Perplexity have not published firm commitments for their consumer search products either. Some documentation platforms and coding tools do read llms.txt, which is where the format has found real use.

What our own server logs show

We published llms.txt on ruler.agency in July 2026, partly as a test. Here is what one week of access logs (18 to 25 October 2026) showed:

• AI crawlers visited the site roughly 860 times. ClaudeBot accounted for 466 requests, OpenAI's OAI-SearchBot for 290, ChatGPT-User (fetches triggered by a person asking ChatGPT something) for 56, and GPTBot and Claude-User for 24 each.
• Several of those bots fetched robots.txt regularly. OAI-SearchBot alone asked for it 263 times.
• None of them requested llms.txt.
• The file was fetched about 40 times that week, mostly by SEO audit tools and site scanners checking whether it exists, plus a handful of ordinary browsers.

One week on one site is a small sample, so we would not stretch it into a rule. It does fit what larger studies have reported. Tens of thousands of sites have added the file, and no independent study has linked it to higher citation rates. In our logs, the AI crawlers spent their requests on content pages and on robots.txt.

Why we still keep an llms.txt file

It is cheap and does no harm. Ours took about twenty minutes to write. If an AI tool or agent does start reading it, the file is there and it is accurate. Writing it was also a useful exercise. Summing up the agency in two sentences and five links made us notice which service pages were vague, and we tightened them.

We do not sell llms.txt to clients as a visibility tactic. If an agency puts it at the top of a GEO proposal, ask what else is in there.

What gets you cited by AI search

When we work on generative engine optimization (GEO) for a client, the effort goes into the things AI systems demonstrably use. In rough order of impact:

1. Let the right crawlers in

Check robots.txt and any firewall or CDN bot rules. We regularly find sites that block OAI-SearchBot, PerplexityBot or ClaudeBot by accident, usually through a security plugin or a blanket "block AI" setting that someone ticked without thinking about search. Blocking training bots like GPTBot is a business decision. Blocking the search and user-triggered bots means ChatGPT and Claude can't fetch your pages when someone asks about you.

2. Write pages that answer questions directly

AI answers usually quote a passage or two from a page. A service page that opens with "We are passionate about excellence" gives a model nothing to quote. One that opens with "We build custom WordPress websites for B2B companies in Greece, typically in three to four months" does. Put a clear definition or answer in the first sentence or two of every section.

3. Add structured data that matches the page

Organization, LocalBusiness, Service, Product, FAQ and Article schema help search engines confirm who you are and what a page is about. Google's AI Overviews draw on its normal index, so the same structured data that supports regular rankings supports AI visibility.

4. Get mentioned in places AI tools trust

Models lean heavily on third-party sources: review platforms, industry directories, press coverage, Reddit and Quora threads, comparison articles. A brand nobody else talks about is hard for an AI to recommend, however good its own site is.

5. Keep the technical basics solid

That means fast pages, clean HTML, server-rendered content, a working sitemap, and no important text hidden inside images or scripts. Many AI fetchers do not run JavaScript the way a browser does, so content that only appears after scripts load can be invisible to them.

How to add llms.txt if you want one anyway

If you decide to publish one, keep it short and keep it true.

1. Write the file in Markdown: H1 with your company name, a one or two sentence summary, then a handful of H2 sections with your most important pages and a one-line description each.
2. Link to pages with full URLs, and leave out PDFs and login areas.
3. Upload it to the root of your domain so it loads at yoursite.com/llms.txt.
4. On WordPress, you can drop the file into the site root over SFTP, or use one of the SEO plugins that now generate it. Check the generated version, because automatic ones tend to list every page and post, which defeats the purpose of a curated map.
5. Add a reminder to update it whenever you change your main services or URLs. An outdated llms.txt pointing to dead pages looks worse than not having one.

Skip llms-full.txt and the .md page copies unless you run documentation, an API or a developer product.

FAQ

Does llms.txt improve my Google rankings?

No. Google has said it does not use llms.txt for Search, including AI Overviews and AI Mode. Having the file neither helps nor hurts your rankings.

Does ChatGPT read llms.txt?

OpenAI has not said that ChatGPT search uses it, and in our own logs neither OAI-SearchBot nor ChatGPT-User requested the file over a full week, while both fetched other pages and robots.txt. ChatGPT finds your content through its search crawler and the pages themselves.

Is llms.txt the same as robots.txt?

No. robots.txt is an established standard that tells crawlers what they may and may not access, and the major AI bots respect it. llms.txt is an optional summary of your content with no access rules in it. If you want to control AI crawlers, robots.txt is the file to edit.

Should a small business bother with llms.txt?

Only if it takes you less than half an hour and you will keep it updated. The same time spent rewriting your main service page so it answers customer questions clearly will do far more for your AI search visibility.

Where to put your effort instead

Publish llms.txt if you like, but don't expect it to move anything. The sites we see cited in ChatGPT, Perplexity and AI Overviews have specific pages, accurate structured data, crawlers allowed in, and people talking about them on other domains.

That is the work our SEO and GEO team does for clients, alongside the AI integration work described on our AI services page. If you want the wider picture first, read our guides on GEO vs SEO and how to rank in ChatGPT. And if you would like us to check what AI crawlers are doing on your site, get in touch.