llms.txt: does your website need one?
What the file does, who actually reads it, what Google has said, and the cheaper fixes that do more for how AI tools see your business.
If someone has told you your website needs an llms.txt file, they are not wrong that something is changing. More people now ask ChatGPT, Perplexity or Google's AI answers about a business before they visit its site. It is natural to want a file that speaks to those tools directly.
The file exists, it is easy to make, and some well-known companies publish one. The evidence that it changes anything for a typical business website is thin, though. Here is what it is, who reads it, and what we would do first.
What the file is
Jeremy Howard of Answer.AI proposed llms.txt almost exactly a year ago, in September 2024. The idea is simple. Web pages are full of menus, scripts, cookie banners and layout that a language model has to wade through to find the useful part. So you put a plain Markdown file at yourdomain.com/llms.txt that gives the short version: the name of the business or product, a one-paragraph summary, then lists of your most useful pages, each with a line saying what it covers.
The proposal also suggests offering clean Markdown copies of important pages, so a model can read the text without the clutter. Some documentation platforms go a step further and generate an llms-full.txt, which is the entire documentation in one file.
Think of it as a table of contents written for software rather than people.
Who uses it today
Developer tools adopted it first, and for good reason. Companies like Anthropic and Cloudflare publish llms.txt files for their documentation, and the docs platform Mintlify generates one automatically for every site it hosts. When a programmer asks a coding assistant how to use an API, the assistant can fetch that one file and get to the right page without crawling the whole site.
That is the use case where it clearly works: an AI tool that has been sent to your site, on behalf of a user, looking for something specific.
Who does not read it
The crawlers that build AI answers are another matter. None of the big AI companies has said that its crawler uses llms.txt. OpenAI, Anthropic and Perplexity document how their bots treat robots.txt and say nothing about llms.txt.
Google has been the most direct. In April, John Mueller of Google's search team said no AI system had announced it uses the file, that server logs showed crawlers were not even requesting it, and compared it to the old keywords meta tag, the one search engines stopped trusting years ago. In July, Gary Illyes said at a Google event that Google does not support llms.txt and has no plans to.
There are scattered reports of OpenAI's crawler fetching the file on some sites. Fetching a file is not the same as using it in an answer, and nobody has shown that it does.
If a file's main benefit is that one crawler occasionally downloads it, it is not an SEO strategy.
What matters more right now
If your goal is for AI tools to describe your business correctly, a few older, duller things carry more weight.
First, make sure your pages show their text without JavaScript. Several AI crawlers fetch the raw HTML of a page and do not run its scripts. A site that builds everything in the browser, as many single-page apps do, can look nearly empty to them. Server-rendered pages avoid that problem entirely.
Second, decide what you want in robots.txt. OpenAI, Anthropic, Perplexity and Google each have named crawlers, and you can allow or block them separately. In July, Cloudflare started blocking AI crawlers by default for new sites it serves, so it is worth checking whether yours is being blocked without anyone deciding to.
Third, put the facts in plain words. Your services, the areas you cover, your prices or price ranges, your hours, how to book. AI answers are built from text they can find and quote. A page that says "we deliver across Dubai, Sharjah and Abu Dhabi within 48 hours" is far more useful to them than a hero banner with a slogan.
Fourth, keep your expectations honest. Studies so far show AI tools send very little traffic to most websites. Ahrefs measured AI referrals at well under one percent of visits across thousands of sites earlier this year. That will grow, but for now Google search still sends most of the visitors.
So, should you add one?
If you run a software product with documentation, yes. It takes an afternoon, and it helps coding assistants and support bots find the right page.
For a typical business website, it does no harm, and if your developer can add it in half an hour, fine. Just do not pay for it as an "AI visibility" service, and do not let it jump ahead of the basics above.
