We added an llms.txt file to our own website this week. It took under an hour, most of which was spent deciding what to leave out. Whether you should do the same depends on what you believe it does, and this is a topic where the sales pitches have drifted a long way from the evidence.
The short version: llms.txt is a plain markdown file at your site's root that gives AI systems a curated summary of who you are and where your important pages live. It costs almost nothing to add, it can't hurt you, and some AI tools do read it. But no major AI platform has committed to using it, so treat it as cheap insurance rather than a visibility lever. Anyone charging you serious money for "llms.txt implementation" is selling an afternoon's work at consulting rates.
Here's the fuller picture.
What llms.txt actually is
The proposal came from Jeremy Howard of Answer.AI in September 2024, and the spec lives at llmstxt.org. The idea is simple. Websites are built for browsers: navigation menus, cookie banners, scripts, layout markup. A language model trying to understand your site has to wade through all of it. An llms.txt file skips the wading by offering a clean, human-written summary in markdown: a short description of the site, then organised lists of links to the pages that matter, each with a line of context.
Think of it as the difference between handing someone your brochure and making them reconstruct your business from your filing cabinet.
It sits alongside two older root files that do related but different jobs. robots.txt tells crawlers what they may access. A sitemap tells them what exists. llms.txt tells an AI what's worth reading first, and what it means. It's curation, which the other two deliberately aren't.
The honest part: adoption is one-sided
Here's what a lot of coverage skips. Adoption has two sides, and only one of them has shown up.
On the publisher side, uptake is real. Documentation platforms like Mintlify generate llms.txt files automatically, and companies including Anthropic, Perplexity, and Cloudflare publish them for their docs. Directories tracking the standard list thousands of sites.
On the consumer side, the picture is much thinner. None of the major AI platforms have publicly committed to using llms.txt when generating answers. Google's John Mueller said directly that no AI system currently uses it, comparing it to the keywords meta tag of the early web: a place for site owners to say whatever they want about themselves, which is exactly why answer engines learned not to trust such places. Server-log studies through 2025 and into this year have found AI crawlers requesting llms.txt files only rarely.
So the sceptical case is respectable. If the engines don't read the file, the file does nothing.
Why we added one anyway
Three reasons, none of which require the sceptics to be wrong.
First, the cost is genuinely trivial. Ours took under an hour, and it now serves as a canonical, always-current summary of our services that any system, AI or otherwise, can fetch in one request. When the downside is an hour and the upside is being early to a standard that might matter, the maths isn't hard.
Second, "no major platform" isn't "nobody". AI coding assistants and research tools fetch llms.txt today when a user points them at a site, and smaller answer engines experiment freely. The long tail of AI tools is where new conventions get proven before the big platforms adopt them. robots.txt itself started as an informal convention that engines chose to honour.
Third, writing one is a useful audit even if no machine ever reads it. Forcing yourself to summarise your site in twenty lines exposes exactly the pages that are hard to explain, redundant, or missing. Ours flagged that our GEO articles had no single hub page tying them together, which is now on our own to-do list. We covered the broader principle in our guide on getting cited in AI search: the work that makes you legible to machines usually makes you clearer to humans too.
What we did not do is treat it as a substitute for the things that demonstrably move AI visibility. Extractable content, schema markup, credible sourcing, and being crawlable in the first place all have evidence behind them. llms.txt doesn't yet. Our AISEO service sequences the work in that order for a reason.
How to write one
The format is ordinary markdown with a loose structure: an H1 with your site or company name, a blockquote summary, then H2 sections containing link lists.
A minimal example for a Singapore services business:
# Acme Renovations
> Acme Renovations is an HDB-licensed renovation contractor in
> Singapore specialising in kitchen and bathroom remodelling.
## Services
- [Kitchen renovation](https://example.sg/kitchen): Scope, timelines, and typical costs
- [Bathroom renovation](https://example.sg/bathroom): Waterproofing-first approach
## Company
- [About](https://example.sg/about): Licences, insurance, and track record
- [Contact](https://example.sg/contact): Quote requests, response within one working day
The judgement calls that matter more than the syntax:
- Curate hard. Ten well-chosen links with honest one-line descriptions beat a dump of your entire sitemap. If everything is important, nothing is.
- Write the descriptions for a reader who has never heard of you. The file is context for a system with no memory of your brand.
- Keep it truthful and current. A stale llms.txt that contradicts your live site is worse than none, because inconsistency is precisely what makes machines discount self-descriptions.
- Put it at the root (yoursite.com/llms.txt), serve it as plain text, and don't block it in robots.txt.
If your site runs on a framework like Next.js, the file goes in your public folder and deploys with everything else. On WordPress, several SEO plugins now generate one, though the auto-generated output usually fails the curation test above and is worth rewriting by hand.
Frequently asked questions
Does llms.txt improve my Google rankings? No. Google has said its search systems don't use it, and there's no evidence of any ranking effect. Whatever value it has lives with AI assistants and agents, not the classic results page.
Is llms.txt the same as robots.txt for AI? No, and mixing them up can hurt you. robots.txt controls access, and it's where you'd block or allow crawlers like GPTBot. llms.txt grants nothing and blocks nothing; it only describes. If you want AI platforms to cite you, your robots.txt needs to let their bots in regardless of whether an llms.txt exists.
Should I pay an agency to create an llms.txt file? Not as a standalone deliverable. It's an hour of work that mostly requires knowing your own business. Where an agency legitimately helps is the surrounding system: deciding which pages deserve to exist, structuring them so AI systems can extract answers, and measuring whether any of it changes what the platforms say about you.
What's the difference between llms.txt and llms-full.txt? The spec allows a companion file, llms-full.txt, containing the full text of your key content in one document rather than links. It's mainly useful for documentation sites where a tool might want everything in one fetch. For a typical business site, the standard file is enough.