A‑Vintage Digital
llms.txt is a proposed plain-text Markdown file, placed at the root of your website, that gives large language models a concise, curated map of your most important content. Think of it as a table of contents written for AI tools, in contrast to robots.txt (which tells crawlers what they may access) and sitemap.xml (which lists URLs for search engines).
Where llms.txt comes from
The idea was proposed in 2024 and is described at llmstxt.org. The reasoning is that a language model has a limited context window and HTML pages are noisy: navigation, scripts and layout bury the useful text. A short Markdown file that links to clean versions of your key pages makes it easier for a tool to fetch exactly what it needs at the moment a user asks a question.
How it differs from robots.txt and sitemap.xml
| File | Purpose | Audience | Status |
|---|---|---|---|
| robots.txt | Controls what crawlers may access | All crawlers | Established standard |
| sitemap.xml | Lists URLs you want indexed | Search engines | Established standard |
| llms.txt | Curated, human-readable map of key content | AI tools and agents | Community proposal |
The llms.txt format
The proposal uses a simple Markdown structure:
- An H1 with the site or project name (required).
- A blockquote with a short summary of what the site is.
- Optional paragraphs with extra context.
- H2 sections containing bullet lists of links, each with a short description, in the form
- [Page name](https://example.com/page): what it covers. - An optional "Optional" section for secondary links that tools can skip when context is limited.
A companion file, often named llms-full.txt, can contain the full text of your key pages in one document.
A real example
We publish both files for this site. Our llms.txt opens with a summary of A-Vintage Digital, then lists our services, the products we build, client work, our guides and contact details. The llms-full.txt file includes the full text of every guide. Both files are generated automatically from our content management system, so they never go out of date when we publish.
How to create an llms.txt file
- Pick your 10 to 40 most important pages. Services, products, pricing, documentation, key guides and contact information.
- Write a two-sentence summary. State plainly who you are, what you do and who you serve.
- Group the links. Use clear H2 headings such as Services, Products, Guides and About.
- Add one-line descriptions. Describe what each page contains, not marketing slogans.
- Serve it at /llms.txt as plain text with a
text/plain; charset=utf-8or Markdown content type, and keep it accessible to crawlers. - Automate it. Generate the file from your CMS or build process so it stays accurate.
What llms.txt can and cannot do
Realistic benefits
- Helps AI agents and developer tools that fetch your pages on demand find clean, relevant content quickly.
- Forces you to decide which pages are canonical and how to describe your site in plain language, which is a useful exercise on its own.
- Costs almost nothing to maintain once automated.
Things it does not do
- It does not guarantee citations in ChatGPT, Perplexity or Google AI features.
- It is not a replacement for robots.txt rules, a sitemap or good on-page content.
- It is not a way to hide or protect content. Anything you list is public.
Google's own documentation on AI features and your website says there are no special requirements beyond standard SEO fundamentals, so do not expect an llms.txt file to change what appears in AI Overviews.
What to prioritise instead
- Make sure AI search crawlers are allowed in robots.txt and not blocked by your CDN.
- Write answer-first content, as explained in our AEO guide.
- Add accurate structured data and a clear About page so your brand is an unambiguous entity.
- Earn mentions on other trusted sites.
- Keep your site fast and crawlable. See how web design affects SEO.
Frequently asked questions
What is llms.txt?
llms.txt is a proposed Markdown file at the root of a website that gives AI tools a curated, easy-to-read map of the site's most important content, with short descriptions and links.
Does llms.txt improve my rankings in ChatGPT or Google AI Overviews?
There is no public confirmation that major AI search products use llms.txt for ranking or citations, and Google has indicated it does not rely on it. Treat it as a low-cost extra, not a ranking factor.
Is llms.txt the same as robots.txt?
No. robots.txt controls which crawlers may access which paths. llms.txt is a content guide for AI tools. It does not allow or block anything.
Should I create an llms.txt file?
It is cheap and harmless, and helpful for AI agents and developer tools, so many sites publish one. Just do not let it distract from fundamentals like crawl access, quality content and strong SEO.
What should I put in llms.txt?
A short summary of your site, then grouped links to your key pages (services, products, documentation, guides, contact), each with a one-line description. Optionally add secondary links in an Optional section.
What is llms-full.txt?
It is a companion file that contains the full text of your key pages in one document, so a tool can load everything in a single request.
Need help with AI search visibility?
We audit crawl access, content structure, schema and entity signals for brands that want to be found in both Google and AI tools. See our SEO, GEO and AEO service or request a free audit.
