llms.txt: What It Is, an Example File and Whether It Helps SEO

llms.txt is one of the most talked-about files in AI search, and one of the most overhyped. I've had founders ask whether it's the secret to getting cited by ChatGPT, and I've seen vendors sell "llms.txt optimisation" as a standalone service. This guide explains what the file actually is, shows a practical llms.txt example, and gives you an honest view of what it can and can't do.

Short version: there is no proof it helps visibility, but there's no harm in adding it, so I add it by default, alongside a companion file called ai.txt. Neither is anywhere near as important as the fundamentals.

What is llms.txt?

llms.txt is a proposed standard for a plain markdown file at the root of your domain, at /llms.txt. It was proposed in September 2024 by Jeremy Howard of Answer.AI, and the specification is published at llmstxt.org.

The idea is simple. Web pages are full of navigation, scripts, cookie banners and layout that make them awkward for a language model to read within a limited context window. An llms.txt file gives a model a clean, curated summary: who you are, what the site covers, and links to the pages that matter most, each with a short description.

The proposal also suggests offering markdown versions of key pages. Most business sites only need the basic file.

What is llms-full.txt?

llms-full.txt is an optional companion file. Where llms.txt links to your key pages, llms-full.txt puts the full text of those pages into one file, so a tool can load everything in a single request. It suits documentation sites, where developers and coding assistants want a whole set of docs at once. If you publish one, keep it current: an out-of-date full-text file gives AI tools old information in a very convenient form.

The llms.txt format

The spec is deliberately light. A valid file follows this order:

1. An H1 with the name of the site or project (the only required element)
2. A blockquote with a one or two sentence summary
3. Optional paragraphs or lists with extra context
4. H2 sections, each containing a list of links in the form: link, colon, short description
5. An optional H2 called "Optional" for secondary links a model can skip if it's short on context

There's no special syntax beyond ordinary markdown. That's part of the appeal: anyone who can write a README can write an llms.txt.

An llms.txt example for a consultancy

Here is an example for a fictional independent marketing consultancy. Each row below is one line of the file, exactly as you'd type it (blank lines between sections are omitted for readability).

This site publishes its own llms.txt, built on the same structure, if you want to see a live version. To draft one from your sitemap, or validate an existing file, use my free llms.txt generator and checker.

llms.txt vs robots.txt vs sitemap.xml

The most common confusion I see is people treating llms.txt as a control file. It isn't. Here's how the three files differ.

If you want to block or allow AI crawlers such as GPTBot or Google-Extended, that's a robots.txt decision. llms.txt has no effect on access at all. My AI crawler checker shows, bot by bot, what your robots.txt allows today.

What llms.txt can and can't do

What it can't do, as of today:

• It won't improve Google rankings. Google's guidance on AI features says you don't need to create any new machine-readable files or special AI text files to appear in AI Overviews or AI Mode.
• It isn't a confirmed citation signal. No major AI provider has confirmed that ChatGPT, Gemini, Claude or Perplexity use llms.txt to decide what to cite or recommend.
• It won't fix a weak site. If your pages aren't indexed, aren't trusted or don't answer questions clearly, a summary file changes nothing.
• Fetched isn't the same as used. Your server logs show whether any bot requests /llms.txt, and site owners report anything from no requests at all to frequent ones. A request shows that a bot fetched the file, not that an AI platform uses it to choose what to cite.

What it can do:

• Give AI agents and tools that do fetch it a clean map of your site. AI agents increasingly browse and act on someone's behalf, and a short curated file is easier for them to use than a full page of navigation and scripts. Some developer tools, coding assistants and documentation platforms already support it, which is why it's most common on software documentation sites.
• Force a useful exercise. Writing a one-line description for your ten most important pages exposes vague positioning fast.
• Future-proof cheaply. If a provider does adopt it, you're already there.

Chrome Lighthouse now checks for llms.txt

Chrome's Lighthouse (version 13.3) now checks for llms.txt in an experimental agentic-browsing audit. That's about AI agents browsing on someone's behalf, not a Google Search ranking signal; Google Search says you don't need it. It is a sign that browser makers are paying attention to the file, nothing more yet.

So, does llms.txt help SEO? Not directly. Treat any "llms.txt SEO" claim with scepticism unless it comes with evidence from the provider itself. Getting cited by AI assistants depends on the things I cover in how to rank in ChatGPT: being crawlable, being clearly and consistently described across the web, and publishing content worth citing.

Why I add llms.txt and ai.txt by default

My own view has moved on this. I now add llms.txt to client sites by default, and I've seen some requests for it in server logs. I also recommend adding ai.txt.

ai.txt is a separate proposal from Spawning, a company working on consent for AI training. It's a file at your domain root that states your permissions for AI training use of your content and media, such as text and images. It is honoured through Spawning's own network and partners, so it isn't a universal block; robots.txt remains the way to allow or block specific AI crawlers.

There is no proof either file helps visibility, but there's no harm in adding both, so I add both by default. Google's position stays the same: Search doesn't need special AI text files for AI Overviews or AI Mode, as its AI features guidance explains. So both files are a small, cheap task, never a priority over content, entity consistency and third-party mentions.

How to implement llms.txt in 15 minutes

If your fundamentals are in place, here's the quick version.

1. List your 8 to 15 most important URLs. Service pages, your best guides, about and contact. Skip thin or outdated pages.
2. Write the H1 and summary. One or two sentences a stranger could repeat accurately: who you are, what you do, for whom, where.
3. Group links under H2 sections. Services, Guides, About is enough for most businesses.
4. Write one-line descriptions. Say what the page answers, in plain language.
5. Save as llms.txt and upload to your domain root. It should load at yourdomain.com/llms.txt as plain text.
6. Check it's not blocked. Make sure robots.txt, a firewall or a CDN bot rule isn't blocking the path.
7. Add a review reminder. Update it when you launch or retire a major page. A stale llms.txt is worse than none.
8. Check your logs after a month. Look for requests to /llms.txt and note which user agents made them. No requests means nothing is fetching it yet; requests mean it's being fetched, not that it's affecting citations.

You don't have to write it by hand. Yoast and Webflow can generate the file and documentation platforms such as Mintlify publish one; review what they produce, because auto-generated versions often dump every URL and lose the point of curation.

Where llms.txt sits in the priority list

I add llms.txt by default, but it never jumps the queue. Fix these first:

• Important pages that aren't indexed or have technical errors
• No clear positioning or answer-first content
• No conversion tracking, so you can't measure any AI search work
• A list of higher-impact fixes competing for limited hours

In an AI visibility audit, llms.txt and ai.txt land near the bottom of the priority list: quick tasks to tick off, not a strategy. The bigger gains come from entity consistency, third-party mentions and content that answers the questions buyers actually ask assistants. If you're working out where it fits in the wider picture, my overview of SEO vs AEO vs GEO is a useful next read, and it's the kind of prioritisation I help with as a GEO consultant.