← HomeHow to Get Cited in AI
Answer

What is llms.txt and does it actually work?

By Jason Burns, Founder of HurcuLeads · Stuff Doer at Adolicious · Updated

llms.txt is a proposed plain-text file at a site's root that lists the URLs and short descriptions a site owner wants LLMs to prioritize when summarizing that site — and as of mid-2026 no major answer engine has confirmed that it uses the file for retrieval or citation.

The proposal comes from Answer.AI and is documented at llmstxt.org. It defines a markdown-style file (/llms.txt) with a title, short description, and grouped links, plus an optional /llms-full.txt variant containing full content or excerpts. It is deliberately similar in spirit to robots.txt and sitemap.xml: a well-known path a bot might look for.

Adoption is asymmetric. Publishers ship the file readily; the answer engines — ChatGPT, Perplexity, Google AI Overviews, Gemini, Claude — have not publicly documented consuming it. That may change, and there is no downside to publishing one: it is cheap, machine-readable, and gives a clean summary if any crawler does start reading it.

Practical guidance: ship llms.txt pointing at your best pages, ship llms-full.txt with short excerpts (not full text — that hands scrapers a bulk copy kit), and do not treat either file as a substitute for the things that do demonstrably work — server-rendered HTML, QAPage markup, and IndexNow discovery.

If an engine announces that it consumes llms.txt, publishers who already ship it get a free head start. If none do, the file is harmless — it costs nothing and gives human reviewers a useful summary of what the site claims to cover.

Related terms

Sources

Related questions people ask next

  • What are markdown twins and do LLMs use them?

    Markdown twins are plain-text or markdown copies of an HTML page served at a parallel URL (typically {url}.md) so retrieval systems can ingest the content without an HTML parser — a practical convention endorsed by the llms.txt proposal but not yet formally consumed by any major answer engine.

  • How should large sites structure their sitemap?

    Large sites should structure their sitemap as a sitemap index pointing at multiple child sitemaps, each holding up to 50,000 URLs or 50MB uncompressed, and reference the index from robots.txt so every crawler — search and AI — can discover the full URL surface in one round trip.

  • How should robots.txt handle AI crawlers?

    Handle AI crawlers in robots.txt by explicitly listing each user agent — GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Perplexity-User, Google-Extended — and setting Allow or Disallow per bot to match your policy on training, search citation, and live-fetch use.

  • Does schema.org markup affect AI citations?

    Schema.org markup affects AI citations indirectly but reliably — it collapses the model's attribution problem by making the author, publisher, and answer text machine-readable, and it is the primary signal Google uses to render both classic rich results and the source strip in AI Overviews.