Build a compliant llms.txt file to help LLMs understand your website's structure and content. Define sections, add links, and export the markdown.
An llms.txt file tells AI models and LLM crawlers how they can interact with your website content, similar to robots.txt for search engines.
Yes. robots.txt controls search engine crawlers, while llms.txt specifically targets AI and LLM crawlers. They serve different purposes.
The llms.txt standard is emerging, with growing adoption by major AI companies. It's a forward-looking best practice.
An llms.txt file uses a simple markdown format. It starts with an H1 heading for your project name, followed by an optional blockquote summary, then H2 sections containing links in the format [Name](URL): Description. This structure helps LLMs quickly understand your site's content and documentation.
Place your llms.txt file in the root directory of your website so it is accessible at yourdomain.com/llms.txt. This is the standard location where AI crawlers and LLMs will look for it, similar to how robots.txt works for search engine crawlers.