My llms.txt Found No Audience

I implemented llms.txt on my website and monitored the server logs for several weeks. Not a single AI system accessed the file.

llms.txt is a format proposed by Jeremy Howard in September 2024.¹ The file resides in a website’s root directory and summarizes its content for LLMs. When ChatGPT or Perplexity crawl a website, they could load this file first to quickly understand what the site offers. Documentation frameworks like Mintlify generate it automatically.² Anthropic, Stripe, and many other documentation sites now have one.

GPTBot, ClaudeBot, and PerplexityBot still crawled my website the traditional way, ignoring llms.txt. Even when I explicitly asked ChatGPT to summarize content, it didn’t access the file.

Not an Isolated Case

SEO tool provider Semrush ran the same test on their own website. From mid-August to late October 2025, they recorded zero AI crawler accesses.³

Google sees no need for a separate standard. John Mueller wrote in June 2025 that no AI system uses llms.txt.⁴ Gary Illyes confirmed in July that normal SEO works for AI Overviews too, and Google won’t support llms.txt.⁵

The major labs ignoring llms.txt isn’t due to the specification itself. It deliberately leaves open how AI systems should retrieve the file and relies on voluntary adoption.⁶ That adoption never came.

In practice, llms.txt remains limited to manual use. To give Cursor a library’s documentation, you paste the URL yourself. ChatGPT Search and Perplexity don’t fetch these files on their own.

And Then Google

On December 3, 2025, SEO expert Lidia Infante discovered an llms.txt on Google’s developer documentation.⁷ The same company that had publicly rejected the standard. The file has since disappeared.