For almost a year, LLMs.txt was sold to the SEO community as a simple, low-effort win for AI visibility — the AI-era equivalent of robots.txt. New data published this month suggests that story was mostly wishful thinking. And the most useful thing any SEO professional can do right now is look honestly at what changed.
I wrote a complete implementation guide for LLMs.txt covering exactly what it is, how it works, and how to deploy it — because at the time, the available evidence and Google's own framing suggested it was a meaningful piece of AEO and GEO strategy. New data from Ahrefs, combined with direct statements from Google's John Mueller, tells a more complicated story. This isn't a case of the tactic being a scam. It's a case of the actual adoption and impact being far smaller than the hype suggested — and that distinction matters enormously for how you should be spending your time.
This guide walks through exactly what the new data shows, what Google has said on the record, where LLMs.txt still has genuine value, and — most importantly — what you should actually be prioritising instead if AI search visibility is your goal.
The Headline Number
Ahrefs analysed server logs and live traffic across 137,000 domains in its Web Analytics dataset. They checked each domain for a valid LLMs.txt file returning a clean HTTP 200 response, then examined every request made to that file path across the entire population — classified by user agent, channel, and bot type. To rule out false positives, they confirmed each file was genuine Markdown content rather than a soft 404 page.
Who Is Actually Requesting These Files? The Real Breakdown
This is the part of the data that surprised me most. Of the small sliver of LLMs.txt files that did receive requests, the traffic composition wasn't dominated by AI retrieval bots at all. Here's the actual breakdown of who is requesting these files:
The single most telling finding in the entire study: Ahrefs analysed every request to LLMs.txt paths that returned a 404 — meaning the file didn't exist — and found that zero of those probing requests came from AI bots. The people checking for missing LLMs.txt files were humans, typing the URL directly into a browser, almost certainly SEOs auditing competitor sites. This kills the core assumption that drove most LLMs.txt adoption: that AI systems are actively hunting for this file, and a site without one is missing a knock at the door. They simply aren't checking.
What Google Has Actually Said — In Its Own Words
The data alone would be significant. What makes this a genuine reversal is that Google's own statements over the past month line up with it almost exactly. In late May 2026, Google published a new guide on optimising for generative AI features, with a section literally titled "mythbusting" that states plainly: machine-readable files like LLMs.txt aren't needed to appear in generative AI search.
Then, when SEO journalist Lily Ray pressed Google's John Mueller directly on the apparent contradiction with Chrome's Lighthouse tool (which does check for LLMs.txt), Mueller gave the clearest statement yet on the record:
Mueller had made a similar point earlier this year in a Reddit post, comparing LLMs.txt to the old meta keywords tag — a once-popular but ultimately ignored signal. His direct quote from that post: "AFAIK none of the AI services have said they're using LLMs.TXT (and you can tell when you look at your server logs that they don't even check for it)." The new Ahrefs data is essentially the large-scale empirical confirmation of exactly what Mueller had already been saying anecdotally.
The Confusing Part — Google Sent Mixed Signals Within Days
Here's where it gets genuinely strange, and worth understanding if you've felt confused by conflicting advice. In the same short window, Google's Chrome team shipped an LLMs.txt check inside Lighthouse's experimental "Agentic Browsing" audits — with documentation explaining that without the file, AI agents may spend more time crawling a site to understand its structure.
Once you understand Mueller's distinction, the contradiction dissolves. LLMs.txt isn't being framed as an SEO discovery tool — Google never claimed it determines whether your site gets recommended. Its legitimate use case is narrower and more technical: helping an agent that has already landed on your site navigate it more efficiently, similar to how a sitemap helps a crawler navigate once it's already there, rather than helping it get discovered in the first place.
"I implemented LLMs.txt for several clients this year based on the genuine belief, supported by the available evidence at the time, that it was a meaningful piece of AI visibility infrastructure. The Ahrefs data and Mueller's statements don't make that decision wrong in hindsight — they make it clear the impact ceiling was always lower than the hype suggested. The lesson I'm taking from this isn't 'never trust new SEO tactics.' It's 'separate the technically interesting from the practically impactful, and keep checking your own server logs rather than relying on industry consensus alone.'"
So Is LLMs.txt Worthless? Not Quite — Here's the Honest Verdict
- ✓ Keep an existing LLMs.txt file — it causes no harm and costs nothing to maintain
- ✓ If you already have structured product or developer documentation, compiling one is trivial
- ✓ May help AI coding agents navigate developer docs more efficiently once on your site
- ✓ Website builders like Wix, Framer, and Lovable are starting to auto-generate them — frictionless if your platform supports it
- ✓ Adoption may grow as a CMS default, similar to sitemaps, even without proven ranking impact today
- ✕ Treating LLMs.txt as a primary AI visibility strategy
- ✕ Telling clients it will increase AI Overview or ChatGPT citation frequency
- ✕ Spending significant budget or time crafting an elaborate LLMs.txt over fixing actual content gaps
- ✕ Auto-generating LLMs.txt content via CMS plugins without reviewing it — see the security risk below
- ✕ Assuming a competitor's strong AI visibility is connected to their LLMs.txt file
Ahrefs flagged something genuinely important buried in the data: a crawler was found studying LLMs.txt files specifically as a potential prompt injection risk. Because AI agents are trained to trust ingested content, a compromised or maliciously edited LLMs.txt file could become an attack vector. If you publish one, treat it like code — version control it, restrict editing access, set alerts for unauthorised changes, keep content to plain factual links and descriptions (nothing instruction-shaped), and review anything a CMS plugin auto-generates on your behalf before it goes live.
What to Prioritise Instead — Where AI Visibility Actually Comes From
If LLMs.txt isn't the lever the industry hoped it would be, the obvious next question is: what is? Based on the same body of evidence — Google's own guidance, Mueller's statements, and corroborating research from Ahrefs and SE Ranking — here's where the actual leverage sits:
| Lever | Why It Actually Moves AI Visibility |
|---|---|
| Standard Search Eligibility | Google's own guidance: good AI search visibility still depends on strong fundamentals — indexable pages, useful content, and eligibility for normal Search snippets |
| Strong EEAT Signals | Verifiable authorship, real credentials, and editorial trust signals directly feed how AI systems evaluate source quality before citing |
| Schema Markup (Article, FAQPage, Person) | Confirmed structured data that helps both traditional ranking systems and AI extraction — unlike LLMs.txt, this has measurable, documented impact |
| Third-Party Authority & Citations | Being referenced and linked to by other authoritative sources strengthens the entity signals AI systems use to evaluate trustworthiness |
| Clear, Specific Content | AI systems extract and cite specific, well-structured, fact-dense content far more readily than vague or generic copy — content quality remains the dominant signal |
| Direct Bot Analytics Monitoring | Tracking which AI bots actually visit your site gives real data instead of hoping a text file makes a difference — far more actionable than LLMs.txt alone |
The Bigger Pattern Worth Recognising
This isn't the first time the SEO industry has collectively over-indexed on a new technical tactic before the evidence caught up. Ahrefs' own framing of this is worth repeating: people latch onto ideas that feel like control over AI visibility, because right now, the actual tools to measure and influence that visibility are still immature. A simple text file you can publish yourself feels actionable in a way that "produce more original, expert-authored content" doesn't — even when the latter is what the evidence actually supports.
I'd add one more pattern from my own five years in this industry: every emerging technical standard goes through the same lifecycle. Early enthusiasm outpaces adoption. Adoption outpaces measurable impact. Eventually, hard data arrives and recalibrates expectations to reality. LLMs.txt is squarely in that recalibration phase right now. That doesn't make it dead — Mueller himself left room for it to mature as agent workflows develop further. It does mean the conversation needs to shift from "must-have AI SEO tactic" to "low-cost hygiene item with unproven upside."
Your Action Plan — What to Actually Do This Week
Check Your Own Server Logs Before Making Any Decision
Don't rely on industry-wide averages alone. If you use Ahrefs, filter Bot Analytics by Page URL containing "llms.txt" to see exactly which bots, if any, are requesting your file. A 97% chance of zero readership is the base rate — your site may differ, and you should know your own number.
Keep Existing Files, Stop Treating Them as a Growth Lever
If you already have an LLMs.txt published, leave it in place — there's no evidence it causes harm. Simply stop allocating significant time or client budget to elaborately maintaining or expanding it as if it were a primary AI visibility strategy.
Redirect That Effort to Schema Markup and EEAT
The time previously spent crafting LLMs.txt content is better spent strengthening Article, Person, and FAQPage schema, building out genuine author credentials, and producing original content with specific, citable detail — all of which have a documented connection to AI citation behaviour.
Secure Any File You Do Maintain
Treat LLMs.txt as a controlled asset rather than a marketing afterthought. Version control it, limit edit access, keep content factual and link only to pages you control, and manually review anything a CMS or plugin generates automatically before publishing.
Re-Brief Any Clients You've Advised on LLMs.txt
If you've recommended LLMs.txt to clients as a meaningful AI visibility tactic, this is worth a short, honest update. Clients respect proactive correction far more than silence — and it's a natural opening to redirect their budget toward the higher-impact work outlined above.
Frequently Asked Questions
- Check your own Bot Analytics or server logs for actual LLMs.txt request data this week
- Stop presenting LLMs.txt as a primary AI visibility tactic to clients or stakeholders
- Redirect time previously spent on LLMs.txt toward schema markup and EEAT strengthening
- Apply security best practices to any LLMs.txt file you continue to maintain
- Re-brief any clients you've previously advised on LLMs.txt with this updated evidence
The Bottom Line
The Ahrefs data, combined with Google's own statements, gives the SEO industry something it rarely gets: a clear, evidence-based correction to a tactic that outran its actual impact. LLMs.txt is not a scam and not worth removing if you already have one — but it is not the AI visibility shortcut it was sold as, and 97% zero-request rate makes that conclusion hard to argue with. The real work for AI search visibility in 2026 remains exactly what it's always been: strong fundamentals, genuine expertise, structured data that's actually been proven to matter, and content built for citation rather than for checking a box. Keep your LLMs.txt if you have one. Just stop expecting it to do the heavy lifting.
.webp)
Driven by advanced SEO expertise, deep marketing analytics, high-impact content strategy
With 5+ years of hands-on experience, I specialize in holistic search strategies that don’t just rank—they drive real, measurable business growth. I’ve worked across industries including healthcare, hospitality, legal, e-commerce, and professional services, helping brands dominate their target markets. My approach bridges the gap between raw data and creative execution. Every strategy I build is rooted in rigorous market analysis, structured SEO frameworks, and tailored content ecosystems—no templates, no shortcuts. Whether you’re a single-location brand or scaling across multiple cities, I create data-driven marketing systems designed to compound results and grow with you.
Ready to build your local SEO strategy?
Copy the prompt above, fill in your five details, and paste it into your AI assistant. Share this guide with anyone running a local business who needs more leads.
