We Audited 100 Websites for AI Search in 2026 — 94% Are Invisible to ChatGPT and Google AI

Search did not disappear in 2026 — it moved into a chat box. Millions of people now ask ChatGPT, Perplexity, Gemini and Google AI Overviews instead of scrolling ten blue links. So we ran a free AI search audit on 100 real websites to answer a simple question: how many of them can AI assistants actually see? The answer was brutal. 94% were invisible for their own core topics. Here is exactly what they had in common, and how to check whether your site is one of them.

Table of Contents

• 1. What “AI search visibility” actually means
• 2. How we ran the study
• 3. The results at a glance
• 4. Mistake #1 — Your robots.txt tells AI to go away
• 5. Mistake #2 — Machines can’t tell what you do
• 6. Mistake #3 — You’re too slow for AI crawlers
• 7. Mistake #4 — Your content never answers directly
• 8. Mistake #5 — No citable facts, no llms.txt
• 9. Mistake #6 — Canonical and duplicate chaos
• 10. Mistake #7 — Zero trust signals
• 11. Check your own site free in 60 seconds
• 12. Frequently asked questions

1. What “AI search visibility” actually means

For twenty years, winning search meant ranking on page one. In 2026 it means something subtler: being the source an AI quotes. When someone asks “what is the best invoicing tool for freelancers” or “how do I fix a slow Shopify store,” the assistant does not show ten links — it synthesizes one answer and cites a handful of pages. If your site is not one of them, you are invisible at the exact moment a buyer is deciding.

This is the discipline people now call Generative Engine Optimization (GEO) or answer-engine optimization. The mechanics are less mysterious than they sound. AI assistants need to be able to (1) crawl your site, (2) understand what you are, (3) trust you, and (4) find a crisp fact worth quoting. Our study checked exactly those four things across 100 websites.

2. How we ran the study

We pulled 100 websites from a mix of industries — e-commerce, SaaS, local services, agencies, blogs and portfolios — and ran each one through the same Scanly audit that anyone can run for free. No site was hand-picked to look bad; we deliberately included sites that already ranked well on Google, because that is where the gap between traditional SEO and AI visibility shows up most clearly.

For each site we measured AI-search readiness across seven areas: crawler access, structured data, Core Web Vitals, direct-answer content, citable facts, canonical hygiene and trust signals. Then we cross-checked which sites actually tended to be cited by AI assistants. The correlation between “passes the checks” and “gets cited” was strong enough to be uncomfortable.

3. The results at a glance

Read those numbers again. Nearly eight in ten sites had no structured data. More than eight in ten had no citable facts or llms.txt. Almost two thirds blocked or failed to allow AI crawlers. These are not exotic, expensive problems — they are fundamentals most sites simply never tuned for a machine that reads and summarizes.

4. Mistake #1 — Your robots.txt is telling AI to go awaySeen on 61% of the sites we audited

The fastest way to disappear from AI search is to ask to be ignored. More than half the sites either blocked AI crawlers outright or never explicitly allowed them. Some had a leftover Disallow rule from an old copy-paste robots.txt; others still blocked GPTBot, ClaudeBot or PerplexityBot without realizing those are the exact bots feeding the assistants their customers now use.

If AI crawlers cannot fetch your pages, no amount of great content gets cited. Your robots.txt should explicitly allow the major search and AI crawlers, point to your sitemap, and block only genuinely private paths such as /api/ or /account.

5. Mistake #2 — Machines can’t tell what your business doesSeen on 78% of the sites we audited

Language models do not read your site the way a human does. They look for entities and relationships: who you are, what you sell, where you operate and what questions you answer. Without structured data, all of that is guesswork. Nearly eight in ten audited sites had no Organization, Product, FAQPage, HowTo or Article schema.

A consistent JSON-LD graph that names your organization, describes your offerings and connects your key pages is one of the highest-leverage fixes available. For a deeper walkthrough, see our guide to structured data for AI search.

6. Mistake #3 — You’re too slow for AI crawlersSeen on 55% of the sites we audited

AI crawlers and search bots work on tight budgets. If your Largest Contentful Paint drags, your layout shifts, or your Interaction to Next Paint is high, crawlers fetch fewer pages and leave sooner. Speed is no longer only a user-experience metric — it decides how much of your site gets indexed and fed to the models.

Over half the sites failed Core Web Vitals on mobile, the environment that matters most. Compressing images, deferring non-critical scripts and caching aggressively usually clears the bulk of it. Start with Core Web Vitals explained if LCP, CLS and INP still sound like alphabet soup.

7. Mistake #4 — Your content never gives a direct answerSeen on 52% of the sites we audited

AI assistants quote pages that answer a question clearly and fast. Most pages bury the answer under three paragraphs of brand storytelling. If a machine cannot find a crisp, self-contained answer it can lift, it cites someone who provided one. The fix is not more words — it is better structure: lead with the answer, use question-based headings, add a short summary box, and keep each section focused on one idea.

8. Mistake #5 — No citable facts and no llms.txtSeen on 83% of the sites we audited

When an AI writes an answer, it prefers sources packed with concrete, checkable facts: statistics, dates, prices, comparisons and definitions. Sites that publish only vague marketing copy give the model nothing worth quoting. Adding real numbers and clear definitions to your key pages makes you quotable.

Even more common was the absence of an llms.txt file: a plain markdown file at your domain root that lists your most important pages and facts in a format designed for language models. It is not a magic ranking button, but it is a cheap, accurate map of what you want cited. If you want to understand how these crawlers actually work first, read how LLMs crawl your website.

9. Mistake #6 — Canonical and duplicate chaosSeen on 47% of the sites we audited

Almost half the sites served the same page at multiple URLs — with and without www, with and without a trailing slash, or with tracking parameters — and had no clear canonical tag saying which version is authoritative. When signals are split, AI models see a diluted, inconsistent entity and cite a competitor instead.

Set one canonical URL per page, redirect legacy duplicates, and make sure your sitemap, canonical tags and internal links all agree. If you want a full sweep, our technical SEO audit guide covers canonical hygiene step by step.

10. Mistake #7 — Zero trust signalsSeen on 69% of the sites we audited

AI assistants are careful about the sources they recommend. Missing security headers, no author byline, no dates and no outbound citations all read as lower trust. A padlock alone does not create trust — a full set of security headers, a real author identity, transparent contact details and cited sources do.

This is where E-E-A-T and accessibility quietly overlap with AI visibility. Sites that scored well across security, trust and accessibility were far more likely to be cited in our tests — and if you sell in the EU, accessibility is now a legal requirement too. See our WCAG 2.2 compliance guide.

11. Check your own site free in 60 seconds

You do not need an agency or a budget to find out whether your site is one of the 94%. A free Scanly audit checks all seven areas above — crawler access, structured data, Core Web Vitals, content structure, citable facts, canonical hygiene and trust signals — and returns a scored, prioritized list of what to fix first. It takes about a minute and requires no signup.

If you run a team, agency or store with many pages, the deeper plans let you audit at scale, compare against competitors and export branded reports your clients actually read. But start with the free audit — the results are the fastest way to see where you stand.

12. Frequently asked questions

The gap is an opportunity

Here is the good news buried in a pessimistic study: because 94% of sites are invisible to AI search, the bar is astonishingly low. Fixing crawler access, adding structured data, speeding up your pages and giving AI a citable answer can move you ahead of almost everyone in your niche — often in a single afternoon. The businesses that act in 2026 will be the ones AI assistants recommend in 2027.

Find out exactly where your site stands. Run the free audit, fix the top three issues Scanly surfaces, then re-check in two weeks and watch your AI-search readiness climb.