Is my website blocking AI?
Possibly, and you would not know from looking. A site can load perfectly for a person and still turn away the crawlers that ChatGPT, Claude, and Perplexity use to read it. The block usually comes from a hosting or security default rather than a decision you made, and Google is often allowed through while the AI crawlers are not. The only reliable way to know is to fetch the site the way a machine does and see what comes back.
Two separate systems decide whether a machine can read your site. The first is robots.txt, the file where you state which crawlers are welcome. The second is the security layer in front of the site, a firewall, a bot filter, or a content delivery network, which can refuse a request before your server ever sees it. Most owners only ever check the first. Our study of 252 Northwest Ohio business websites found 17.9 percent blocking an AI agent at the security layer while their robots.txt welcomed it. Google’s own crawler is usually on the allow list, so search rankings hold and nothing in Search Console changes.
Where this matters
Blocking the crawlers that train models is a legitimate choice and costs little visibility. Blocking the crawlers that fetch and cite is what removes a business from AI answers, and the two are often caught by the same broad rule or the same default. Since mid-2025 new Cloudflare accounts block AI crawlers unless told otherwise, and several hosts ship a bot wall that challenges anything that cannot run JavaScript, which describes every AI fetcher. A person in a browser passes that challenge without seeing it. If your site sits on a managed platform, the switch may live in the host’s control panel or behind a support ticket rather than anywhere you can edit. Check first, then decide which crawlers you actually mean to keep out.
To find out in about a minute, run the AI visibility check; for why the block happens and what to do about it, see how AI describes your business.