An AI assistant does not browse your site the way a customer does. It reads whatever HTML arrives in a single request — no clicking, no scrolling, and no JavaScript — or it reads a copy of your page that was collected weeks earlier by a crawler. If the thing it needs is not in that HTML, or your page never reached the index it searches, the assistant does not guess. It says nothing about you, and recommends somebody else.
GPTBot, ClaudeBot and PerplexityBot do not run JavaScript. If your homepage ships an empty shell and fills it in after load, what those crawlers receive is the shell: a few hundred characters of boilerplate with none of your hours, address or services in it.
This is the most common serious failure we find, and the most invisible — the page looks perfect in your browser, because your browser ran the JavaScript. Fetch it with JavaScript disabled and you see roughly what an assistant sees.
Plenty of sites block AI crawlers without anyone deciding to. A security plugin adds a rule, a template ships with a blanket Disallow, or a well-meaning setting labelled "block AI scrapers" gets switched on. The result is that the assistant is refused at the door.
The agents worth checking by name are GPTBot and OAI-SearchBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot, Google-Extended and Bytespider. A group naming one of those beats the wildcard group, so a permissive User-agent: * does not rescue you if a later block names the agent.
Assistants prefer facts they can lift without interpreting a paragraph. Schema.org markup is that: a machine-readable block naming who you are, where you are and when you are open.
Two things go wrong. Either there is no markup at all, or there is markup with the fields left empty — "address": "" is not an address, and we count it as missing, because a crawler cannot quote it either.
This one sits underneath the other three, and fixing them does nothing until it is true. An assistant answers either from a search index it queries live, or from a public web corpus collected in advance. A site in neither is not a site with a visibility problem — it is a site the question never reaches.
A new domain with no links pointing at it is the usual case, and it is not fixed by editing your pages. It is fixed by being crawled and linked to, which takes time.
yoursite.com/robots.txt and search it for the agent names above.application/ld+json. If you find it, check the fields are filled in.site:yoursite.com. No results means the index has nothing of yours.The free scan on our homepage does the first three for you in about a minute, without an account.
Not necessarily. Ranking is about how a search engine orders pages it has already indexed. An assistant has to be able to read the page and find a fact worth quoting, and it will not run your JavaScript to do it.
No. There is no paid inclusion for ChatGPT, Claude or Perplexity. What they read is the public web, plus the search indexes they query.
Editing your pages is immediate; being re-crawled is not. A live search index may pick a change up in days, while a public web corpus is collected on its own schedule and can lag by weeks.
Yes — disable JavaScript and reload, read your robots.txt, and search for ld+json in your page source. Those three cover most of what goes wrong.
Where does your site stand? The scan is free, takes about 60 seconds, and needs no account. It fetches your homepage exactly the way GPTBot, ClaudeBot and PerplexityBot do and tells you what they receive.