IP-based cloaking is when a site picks the page by the visitor’s IP address: a clean page for IPs it flags as crawlers, reviewers or researchers, the real page for everyone else.
How it works
From the Google study of ten cloaking services (Cloak of Visibility):
- Four embedded an IP blacklist; the others let the operator upload one.
- Three of the four mirrored the same blacklist-as-a-service, sold for a $350 annual fee, updated twice daily, with 54,166 IPs tied to search engines and crawlers at the time.
- Four of the ten also ran a reverse DNS lookup on visitors whose IP was not on a list.
- One list covered 51,713,860 addresses, aimed at security companies and researchers.
In the paper’s measurement of fingerprinting in the wild, “Is Google (IP, User Agent)?” scored 14.3% for Google Search URLs and 20.7% for Google Ads URLs.
The study is from 2016. [TODO: newer source on IP list services]
Example
Google’s docs show what a reverse DNS check reads: the IP 66.249.66.1 resolves to crawl-66-249-66-1.googlebot.com. Google tells site owners to run the lookup, check the name ends in googlebot.com, google.com or googleusercontent.com, then run a forward lookup that must return the same IP (Google docs). Cloaking services run reverse DNS on visitors too, per the study above.
How to spot it
- A user-agent-only test misses it. Test from different IPs: datacenter, residential, another country (view a site from another country).
- Our free checker runs from one datacenter exit, so it sees what a datacenter visitor sees. Country and residential views are a deep scan: contact us.
- For your own site, Search Console’s URL Inspection shows what Google fetched (guide).