How ad networks detect cloaking: they crawl the landing page with their own crawlers and enforce written policies. How the live systems compare views is not public. What follows is what is.
What the networks publish
| network | what | source |
|---|---|---|
| Google Ads | AdsBot checks desktop and mobile web page ad quality; user agents AdsBot-Google, AdsBot-Google-Mobile |
Google Ads help |
special-case crawlers such as AdsBot have their own IP list and reverse DNS names (rate-limited-proxy-*.google.com); AdsBot ignores the global robots.txt user agent (*) with the ad publisher’s permission |
verify requests, special-case crawlers | |
| Google Ads | cloaking banned under circumventing systems | policy |
| Meta | cloaking defined as different off-platform content for “our integrity systems” versus users | spam standard |
| Meta | facebookexternalhit crawls shared links and may bypass robots.txt for security or integrity checks |
Meta crawlers |
What a Google study shows
Google researchers built an anti-cloaking pipeline and described it in 2016 (Cloak of Visibility):
- Collect URLs to scan.
- Fetch each one with multiple browser profiles and network vantage points.
- Compare content, structure and the redirect graph of each fetch.
- Feed the differences to a classifier.
On a labeled set of 94,946 URLs it flagged split views with 95.5% accuracy and a 0.9% false positive rate. They applied it to 135,577 search and ad URLs tied to high-risk keywords and found 4.9% of the ad URLs (health, software) cloaked. The authors say they make no claim that Google fails to detect these URLs.
Study from 2016. The paper describes research, not the live ad review of any network.
How cloakers fight back
Same paper, ten cloaking services: IP blacklists, reverse DNS checks, user-agent word lists, JavaScript checks, referrer and country rules. See website cloaking for the table. Detection that only changes the user agent misses the IP-based ones.
Do it yourself
- Free: website cloaking checker.
- By hand: check a site with curl.
- Screening many advertisers: for ad networks.
[TODO: first-hand statement from Google Ads or Meta on how landing-page review works, if one exists publicly]