I’ve documented it before but its basically on a range from
- No protections
- Basic Referer/User-Agent bypass
- IP rep block
- TLS impersonation
- simple CF challenge (cloudscraper)
- advanced CF challenge (flaresolverr)
- new advanced CF challenge (byparr)
solutions:
- TLS impersonation I plan to get around with add support for scraping with surf (tls impersonation) by feederbox826 · Pull Request #6806 · stashapp/stash · GitHub
- cloudscraper is dead so TLS impersonation is hopefully the bypass for that
- flaresolverr seems to be unmaintained but byparr is stepping in it’s place
CDP lies above TLS impersonation but below byparr/flaresolverr. Automation like CDP, Playright and Selenium are all detectable, see GitHub - ultrafunkamsterdam/undetected-chromedriver: Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM) · GitHub for previous attempts on undetected selenium webdriver.
The ultimate solution is
- residential SOCKS5 UDP proxy ($3/GB)
- up-to-date TLS impersonation
- fallback on byparr/flaresolverr for challenges
Playright/ Selenium will not work for CF “under attack” tunstile checks