Menu
Do you know any webapps/online tests/online firewalls that are trying to detect if user is using selenium/puppeteer/phantomJS or any other headless browser?
I've created my puppeteer online crawler. I've changed many different stuff like window.navigator object (user-agent, ~.webdriver etc.).
Now I want to make sure that it is undetectable.
264
There are many ways to detect whether a request is coming from a headless browser, but whether it will be easy to do so depends greatly on how the headless browser is configured. If used for web scraping, hackers do their absolute best to obscure detection.
On the one hand, Google works to detect bots with reCAPTCHA, but on the other hand, they've created Headless Chrome. Google's solution for blocking bots isn't effective against their own technology. For now, DataDome is the only solution I've tested that can detect Headless Chrome.
Headless testing is when you run a UI-based browser test without showing the browser UI. It's running a test or running a script against a browser, but without the browser, UI starts up.
Headless browsers provide automated control of a web page in an environment similar to popular web browsers, but they are executed via a command-line interface or using network communication.
There is a headless browser detection test which tests for the following:
navigator.webdriver set?window.chrome unset?navigator.languages unset?If your browser answers any of these questions with yes, then you fail the test. For more information on the test, check out this post, which is a reply to a post called "Detecting Chrome headless, new techniques".
The author of the latter post also published another test test (code), which claims to be able to detect bots and crawlers. It performs various tests on browser attributes and generates a fingerprint of your browser.
Other "soft" tests done by websites, might include the mouse movement, scrolling behavior, IP address, etc. I doubt you will find many tests regarding these information as this is basically a cat-and-mouse game.
82
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
