Fingerprint surface-based detection of web bot detectors
In this paper, the authors examine browser fingerprinting to distinguish web bots from regular browsers.
They derive a method to define the "fingerprint surface" of a web bot, which they test on 14 different bots. The outcome of their experiment revealed identifying properties in the JavaScript environment for each of these bots. Based on these properties, the authors built a scanner that facilitates detection of web bot detectors to conduct the first scan of the prevalence of bot detection in the wild. Their findings give evidence that at least 12% of websites of the Internet perform web bot detection. Finally, they show that using bots leads to various deviating outcomes, such as missing advertisement, images, blockage and different page content.
The paper was accepted and presented at the European Symposium on Research in Computer Security 2019. One of the authors is Benjamin Krumnow, who currently pursues a PhD track at the Open University in the Netherlands and teaches in our Web Science programme. His supervisors are Stefan Karsch (TH Köln), Hugo Jonker (OU), Harald Vranken (OU) and Marko van Eekelen (OU).
by Hugo Jonker, Benjamin Krumnow and Gabry Vlot
February 2020
