Cloudflare has announced a new tool to help customers detect and analyse the presence of bots on their website. Called Bot Analytics, the new solution provides more detailed detection, informing users of how many bots are present and whether they can be classified as ‘good bots’ or ‘bad bots.’
Although bots have a bad reputation online, not all of them are malicious. In fact, many helpful services that employ some level of automation use bots. These are completely above board and as long as they are open about the practice, shouldn’t be prevented from visiting any sites.
On the other hand, ‘bad bots’ can be an unwelcome presence. They may be hoarding inventory or scraping content from a particular site. However, because bots represent an estimated 40% of all online traffic, determining which bots to block and which ones to allow isn’t easy. This is where Cloudflare’s Bot Analytics comes in.
Bad bots
“Beginning today, we are going to show you the bots that reach your website,” Ben Solomon, product manager for Bot Management at Cloudflare, explained.
“You can see these bots with a new tool called Bot Analytics. It’s fast, accurate, and loaded with information. You can query data up to one month in the past with no noticeable lag. To accomplish this, we exposed the data with GraphQL and paired it with adaptive bitrate (ABR) technology to dynamically load content. If you already have Bot Management added to your Cloudflare account, Bot Analytics is included in your service.”
Boy Analytics can be found under the Firewall tab of the Cloudflare dashboard. Enterprises without Bot Management access will only see a snapshot of their bot traffic, but that should still be enough to determine whether bots are swarming all over a webpage.
Website admins can also set their own bot threshold to determine how much automated traffic they allow. Of course, there’s no sure-fire way of keeping out all bots, but setting the right limit should ensure that more bandwidth is given to genuine website visitors.