For professionals in digital marketing, many of your metrics rely on some sort of measurement of views, be it visits, impressions, or reach. Those numbers are supposed to tell you how many people were exposed to your ads. But the majority of web traffic isn’t people; it’s bots. A 2014 report by Incapsula – corroborated by other sources – found that two-thirds of traffic on the internet was not human.
I’ve been speaking with Dr. Augustine Fou, an independant ad fraud researcher with almost 20 years of experience in digital marketing. He’s been trying to determine how to identify and stop bots from affecting digital ads.
When I talk about bots, I’m not talking about those bots that announce themselves as bots like search indexers like Googlebot and Bingbot. Those good bots properly identify themselves so that everyone else can filter them out of results. The other bots claim that they are humans surfing the internet. They try to act as much like a real person in front of a computer or mobile phone as possible, like the replicants in Blade Runner. And they can cause even more damage than those cinematic fakers.
Fake Visitors, Real Harm
Bots are a serious problem for digital marketing. Bot traffic primarily harms two sets of people, publishers and marketers. It’s not just inflating rates. These bots screw up the very metrics that Internet marketing builds off of. “Everyone knows bot traffic and fraud impressions mean wasted ad dollars,” said Dr. Fou. “But even more pernicious is how the bots mess up analytics, causing advertisers to inadvertently send even more money to their evil overlords – the bot masters.”
For advertisers, bots first and foremost cost money. The point of an ad is to get the advertiser’ message in front of a person who might want their goods and services. A bot won’t be buying anything. But the impressions and clickthroughs that they generate cost an advertiser money anyway. Combined with malicious techniques like stacking ads on top of each other and putting them in tiny iframes, an ad fraudster can really rake in the dough. eMarketer estimated that the 2016 marketing spend was $15 billion – a pretty big pie to steal from.
In chasing premium ad revenue, bot fraud cheats publishers out of premium ad traffic and ad dollars. Imagine you have a niche product for sports fans. Through targeted advertising, you could make sure your ad only shows up on sports related sites, like ESPN. If a human visits ESPN, they see your ad and get a tracking cookie. Bots, however, are liars. They can tell the ad servers that their traffic comes from any site they want, as the URL for ad clicks often contains the source of the traffic. And by getting cookies by visiting sites, they can pretend to have whatever interests pay the best.
For marketers, bots wreck their analytics numbers. You’d have no way to know how well a campaign works if over half of the impressions and clickthroughs were due to bots. Your clients want results for their media spend. They get far less for their money when all that spend falls into the hands of bot traffic. And you as the person responsible for creating their ad campaign will look far less effective and valuable.
But middlemen like ad buyers have a pretty big disincentive to cutting out bot traffic. Much of their money comes as a percentage of that media spend. If bots inflate the amount spent on media, they also inflate the amount the marketer gets paid. And imagine how upset that client would be when you tell them their impressions and clickthroughs dropped to less than half in an instant.
It’s not the big sites that cause problems. The long tail of the Internet holds most of the exploitative sites, those that abuse the ad networks. Back in the early days of the Internet, a scammer site could just copy existing content onto their sites, stuff it with ads, and wait for the search traffic to come flowing in. Now, they don’t even have to bother putting up content. All they have to do is flip a switch and send an army of millions of bots to click on their ads. They create the site and the traffic. It’s basically free money.
If this bot traffic is so harmful, why does it still persist? Dr. Fou points out the ugly truth about bot: they make money for the ad networks. “The more flow through, the more profits — for the middlemen.”
So the ad networks aren’t likely to stop bot clickthroughs. You need to do that yourself. But how?
The first method involves detecting bots through technology. Unless you get lucky and the bot tells you that it’s a bot, you need to get a little more clever. Dr. Fou works to find ways to spot a bot. When a user, real or fake, hits a website, it tells that site a fair amount of information about its browser, operating system, and more. Bots may be liars, but they’re not all great at it. Think of it like fake IDs: the ID may look good at first glance, but inspect it closer and maybe you see the hologram is missing or the ID number follows the wrong pattern.
The second means of attack goes after the business motives around ad fraud. Most of their money comes from serving up a ton of impressions really quickly. If you can change from a model that rewards quantity of impressions to rewarding business results, you can start to eliminate those views that don’t produce results. Right now, it’s absolutely worth it for scammers to buy AWS time to run bots to pretend and view ads. “Solving fraud is not hard – if advertisers focused on business outcomes and not on quantity of ad impressions purchased.,” said Dr. Fou. “Then whether or not it was fraud, advertisers would buy less of the stuff that didn’t work – i.e. didn’t drive more business. This automatically reduces ad spend on bots and fraud, because those don’t work anyway.”
Ad fraud is a real problem that eats up a significant piece of marketing budgets around the world. Recognizing how this problem occurs is the first step. After that, you’ve got to figure out how to stop it. In the next article on my talks with Dr. Fou, we’ll reveal various steps you can take to eliminate bot traffic.