Lying with Statistics: A Primer for Marketers and their Clients
“If you can’t prove what you want to prove, demonstrate something else and pretend that they are the same thing”
These words of wisdom were imparted by author Darrel Huff in a quaint little book called “How to Lie with Statistics”, published in 1954. Huff’s book introduced readers to statistics and their misuse in day-to-day life, explaining how media, advertisers and government routinely manipulate data to create confusion and mask reality. If you don’t engage in such deception yourself, you may not be hip to how data manipulation works – or its impact on your marketing, sales and brand strategy. But it’s time you learned a few things.
Lying with Statistics: Here’s How it’s Done
Let’s say the City of San Francisco wants to cite the average annual income of its SOMA neighborhood, a place with two or three millionaires, a majority of average- income residents and a few cash-strapped retirees. If the city wants to indulge in a little recreational lying with statistics, how could the City manipulate the data represented by this classic bell curve?
1. Poor or Biased Sampling:
Biased sampling in the data collection phase is often used to manipulate statistics. For example, if the City wants to represent SOMA as wealthy neighborhood, it could engage in biased sampling and provide skewed results that present millionaires as the norm.
2. Data Analysis and Interpretation
Data analysis is a subjective task that leaves huge room for misinterpretation. This slippery slope can be demonstrated by the three types of averages used by statisticians: the mean (sometimes called the average or arithmetic mean), the median and the mode. Depending on which is used to analyze data, the City could truthfully claim that SOMA has an average income or $170,000, $120,000 $70,000. But how can that be?
The Statistical Lie: Mean, Median and Mode
When lying with statistics, playing with averages is a classic move. Unless you crunch numbers every day or are fresh out of college, you may not remember the difference between the different kinds of averages. Here’s a refresher:
- Average or Arithmetic Mean: If we add up the annual income of every SOMA household and divide it by the number of households, we’ll get an average annual income of $170,000. That makes SOMA a pretty swanky place to live.
- Median: If we rank numbers from highest to lowest and then find the middle number, we’ve identified the median. In our example, the media would give us an average annual income of $120,000, a good number to report if household samples were normally distributed – which, in this case, they aren’t.
- Mode: If we go with the most common household income, the average annual income of SOMA is a middle-of-the road $70,000, arguably a more truthful number than the mean or median.
Keep in mind that swapping around the way that you measure the “average” is a pretty obvious, if common, move. If a marketer is lying to you with statistics, chances are they’re using a more subtle method… like irrelevant metrics.
In the past, it was difficult for companies to attach hard numbers to their offline advertising channels such as TV and print ads. While increased advertising often led to higher profits, it was all but impossible to tie unique leads back to a specific channel and determine which channel performed better. However, with the rise of digital advertising, it’s now possible to measure the full impact of online advertising efforts. But here’s the catch: even with this new capability, many agencies are still reporting only high-level metrics like online impressions and clicks. What gives?
Put on the Happy Face
Marketers trying to keep their clients happy will often cite impressions and clicks to paint a rosy picture about their client’s brand. The problem is that lying with statistics like this can’t disguise the basic truth of whether or not marketing is adding to the bottom line.
For instance, I could tell my client that their $10k investment in three online channels has doubled their ad impressions and increased clicks back to their sites fourfold. Wouldn’t that be great? Well, if the client ramps up their budget on the channel that drives the most traffic to their site – while ignoring leads, revenue and other key metrics – it may not be great for them after all. And it certainly won’t be great for me either, once I find myself out of a job.
Speaking Truth to Power
To get the most bang for your marketing buck, we suggest looking at data from a different angle. If marketing initiatives are assessed according to how much they increase revenues, rather than impressions or leads, companies will have an easier time separating real value from misleading statistics.
Under this approach, page views, fans, clicks, conversion rates, and other interim metrics are simply a means to an end.
Marketing agencies that can quantify inquiries, MQLs, SQLs and revenue out of converted prospects can make smarter decisions and optimize marketing spend to increase profit – the ultimate marketing goal.
The Good, the Bad and the Ugly
Marketing agencies that use revenue as a measuring stick are positioned to give their clients the unvarnished truth about their marketing efforts. Sure, revenue-based numbers might not be as flashy as telling clients that they’ve quadrupled the number of clicks to their site. But this approach affords less room for misinterpretation and brings a transparency to the client-agency relationship that builds trust, increases collaboration, and fosters a sustainable business model. When it’s all said and done, both the agency and their client will be set up for success.
In fact, marketers who stop hiding behind lying with statistics will not only keep their clients over the long-term, but also increase referrals and win new business.
As noted by the renowned British politician and writer Benjamin Disraeli, “there are three kinds of lies: lies, damned lies and statistics.” But armed with marketing knowledge and some healthy cynicism, you don’t have to be fooled by the numbers. Rather, you can turn them into meaningful knowledge and actionable strategies to help your business thrive.
- UTM Parameters Explained - November 11, 2016
- Lying with Statistics:A Primer for Marketers and their Clients - July 1, 2016
- Google Analytics 360 vs. Google Analytics Standard: Which is Right for Your Company? - May 27, 2016