Fake Stats Abound. Why Do Writers Keep Repeating Them?
Tamilore Sonaike | May 18, 2023
Did you know head injuries usually don’t matter because 90% of the brain doesn’t actually serve any discernible function?
Hopefully not, because it’s of course, not true. Head injuries matter and should always be taken seriously. It’s absurd to think otherwise. Yet an influential first-aid trainer, once profiled by the BBC, holds this opinion and shares it with his impressionable classes. And can you guess where he got the idea from? The now famous idea that “we only use 10% of our brain.” (You can see the trainer’s clever math there.) The 10% myth has been around for more than a century and is so fixed in mainstream belief, it’s the premise of a Luc Besson movie.
And while it has been debunked repeatedly by neuroscientists, the 10 percent myth still thrives. And in fact, at the height of its popularity in the early 2010s, a study found that 48 percent of UK and 46 percent of Dutch teachers believed it. (Another sample found 65 percent.) It might make one wonder if we’ll ever be rid of such “viral” yet incorrect ideas? I would say, probably not, considering 87 percent of writers commonly misuse and perpetuate such incorrect statistics themselves. And that is how this article came to be; borne out of my frustration with having to parse through commonly reported statistical fallacies in things I found for work. I can’t tell you the number of times I’ve had to venture into the deep recess of the internet to validate a statistic because I’ve learned not to take any of them at face value.
In full disclosure, I’ve contributed to this problem as my earliest B2B articles will testify, and have only recently come around to this thinking. But it bears asking: What is it about data that such a high number of writers repeat without questioning? Is it willful, or just negligent?
The answer lies within how readers (read: all of us) are wired to think.
How our brain perceives statistics
In 2016, Professor Adam Kepecs, Joshua I. Sanders, and Balázs Hangya wanted to understand how statistics influenced human confidence. So they conducted a series of tests on human subjects in a controlled environment.
Their results? The human brain uses statistics to form the basis of decisions. Whenever there is incoming information, the brain uses mathematical principles to process it. For example, is it a big number or a small one? Are these two numbers alike or different? Is it as expected or anomalous? By employing calculations during decision-making, they found we tend to feel more confident in our choices. As Kepec’s thesis goes, “statistics—generated by the objective processing of sensory and other data—is the ultimate language of the brain.”
For instance, healthcare providers rely on patient data such as age, medical history, and current health status to make informed decisions about diagnosis and treatment. Stock market traders use statistics to identify patterns to support or refute their desire to buy or sell. Basically, we all calculate to decide.
Thus, we are pre-wired to believe numbers. Our brains hunger for them and we think they are objective evidence. Hence expressions like “In God we trust; all others bring data,” and, “Without data, you’re just another person with an opinion.”
I was curious to know how much statistics contribute to an article’s integrity, so I polled Twitter.
My results showed the majority of respondents (54 percent) found content with stats more compelling. My sample size was small, but SurveyMonkey conducted a similar poll (1,054 responses) and found 74 percent of respondents find content with data more persuasive than otherwise.
Data is, of course, not the problem. Numbers don’t necessarily lie. It’s our interpretation and reliance on them. As Nobel Laureate Ronald Coase said, “If you torture the data long enough, it will confess to anything.” Data can be manipulated, misleading, and sometimes spun out of thin air. The “10 percent” brain myth proves this point.
The most popular stats are frequently misrepresented
Skeptics of statistics are often quick to apply Mark Twain’s witticism, “There are three types of lies: lies, damned lies, and statistics.” I also wanted to, at first. But what if I told you the phrase is widely misunderstood? And also not originally his?
Like many others, I first thought Twain was ridiculing data. But if you read the preceding paragraphs in his biography, it turns out he wasn’t bashing statistics wholesale. He was criticizing the methods by which he interpreted his own data. His exact words were:
“I was very young in those days, exceedingly young, marvellously young, younger than I am now, younger than I shall ever be again, by hundreds of years. I worked every night from eleven or twelve until broad day in the morning, and as I did two hundred thousand words in the sixty days, the average was more than three thousand words a day—nothing for Sir Walter Scott, nothing for Louis Stevenson, nothing for plenty of other people, but quite handsome for me. In 1897, when we were living in Tedworth Square, London, and I was writing the book called "Following the Equator" my average was eighteen hundred words a day; here in Florence (1904), my average seems to be fourteen hundred words per sitting of four or five hours.
I was deducing from the above that I have been slowing down steadily in these thirty-six years, but I perceive that my statistics have a defect: three thousand words in the spring of 1868 when I was working seven or eight or nine hours at a sitting has little or no advantage over the sitting of to-day, covering half the time and producing half the output. Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force:
‘There are three kinds of lies: lies, damned lies, and statistics.’”
Initially, Twain thought his writing rate had declined because he was calculating based on the number of words he wrote per day instead of calculating words per hour. By factoring in the time dimension, his progress became clear.
(Side note: This further proves the need to verify every data and quote before use. The right context could change everything.)
Yet I find misinterpretations like Twain’s (and his misquotation) everywhere around the internet in articles for and about work. Errors range from the sampling methods not being accurate enough to provide clear results, to painfully outdated numbers, or to an instance of the author cherry-picking their favorite number out of convenience. Or perhaps the most common, the statistic is from a “study” by a company with no incentive to report the truth.
For example:
1. Does email marketing really generate $42 for every $1 spent?
Open an article about email marketing and you’re sure to find this one. It’s from a study conducted by the Direct Marketing Association (now Data & Marketing Association) and dotdigital, both of which provide email marketing services or information. Apart from the obvious conflict of interest there, this statistic also suffers from frequent misquotation.
The original report said, “ROI from email marketing now stands at just over £42 for every pound spent," but through translation and repetition, something happened: Writers switched the currency without converting it. I calculated the conversion rate to be an average of 1.2905 when the study was conducted, so this stat should actually read “Email marketing generates $54.20 for every $1.” (And still remains questionable.)
2. Did the average attention span truly drop from 12 to 8 seconds in 2020?
Our attention spans are dwindling at an alarming rate, it would seem, which appears to fit the world’s conception about phones and online media. The only problem? The source doesn’t exist, at least not on the internet. Although many big names credit its origin to Microsoft’s research, they are mistaken. Microsoft themselves credit the statistics-aggregating website Statistic Brain, which itself has no record of this data, and does not claim to validate data, only share it. So, where did it originate from? And have you ever read something for longer than eight seconds? (I’d estimate you’re about eight minutes into this article.) It doesn’t quite hold up.
3. Does a five percent increase in retention really equate to a 25-95% increase in revenue?
I first heard about this example from Fenwick CEO Chris Gillespie, and it’s one of the most dubious on this list. For one, the study behind it was conducted in 1990 on a single credit card company which is no longer in business, and is still being referenced by customer experience professionals more than 30 years later.
But that aside, let’s continue. The original report published in the Harvard Business Review stated “reducing defections 5% boosts profits 25% to 85%.” But somewhere along the way, the figure was altered and it’s now widely quoted as 95 percent, and as revenue, not profit. Why the inflation and adjustment?
Another problem with this statistic is its potential absurdity. Are you telling me a company discovered it could double its revenue (or profit) with a moderate change to customer retention? Is it perhaps … too good to be true? Don’t ask customer experience professionals. It suits their well-meaning but preconceived biases too fittingly.
4. Does content marketing really cost 62% less than traditional marketing?
I like to think of this particular one as the gold standard in content marketing tropes. It’s from a sketchy 2013 infographic by Demand Metric which cites no source and, furthermore, doesn’t provide any context. Is there a bright line between these two activities—marketing and content marketing? Is content marketing the money you spend on assets, or might it include the ads budget for promoting those articles? If we’re trying to tease the two apart, which group does the marketing manager behind its salary fall into?
It’s alluring. I understand why it is so commonly used. But it suffers the same lack of credible sourcing as all the others. And there are many, many others.
Why is the misuse of statistics so widespread?
A few weeks ago, I was reviewing a friend’s article and I noticed he linked almost all his statistics to roundup pages, which aggregate statistics from many sources, and not to the original. Curious, I asked him why. His reply, paraphrased: “Sometimes, there’s no link to the original source, or it leads to the brand’s website, not the study. Also, laziness.”
The last word, ‘laziness’ is the crux of the issue. Barring it, he might have dug up the actual sources. He’s not alone in this. Often, writers are not paid to take the extra step and dig deeper.
I should also note my friend seemed genuinely surprised he shouldn’t link to roundups. He thought it was acceptable. Many new writers fall into this category. They are either unaware of how to use data in content or the implications of using suspect data.
In my experience, most research reports are boring, jargon-laden, and difficult to understand. Take the earlier study on how humans use data to boost confidence and make decisions. I struggled to interpret that report. I spent hours going through it to avoid misinterpretation. Not everyone does the same. Some might give up and run with their interpretation, even if it’s incorrect.
I can’t help but mention the poor writers with impossible deadlines. Time constraints make it difficult to fact-check, so they often rely on reputable data sites like the data aggregate site Statista. Unfortunately, these sites are wrong sometimes too. Lily Ugbaja, a content strategist, shared how verifying one Statista statistic changed its context: What Statista claimed were “followers” were in fact “views.” Her incident is not isolated.
Another common reason is when the writer deliberately skews the data in their favor because the original doesn’t bolster their argument. It’s easy to see why they do this. Imagine spending hours hunting for a statistic, to only find it doesn’t live up to expectations. The temptation to alter the results in a world where you’re paid per submission might be too much to withstand.
What can we do instead?
A few years into my writing career, I have consciously, and with great effort, broken free from data misrepresentation. The turning point was when I stopped accepting projects or working with people who set unrealistic timelines. Without the pressure, I no longer have to cut corners. I have more time, so I verify every stat I use and link back to the original source.
The above option isn’t achievable for everyone, so alternatively, I prefer to quote subject experts to support my arguments. Their opinions go a long way in lending credibility to a piece and is certainly better than using thin stats. I source these quotes directly by reaching out to SMEs or lifting their words from their articles, videos, or podcast episodes. Or, I use websites like Help a B2B Writer to get relevant quotes.
“The turning point was when I stopped accepting projects or working with people who set unrealistic timelines”
My friends suggested Waldo and PrimoStats when I sought advice on researching faster. After trying Waldo, I found that Google worked better for me. Waldo is meant to aid people in finding quality information and data quickly, but I was overwhelmed by its interface and commands, so I returned to Google. Even though it wasn’t the right fit for me, others may find it more helpful because it comes highly recommended. PrimoStats is still a work in progress, and I am confident that it will soon be serviceable. However, it has not met my expectations as it is restricted to certain industries. Publications such as JSTOR and Google Scholar may also prove useful.
Breaking the statistical fallacy habit hasn’t been easy. I’m often tempted to take the easy road. But I’ve grown more conscious of the quality of work I put out. It might seem silly, but the thought of my work being a case study for bad writing practices is scary, so I avoid misusing stats.
What have I gained?
Well, for one, some of my editors no longer verify the stats I use. They’ve come to trust me, so I guess I’m building something of a reputation in this regard. The sense of satisfaction I feel knowing that I’m not contributing to misinformation is also great.
Where do we go from here?
The story of the 10 percent brain myth shows how misinformation spreads. Tantalizing but untrue statistics make for juicy reading and watching. But we must remember stories can be both thrilling and true. And that’s where we writers come in. We need to craft more of these types of stories.
Fenwick's straightforward approach to sharing information is one of my favorite things about them. Like they stated in their rebrand announcement:
“As writers, we have a responsibility to report things faithfully. If we don’t—if all that writers in business produce is propaganda—it distorts the world.”
The world already views marketers as dubious, and this industry’s shoddy data reporting practices don’t do much for our reputation. If we keep going down this road, I predict readers will come to distrust every stat they come across. For instance, they might no longer believe in the value of customer retention because the inflated figures have made it seem like it’s the end all to make profits. So if they still lose money despite their retention efforts, they become cynical.
But of course, it’s work. It takes a lot of time to go fact-check every statistic, like the one I used at the beginning of this article, when I reported that 87% of writers misuse statistics. Sounds true. May be true. But I made it up. Misinformation is easy to spread, huh? But I prefer to put in the work to fight it.
I’d encourage you to click more, trust less, and help all of us report the world more faithfully.
Want to keep in touch? We publish a twice-monthly newsletter about writing craft.
