VOL. 132 | NO. 186 | Tuesday, September 19, 2017
Don’t Give Me That Data-tude
Humans are a limited species. We cannot with any certainty, given the dynamic nature of life, predict the future.
In some cases you have better odds when running models and scenarios based on judging the future by analyzing past behaviors and patterns, such as insurance actuary tables or credit scores; however, most predictions are wrong.
The reasons are many, but the primary fault factor is that data are always limited and lacking context, and yet in the world of business and nonprofits, we act as if it is holy. For example, how many times have you heard, “Where is the data to support that?” or, “Is that just your opinion or is that statement backed with real data?”
Yet, an over-reliance on data and an underlying and unacknowledged fear of failure makes us give data a higher role than it deserves. This anxiety drives many middle managers to try and use data to predict the future – especially in the murky, gray areas of innovation, new channel development, and branding where markets and the people that make up the markets are psychological rather than logical.
A clear case of “innovator’s dilemma” perplexes many organizations. They yearn to launch something new, but because it is new, there are no data (no benchmarks, no analogues), so they lose courage and pass on the opportunity to crank out yet more of the same.
Data are one of the meta-games we play. Whether they are in politics, sports or markets, we take pleasure participating in data play. Yet most models trying to predict the future are wrong.
And guess what?
They are all based on data. Let’s look at some proof-based evidence in politics, market, and sports predictions.
Look no further than the Pew Research Foundation’s headline about the 2016 U.S. presidential polls: “Why 2016 election polls missed their mark.”
According to the Telegraph: “American statistics and polling analysis site FiveThirtyEight gave Trump a 30 percent chance of victory going into the final few days. This chance – the same as winning at rock, paper, and scissors – was generous compared to other forecasters.”
As for money markets, take the great Forbes headline: “It’s Official! Gurus Can’t Accurately Predict Markets.”
Then there are sports where the professional data analysts fare no better than amateurs with the fantasy leagues. Here’s how well the biggest services – Yahoo, ESPN and CBS – translated their player stats into player rankings. After the first nine weeks of the 2010 NFL season, here’s how each site’s projections ranked in overall accuracy (combines weighted scores for the QB, RB, WR and TE positions): Yahoo Sports – #18, CBS Sports – #24, ESPN – #39.
My point is simple: All of this reliance on data gets in the way of market testing to see how well something new actually works. Data provide excuses not to take risks but are mostly unreliable, save for a few fields.
Michael Graber, managing partner of the Southern Growth Studio, can be reached at southerngrowthstudio.com.