Big data in the media industry: Overcoming a preference for Prophets, Pundits, Polemics, and Pedantics.

If you look closely at the use of data—big or small— in the media industry, you find (most likely) a preference for three make that four sorts of interpreters: Prophets, Pundits, Polemics, and Pedantics. For example, this post is itself an example of Pundit-style analytics.

Frankly speaking, this preference for one of four P’s makes the industry no different from societies-at-large. In just about any society, particularly as the underlying reasons for events become obscure, these sorts of analytical heroes surface.

Prophets—people who can see the future and are here to prepare us.

The insight on offer for many industry luminaries is their professed capacity to see or anticipate that which will happen next. Of particular interest to a prophet is truly the next big thing. These sorts of insights usually begin with phrases like, “The future of your industry looks like this….,” or “The DVD market will….,” or “Three dimensions of consumer demand in 2020….” Many people have built their careers on being prophetic once, or  maybe a few times, and being apparently correct.

The use of data by prophets is, well, very prophetic. A few crumbs of data that can be organized in such a way as to provide anecdotal evidence of a trend are all a prophet requires. Take a chart with a line sloping up or down and just extend the line into the future. Hang out with some “generation [blank]-ers” and describe their behavior as not only eventually widespread but also inevitable.

Let’s be clear and honest about data and analytics—neither can predict the future. The combination of the two can only hope to provide you with some better understanding of the past and present. That understanding might be applied towards the future, but only to the extent that the future is an awful lot like the past.

Pundits—people who can see the truth in the past requiring little more than a bar or pie chart.

Pundits are a powerful class of interpreters as they claim to have the ability to impute general trends from wonky data. Wonky data comes in many flavors, such as: rather bland (like a bar or pie chart) or skimpy (like a sample of ten) or truly sparse (like a single case study).

The favorite sorts of phrases from pundits sound something like: “This industry is failing because of X….,” or “This industry has a preference for three….”

Punditry is powerful because it usually taps into some hunch or belief we have in the underlying cause(s) for experience for which we might lack sufficient evidence (aka, confirmatory bias). The pundit provides a sort of social confirmation upon which we can rely to overcome the uncertainty in the data.  Explanations gets more highly weighted alongside support from a authoritative pundit.

The unfortunate truth is that no analytical method, at least none that is firmly stationed within the class of tools know as scientific or by falsification, will ever prove something beyond doubt. In fact, except in the case of certain methods that treat the counterfactual in a particular way, most tools—like the ever-popular regression—are actually setup to simply question the counterfactual (or null hypothesis). As a result, we simply are reasonably certain that the opposite of that premise may not be the case, or that the data seem to conform to some explanation/expectation within some margin of error.

In simple-but-convoluted english: Data and analytics never prove any premise. Get over it.

Polemics—people who only reveal analyses with the “right” conclusion.

Polemics are a powerful class of interpreters and their impact can be seen in the political debates of governments, industries, and organizations. To the polemic, data and analysis may be tools for understanding but are most importantly tools for influence.

So important is this influential dimension of interpretation that data/analysis that is not sufficient to “move the needle” will be discarded or, in some circumstances, massaged.

Rationally, when there are two sides to an issue and both appear to have support built upon decent data and analysis, its probably time to understand why and how the truth may involve a little bit of both sides from the debate.

Pedantics—people who only believe the sort of analyses that cannot exist

Pedantics operate on what is probably the opposite extreme from prophets and pundits as far as their analytical threshold. For the pedantic, pretty much any and every conclusion is lacking in sufficient rigor for a conclusion to be drawn.

Since no analysis that can be performed will every be a perfect analysis, and those trained in scientific (or highbrow, in the minds of some people) methods are highly sensitive to this inadequacy, the pedantic can shut down any analysis with simple comments like: “But what about….,” or “Now how big is this sample and is it sufficiently diverse to have included….,” or “I think we need a little more….”

And so, if we are going to make decent use of data and analytics, we have to accept that any and all methods come with a grain of error and maybe some salt as well. Even with a little salt, however, and because we can take error into account, these insights have value.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s