A few years ago, “Mr. Media” Thomas Koch cited Amir Kassei  stating that he does not see anyone in the market research industry who derives insights from the vast amount of available data to create real value. The truth ist that although Big Data is all the rage on client side, the industry has not done much more than putting up Big Data as a buzz word on the agenda while continuing to promote their traditional products.

As more than 80% of the German population is online and of which more than 95% use Google, there is no data source out there that is closer to covering the knowledge about consumer behavior. Of course, our opinion is backed up by Google. Evidence for a changing market is shown by the chart below presenting that Google searches for market research have continuously dropped over 68% within the last 10 years in Germany and international figures show similar results:

What should market researchers do?

Instead of optimizing their tools within traditional market research areas they should widen their perspective to see the big picture. Digital data of all kind holds many insights that – if integrated with traditional market research data and knowledge (!) – brings companies closer to their customers than ever before. To do so, we at td follow 5 key principles when conducting data science:

1. Customer first
Open the black box by arranging your data along the customer journey: New, digital data will help you connecting customer behavior with underlying attitudes. If done properly you will be able to understand the cause-and-effect chains that drive your business.

2. Do less! More often!
Too many huge market research studies focus on too many too detailed questions. While we do believe that it is important to understand drivers behind people’s behavior, we do not believe that these drivers should be understood in every market research study. To be able to isolate single impacts high data frequency is needed. New data sources can measure consumer behavior in real-time. If combined with market research results, always-on behavior becomes visible.

3. Better be roughly right than exactly wrong!
We are big fans of Nate Silver’s work and fully endorse his perspective on over-fitting models explaining incomplete observations. While they show high R²-values, they totally miss out underlying relationships and can only lead to wrong predictions by producing false positive results (see Silver 2013, The Signal and the Noise: The Art and Science of Prediction).

4. Enjoy the ensemble effect!
Joined in an ensemble of models, predictive models compensate for one another’s limitations. The ensemble (of models) as a whole is more likely to predict correctly than its component models are.

5. Stand on the shoulders of giants!
Re-usage of qualitatively proven theories (think of Rogers innovation diffusion theory as base to develop differing predictive algorithms based on the market adoption status of a product), combined with implicit industry knowledge and scientific state-of-the-art methodology will open your eyes on your data. While nobody can disagree with scientifically proven theories, integrating them into the story you let your data tell will help you make your insights become more viable.

What traditional market research needs now is change

In order to be future-oriented, market research professionals have to actively break down closed data silos. Those they have built themselves around their knowledge to make it easier for other professions to see the value of traditional market research. And those of other data professions by being more open to integrate data from other, new sources (e.g. web analytics, search, social, CRM) into their daily work to increase the value they deliver.

Therefore…

LET’S BREAK DOWN DATA SILOS TOGETHER!

ALL ARTICLES