Market Research + Big Data + Predictive Analytics

A cursory glance at today's market research news makes it immediately clear that big data and advanced analytics are hot topics

So why all the hoopla? What is big data in the context of market research and what are some working examples? What is the real promise of big data market research (Big Data MR)? And most importantly, what immediate next steps can you take to harvest some near-term Big Data MR benefits?

Big Data MR: Why All the Hoopla?

If I had to point to three companies that have triggered the most buzz about big data, it would be Google, Facebook, and Amazon.

While there are certainly many other companies that leverage big data strategies, these leaders have hogged the headlines (for better and for worse) and turned consumer big data mining into a combined market capitalization of almost $1 trillion in a very short period. (As of this writing, their market caps are Google: $437B, Amazon: $244B, and Facebook: $262B.)

So, it's no surprise that there is huge interest in the use of big data for market research and other business applications, in order to significantly improve business results and overall market valuations.

Simply put, these and other companies are proving that if you infuse any analytical process with a huge chunk of consumer data and behavioral data — and you harvest it correctly — you stand to gain a tremendous competitive advantage. Ultimately, this advantage is based on your ability to serve your customers' ever-changing needs and demands (in ways that are better, faster, and cheaper than your erstwhile competitors).

A Working Definition of "Big Data MR"

Big Data MR is the art and science of combining consumer data, behavioral data, attitudinal data and advanced analytics to produce better and faster decisions that yield superior business results. It is the convergence of two disciplines — big data (transactions, orders, steps taken, images, etc.) and market research (and the analytical disciplines that go along with all of it) — that yields enhanced products and services, which in turn create a competitive advantage.

A classic application of Big Data MR is the use of research to understand the key drivers of some observed behavior. For example, let's assume that, using traditional data mining methods, you observe an alarming decline in your market share for a high-margin product in a specific geography. Using traditional MR methods, you then study a sample of customers exhibiting reduced purchasing behavior to uncover the attitudinal drivers of the change.

You discover three root causes of the slower consumption of your high-margin product: 1) economic fears, 2) new alternatives, and 3) new direct marketing from a competitor. 

"Big Data MR is the art and science of combining consumer data, behavioral data, attitudinal data and advanced analytics to produce better and faster decisions that yield superior business results."

These three drivers would not present themselves in your consumer or behavioral (or transactional) data. Instead, they are locked away in the hearts and minds of your customers, waiting for you to ask them the "why" behind "what" they are doing (or not doing). In other words, your data demonstrates the effects, while your research provides the causal underpinnings.

Finally, you uncover several preferred remedies from these same customers: 1) offer smaller quantity packages and 2) remind them of the real benefits of the premium product.

In the above example, Big Data MR is the observation of some behavioral change in your market or customer database followed by attitudinal research to understand what is driving that behavior. So how is this different from how companies behave today, where they see a sales problem in one set of data, and go learn about it in another? Predictive analytics allows you to precisely forecast how behaviors will change (and market shares will move) under different packaging, pricing, and messaging scenarios by directly linking the sales information to the attitudinal information you've collected.

The bottom line is this: the integration of multiple data sources and multiple analytical disciplines yields superior insights that can be immediately applied to improve business performance.

From Old Silos to New Success Stories

At a broader level, the typical silos of Big Data MR face some limitations when examined as separate entities.

Silo 1 — Market Research

This is the "why" silo. What are customers thinking? Why are they behaving this way? What do they prefer and will they act if we make changes? These are the questions addressed by primary market research. Clearly, this silo has a strong bias towards asking questions to uncover how to change and improve in ways that will also serve the customer better.

While MR alone has tremendous value to offer, it often lacks the ability to spot or predict behavioral changes at the earliest stages. Furthermore, MR's recommendations are not based on a comprehensive assessment of all customers and all customer data. Instead, it relies on opinions and stated behaviors from a very limited sample of customers (or prospects). With these limitations, MR only gives us part of the story.

Silo 2 — Big Data

This is the trendy name for a discipline that has likely existed a long time in your organization. At its most basic level, this silo could be considered your database of appended consumer data and customer transactions. It knows what your customers look like, where they live, what they purchase, which marketing and social media channels they prefer, their loyalty and engagement levels, and so on.

At the most sophisticated level, this can include massive data warehouses that collect real-time customer behavioral data, as well as dozens of other information streams (from internal and external sources). Today's relatively inexpensive computing power and data storage solutions have opened up new and more efficient analytical possibilities. This means you can apply all of the advanced analytics required to mine the data, predict behaviors, and implement real and practical business strategies that create better results.

The obvious Achilles' heel in the big data silo is the lack of attitudinal data—you can see what customers do (and predict what they will do), but you do not know the "why."

Big data has a very strong bias towards quantifying behaviors and predicting future outcomes and results based on historical views. The obvious Achilles' heel in the big data silo is the lack of attitudinal data — you can see what customers do (and predict what they will do), but you do not know the "why."

With this perspective on the two silos, it's easy to see how the convergence of both disciplines can lead to better insights, better decisions, and better business performance.

Big Data MR: An Early Harvest

So what can you do to harvest some real benefits from big data MR? Here are three steps to help get you started:

Step 1 — Bring Both Disciplines Together

This is not to suggest that you must immediately create a formal merger of the two, rather, that you at least get both groups talking. It sounds like a simple approach, but this isn't always the case. Many times these groups are fighting for shared resources as well as the allegiance of key internal stakeholders. Each fear that if the other group prospers it will result in fewer opportunities (and less limelight) for the other. It doesn't have to be that way. The right leader or bridge builder can proactively create an environment where both groups effectively collaborate. Today's visionaries know that 1 + 1 = 3 in the realm of Big Data MR.

Step 2 — Brainstorm Collaboration Opportunities

Once an alliance has been formed between big data and MR, begin to brainstorm real opportunities for collaboration and practical applications. What are the major corporate initiatives that both trying to serve? What are some specific projects that overlap in terms of objectives? If both groups are open and transparent, this will lead to many opportunities to create practical applications that will generate amazing new results.

"Focus on delivering early wins based on specific, practical and immediately applicable insights from your Big Data MR collaborations."

Step 3 — Find an Executive Sponsor

Now it's time to find a qualified sponsor or champion to support your recommendations. This is typically an executive with the broad view and political capital to see the opportunity and find the support (and budget, if needed) to help you. Put a real plan in front of this person, complete with benefits you can generate and specific projects you want to try. Start with a very specific project that gives you the greatest opportunity to succeed and truly demonstrate value from Big Data MR. Focus on delivering early wins based on specific, practical and immediately applicable insights from your Big Data MR collaborations.

Get Moving. Jump!

Hopefully you now have a better understanding of Big Data MR and how you can get started. It's a broad topic, but it has real value for almost every company and organization.

Build your alliances now and get started with a real project as soon as possible. If Google, Amazon, and Facebook can do it, so can you. Your first success story is just around the corner at the intersection of big data and market research.

Take Action Today >>

At Data Decisions Group, we are leaders in the integration of consumer data, behaviorial data, and market research data. Contact us today to discuss your market research and data needs.  

Click below now to request more information or a conference call with our experts.