Register

Marketing Research is not Rocket Science, but it is Science

It remains a fact of business life today: after the Great Recession, two and even three jobs were combined into one. This occurred just as Big Data (and small) was becoming part of everyone's job—whether or not they had any data experience.

This isn't about CRM data, which is a huge but totally separate issue. This is about the qualitative and quantitative data that's critical to every stage of product and campaign lifecycles. It's statistical science, the complexity and importance of which are frequently minimized in the rush to market.

It remains a fact of business life today: after the Great Recession, two and even three jobs were combined into one. This occurred just as Big Data (and small) was becoming part of everyone's job—whether or not they had any data experience.

This isn't about CRM data, which is a huge but totally separate issue. This is about the qualitative and quantitative data that's critical to every stage of product and campaign lifecycles. It's statistical science, the complexity and importance of which are frequently minimized in the rush to market.

Cheap DIY survey websites created by non-research developers have been used for years by managers in an effort to have some insights, instead of none. Sounds good, right? Not so much, when the likely (but often overlooked) result of this approach is:

  • Flawed survey execution
  • Biased and misleading results.

Many B2B marketers and consumer product managers who lack requisite background are given the task of gathering data. That's ok: when you give smart people smart tools they can usually figure it out and get the job done properly.

What tends to go wrong with a blind spot in data chops isn't typically about the individual; it's about the tool they're using. Inaccurate data from poorly designed DIY surveys is a titanic problem in business, and it's largely undiagnosed

Thankfully this is going away, as the latest self-service market research tools are debuting. They're faster, cheaper, more accurate, and they actually help the user create valid surveys by automating intricate data science.

What's Wrong With My DIY Survey Tool?
This can be answered with a question: who built it? From the late 1990s to the mid 2000s as web browsers, servers, and bandwidth made big advances, entrepreneurs jumped in with DIY survey tools. Ironically, almost none of these players were market research firms. They were mostly Internet startups with IPO dreams. They took a database programming approach, a simplistic UI/UX, and did their thing.

What's wrong with that? If you need quality data, quite a bit.

While the old DIY tools have certainly improved, most of these companies didn't have any expertise in market research to begin with. They approached it as a database solution, not a data solution. Huge difference. Accordingly, the old guard DIY survey tools were never able to perform the kind of concept rotations, quota assignments, randomization of responses, questions, screens, or groups of questions & screens, and other complex processing required to collect quality data. No disrespect—the older tools helped a lot of people over the years to move things along and shed some light. They just didn't do it very well.

Here's an example:
You have two concepts that you want to measure (basic A/B testing); you want to present an ad concept as a simple graphic, then on the next screen ask Purchase Intent and Attribute Communication. So you have two concepts, two screens containing three questions for each concept. What you need to do is group the concept A screens together, and group the concept B screens together, then rotate/randomize which concept group is evaluated first and second.

To add one more level of complexity, let's say you want to balance customers and non-customers across each concept so you have a readable sample of each group for each concept.

With many older DIY survey tools, you're out of luck. It can't be done. In the more advanced tools you may have the basic randomization or grouping ability, but it is locked by an Upgrade button to the most expensive service levels, and still falls short of the control you need.

Moving forward without proper survey flow control leads to something called “position bias” which registers more favorable ratings on the first concept than the second. This fundamental flaw in the older tools can present across many different types of research design.

New self-service survey tools allow you to easily group concept questions and randomize or rotate how they're presented. First concept first for some people; second concept first for other people. It's how you avoid position bias. Database solutions don't correct for this, because they're generally not built on statistical science. New self-service tools give you this advanced control at the free service level because that is quality research.

Here you can see an example of this functionality in use:

Go With the Flow
It's hard to build a system that stops somebody from making mistakes. For example, no automated system can know in a sentient way the qualitative content of the questions you're writing for your survey. That's why newer survey tools have actual data scientists available to answer questions. This alone is a serious enhancement.

Another strength of newer tools is managing the flow of survey creation from the perspective of a market researcher. It avoids logic flaws that can also induce bias. Yes, the older tools can perform skip patterns and randomization. But they don't necessarily allow you to group questions or group screens together so you can really manage extremely complex rotations. That's a key difference between data and database.

If your eyes glaze over at the mere mention of randomization or rotation against target quota cells, don't beat yourself up. That happens to everyone except market researchers. Their eyes twinkle when someone says “complex research design.”

Which brings us back to the question of “who built it” as it relates to DIY surveys and outmoded tools. If you need one hundred people to view a concept in a certain way, and then that one hundred people needs to be balanced by gender, age, ethnicity, and other variables, you need a data solution—not a database tool.

The intelligence of professional market research systems developed over years has now been put into a usable form. New survey tools are capable of executing complex research design; balancing randomization and rotations for concepts across your sample; and performing core functions like branching and piping. It sounds like jargon, but it's actually an industry sea change: powerful research systems becoming available to non-professional users in free versions marks a vast improvement over older DIY systems simply upgrading technical capabilities.

Add to this responsive design that allows you to build a perfect survey on any mobile device, and the new breakthroughs come into focus. It's amazing: twenty years of advanced research methodologies are now utterly native to mobile.

Here's a parting thought: when you crank out that next infographic, better to make it with a tool created by actual researchers. Otherwise, it's just you alone out there with a fancy database thingy, hoping for the best. That sucks.

At least you have options.

linkedin facebook pinterest youtube rss twitter instagram facebook-blank rss-blank linkedin-blank pinterest youtube twitter instagram