When faced with a decision, we often look for reassurance that we’re making the right choice. Should I take this job or that job? Should I move here or there? The same applies to business: What product features should I prioritize before launch? Should we target this buyer or that buyer? Will this product be successful in the market?
While we can’t know anything with 100% certainty, mixed methods research is one way to mitigate the risks of making product decisions based on assumptions rather than evidence.
What is mixed methods research? What are the benefits? And how has it helped others make better product decisions? Let’s get into it.
What is mixed methods research?
Mixed methods research involves intentionally collecting and analyzing quantitative and qualitative data to understand a phenomenon or to investigate an inquiry. At Blink, we use this approach to examine different business questions from multiple angles, better understand what’s happening, and make more confident business decisions.
Blink’s mixed methods framework is one of the tools we use to design our mixed methods studies. It lists the most common research methods on a qualitative versus quantitative scale and also contextualizes studies based on which will tell us what people are doing versus what they are saying.
By choosing methods from various quadrants, you can ensure your methods and findings will complement each other. Each research method has its own strengths and weaknesses, so you can leverage the strengths of one method and supplement the weaknesses of that method with the strengths of another.
What is the difference between qualitative and quantitative research?
Qualitative research helps us understand why something is happening (i.e., Why are people not purchasing socks on our website? Why are people purchasing blue socks over green socks?)
Qualitative findings include deep descriptions of people’s emotions and stories. Qualitative studies typically have lower sample sizes and help provide more context to our findings by talking to or observing participants. Example methods include usability testing, in-depth interviews, diary studies, and focus groups.
Related: How Many Research Participants Do I Need For Sound Study Results?
Quantitative research, on the other hand, helps us understand what is happening (i.e., how many people are buying blue socks on my website?) Quantitative data collection and analysis often include numbers, measures, counts, and analytics. It encompasses “big data” and needs a much larger sample size for reliable results. Example methods include surveys, experimentation, and analytics.
Quant research is well suited to:
- Measure design or product improvements
- Determine how broadly research findings apply to a larger user base
- Predict the behavior of specific user groups in particular conditions
- Identify user or customer segments based on their behavior or sentiment
What are the benefits of a mixed methods study?
Every research method has its own strengths and constraints. By using qualitative and quantitative methods, you can leverage the power of one method while minimizing its limitations with another.
“To form a complete picture, both big (quantitative) and thick (qualitative) data are critical because they produce different types of insights at varying scales and depths.” —Tricia Wang, Why Big Data Needs Thick Data (2013)
5 reasons to mix methods
Get a deeper, fuller picture of your problem space
Mixing methods of research completes the story. You can learn how people feel about a product feature by talking to them about it (qualitative data), and then you can understand how many people share those same feelings among a larger user base (quantitative).
Scale customer personas
If you’ve conducted qualitative research to define customer personas, you can do quantitative research to see how and if they scale to a broader audience.
Develop and test your hypotheses
If your qualitative research is genuinely exploratory, it allows you to develop a hypothesis that can be tested further through quantitative work.
Reduce potential biases
Adding a quantitative component to your project can strip away some of the confirmation or cognitive biases more likely to slip into qualitative research.
Action your data with confidence
Let’s say your website analytics (quantitative data) show that 50% of users leave halfway through your sign-up process. If you know there’s a problem with your sign-up flow but don’t know what it is, a usability study (aka qualitative data) will give you a better ROI because you can find out why they are leaving instead of guessing why.
“Relying on big data [quantitative data] alone increases the chances we’ll miss something while giving us the illusion that we know everything.” —Tricia Wang
Common ways to mix methods + client examples
Example 1: In-depth interviews (qualitative) + survey (quantitative)
Background
Our client was launching a new product that bundles niche streaming services into a single subscription. To inform their product roadmap, we set out to better understand the motivations and decision-making considerations of people who subscribe to niche services like Acorn, BET, and MLB sports.
Research Overview
First, we conducted 45 in-depth interviews with niche service users to collect deep, qualitative feedback about their motivations, viewing habits, and subscription behaviors.
After the interviews, we wanted to understand the scale of our findings (i.e., how many people shared the same feelings as our interviewees?), so we designed a survey and collected 2,000 responses. The survey helped us understand the prevalence of our findings and gave our clients a clear roadmap for the features they needed to add or improve before they launched the new product.
Example 2: Hardware usability (qualitative) + statistics (quantitative)
Background
Our client came to us with a question: When it comes to augmented reality (AR), how “real” is real enough? Hardware studies often involve qualitative evaluation as well as quantitative data collection, so to answer our client’s question, we conducted in-lab user testing with 52 VR users.
Research Overview
During the usability test, participants quantitatively rated how real objects looked in specific AR settings. Afterward, participants answered a qualitative survey, which helped our team understand which variables make the objects look more real and why those variables make them look more real.
Example 3: Benchmarking (quantitative and qualitative)
Background
Our client wanted to find out how their product compared to a competitor’s product in order to inform their strategy and become a market leader. We chose to conduct a benchmarking study. Benchmarking involves evaluating a product’s usability using qualitative and quantitative metrics and then comparing those metrics to the competitors.
Research Overview
We conducted 600 1:1 remote interviews with students and teachers across the US and Australia. During the sessions, we collected quantitative data, such as task completion rates, and qualitative data about their experiences while completing the tasks.
Confident decisions lead to captivating products
By conducting mixed methods studies that leverage qualitative and quantitative data, product designers and managers can tell a more holistic story about their customers and build better experiences they use, love, and remember.
Need to figure out the best design for your research study? Contact our team of expert researchers.