You know that old family recipe you’re trying to make for the first time? That semi-legible, written-in-cursive, almond-extract-stained, only-lists-ingredients, and has been adjusted over the years by memory, recipe? That, “no matter what you do, doesn’t turn out how you remembered it,” recipe? 

It’ll always be a disappointment because you don’t have the complete story; you have all the right ingredients, but no measurements to go with them. Everything you’re doing is a guess, which more often than not doesn’t result in the outcome you desire.

The same applies to how you go about building your website, or any other user-facing platform. Just because you interviewed someone who said “I don’t like making payments online”, does that mean you should stop supporting online payments? You could, but I bet that would be a risky move for your business. 

Instead, you could look at traffic to the payments page, how many people submit online payments, how much money you make through the online payment platform, and how many people drop off before they finish paying. Just like the measurements to the ingredients in that old family recipe, this is the quantitative data that needs to be balanced against the qualitative data collected in your interviews.

Let's Break It Down. What Is Quantitative Data?

In short, it’s the numbers. How many people do this, how long they spend on this page, how long it takes the page to load. This data, especially in an analytics dashboard, can show you trends from a larger audience over a set amount of time. This is the type of data that resonates with your stakeholders, executives, and clients; it’s tangible, not data based on opinion or emotion. As such, this type of data can also be used to track your progress towards a goal, or measure the impact your changes have made.

In our scenario the quantitative data could answer,  “What percentage of people pay their bill through the mail?”. Maybe you’ve spoken to 3 people who all pay their bill by check, but when combined with your site traffic data, you find out that they represent only 5% of your total customer base. See how your perspective would’ve been very different without this data?

Pro Tip: Make sure the person analyzing these results is trained to handle this type of data; they’ll know what to watch out for, how to clean up the data, and how to interpret the results.

What's That Other Stuff? The Qualitative Data?

This is the part where you can really get to know your user base; understand their motivation, see how they accomplish a task, and uncover why someone does something the way they do. Maybe you’ve heard of the ‘five whys’ - this is an activity that systematically drives deeper insights. In our example, you shouldn’t just accept “I don’t like making payments online” as the answer - make sure to ask why, so you’re not just assuming it’s because they don’t like your payment system. Maybe that conversation goes something like this:

“Why don’t you like making payments online?” 

“I prefer to mail a check.” 

“Why do you prefer to mail a check?” 

“It’s easier for me.” 

“Why is it easier?” 

“It’s faster.” 

“Why is it faster for you?” 

“Logging in is a hassle.” 

“Why is it a hassle?” 

“I always forget my password.” 

Aha! By asking why, you can see that it wasn’t your payment system that this person disliked, rather their actions were driven by a misplaced password. Building a “quick pay” option that doesn’t require signing in may tempt them (and potentially thousands of others) to pay online.

Pro Tip: Always consider how you might be biasing your participants. Don’t lead the interviewee into the answer that you want, intentionally or not. For example, ask questions that start with “How do you feel about …” rather than “Do you like…”.  Your word choice is very important.

This Seems Like A Lot Of Data. Why Should I Care?

Just like in Grandma’s recipe, you don’t want to be looking at an incomplete picture. By having both sets of data to balance each other, you’ll get a whole new perspective. Qualitative and quantitative data have a symbiotic relationship.

• Let’s say you see a drastic change in the web traffic. That’s what the numbers tell you, but isn’t your next question ‘why is that happening’? You can more fully answer this with qualitative data. Dig further, talk to some users about their experiences on the site, check the news for impactful events, ask a developer if something was broken. There’s always an explanation for the quantitative data, you just need to ask the right questions.
• Conversely, let’s say that a unique perspective comes up during user interviews. You may want to know if this is a popular opinion among all your users. To mitigate the risk of basing a decision off of one user, you can use quantitative data to help validate or add color to the hypothesis that came out of your interviews.

If you don’t have the complete picture, any actions you take present a new risk. You may not be solving the right problem, or you may make the problem worse.

This Is Cool And All, But How Do I Begin?

Start small and deliberate. You don’t need all the data in the world. Think about what questions you have that you’d like an answer to. If we learned anything from the ‘Dear Data’ project, it’s that there are multiple ways to collect data about the same topic, and you can turn even the most mundane things into a meaningful data set (and create cool visualizations along the way).

Collecting qualitative data is easy:

• Go talk to your users about their goals, what they need, what they think about your platform currently. 
• Record what you observe about how someone uses your platform, any struggles they may have. 
• Talk to someone who doesn’t use your platform, find out what’s stopping them, or what’s missing. 
• It’s important to make friends with your UX and Research teams because they do this for a living.

To start collecting quantitative data:

• Think about the goals your business has and what is important. Try Goal Mapping!
• Keep in mind that you should also be protecting the privacy of your users, so only collect the absolute minimum data that you need. 
• Chances are your company probably already has some of the data you’re looking for but it just might be spread across multiple different teams, so…
• Make friends with your Analysts, Developers, and Data Scientists - although they may be hiding in a part of the company you didn’t know existed, they are likely already tracking and have access to a lot of data. They can get you the quantitative answers you need!

While I can’t help you with perfecting Grandma’s signature dessert, if you follow this recipe to see the complete picture, I guarantee that bringing both qualitative and quantitative data to the table will strengthen your insights.