7 Reasons Not To Do User Research with Michele Ronsen
We chatted with Michele Ronsen about when companies should pause before doing qualitative user research
*This post was originally published at User Interviews.
When Michele Ronsen posted her “7 reasons not to do user research” we knew we had to chat with her on the podcast. As user research’s biggest fans, we wanted to chat with Michele about why teams could possibly not want to spend time doing user research. After all, we’re all about using the right tools and methods for the right job—user research isn’t the right tool for every job (though we do think it is way underused!). Michele had a lot to say about the steps teams should take before engaging in user research and when your time could be better spent focusing on something other than qualitative research.
Erin and JH talked to Michele about some of the reasons a company may not want to do user research right now. She provided some really interesting insight, drawing from her experience running her own consulting company, Ronsen Consulting, and teaching new UXers at General Assembly and Berkeley. Below, 7 reasons not to do user research.
1. When you can answer your research question with analytics, secondary research, or by talking to your team.
User research is not omnipotent. It’s not the best way to answer every single question your team could think of asking. In fact, Michele finds that many questions don’t need to be answered through new qualitative user research at all.
When Michele consults with a new client, she likes to start with what they already have. Learning about the analytics they collect, the research they’ve already done, and the help tickets that come through their customer support helps Michele learn a lot about the research that needs to be done. It also helps her learn about the problems the company may be facing without spending extra time on research.
There are many different ways to answer research questions, and choosing the right method is the first step to conducting great research. Michele has created some templates to help people find the right method for their research. Research methods vary greatly, from quick first click tests to in-depth ethnography studies. Choosing the one that best answers your question is a part of ensuring you're using your time and budget wisely during the course of your research project.
2. When you don’t know who you should be talking to.
Not everyone can answer every question. For example, I, a 25 year old Content Creator from Atlanta, GA, probably can’t tell a business owner much about pool installation in Chicago. Since research budgets are tight, and time tables can be even tighter, it’s important to ensure you’re asking the right questions to the right people. This helps your team get the best, most helpful insights from participants.
Michele told a story on the pod about how business owners sometimes don’t make the distinction between feedback and feedback from their target audience.
“I had someone approach me and ask if I would be interested in collaborating on a project with my General Assembly students, and I asked them to tell me a little bit about the project. He was redesigning a website for a makeup provider, kind of like a Sephora-but-not type of retailer. And in conversation with him, he thought it would be a good fit because it would give my students exposure to a real world project, and it would be a good fit for him because he would be able to gather a ton of actual feedback. And I was curious by the requests because, to me, that's not at all the people that he should be gathering feedback from.
Because first of all, half of my General Assembly students are male, and my hypothesis is that most people buying makeup online are female. Secondly, the demographics of my students are probably more educated and perhaps more tech savvy because of where we're located in the Bay Area, and that might not fit their core buyer.
But he was very surprised, and he kind of picked and poked at me and said, ‘But don't you think this would be a great experience?’
And I said, ‘I think this would be a great experience of what not to do.’
And it turned out to be a very fruitful conversation, and he thanked me very much, and he said, ‘I just, I never really thought about it like that.’
And I said, ‘Gathering the right information from the right people is kind of... That's foundational. We don't want to ask the wrong people questions because we're not going to gather meaningful feedback.’”
So how do you know who you should be asking your research questions to? If you already have a product out in the real world, it’s likely that you have some data on who your customers are. From there, you can break them out into personas, find your best customers, or settle on a target market.
For example, if you want to learn more about how to reduce churn in your business, it’s probably best to talk to recently churned customers. If you’re trying to learn about how to launch a new feature to your target market, you can talk to people who don’t use your project, but fit the same description of your target market, or your current customers who could benefit from your new features. You can work with your analytics, customer support, or even marketing departments to use the data you already have to figure out who you need to talk to next.
3. When you don’t know what you want to learn, or how you’ll use those learnings.
Doing user research for the sake of doing user research isn’t the best use of your time. Michele sees this a lot with her consulting clients. Sometimes, a client will come to her for her help with research, but they’re not sure what exactly they want to learn or how they’ll implement their learnings.
“If you don't know why you're doing the research, I don't think you're going to be able to build a great plan and ask the right questions. Knowing that end goal will really be informative, and this is also a key reason to get your stakeholders involved, too, because your stakeholders should all understand why you're doing it as well.
So, if we didn't want to do a general exploration, that's fine. Maybe we're doing it to become a little bit more informed about a product or service or a new profile or target that we might go after. So, maybe we're doing it sort of in a generative way so we can become a little bit more informed about what could be. That's a totally fine answer to why.
But if we're doing something a little more tactical, or we're doing something that requires any sort of task based something or evaluative based something or generative something, we want to know why we're doing it. If we can't answer the question why, I would suggest abandoning ship at that point. And also, if you can't agree on why, there may be more than one why, and that's fine, but if we can't agree on the why, then we're not going to be moving forward in lock step.”
It’s important to start research with a specific, actionable, and practical research question. This means it’s specific enough that you can answer it within the scope of one research study, actionable in that you can reasonably create a solution, and practical in that it is achievable in the time frame and with the resources you have available. Zeroing in on the right research question is the first step in doing research that helps your business grow.
4. When you don’t have enough time to do qualitative user research.
Sometimes, projects have tight timelines, and there may not be enough time for your team to do comprehensive qualitative research. In these cases, Michele suggests that teams use analytics or customer support data they already have, conduct secondary research, or conduct quick unmoderated quantitative tests.
Unfortunately, “we don’t have enough time,” is often a team’s excuse for not doing research in the first place. Michele suggests reserving this particular reason for really extreme situations, like one or two day turnarounds. She also noted that, even if a project doesn’t leave time for research, it’s important to take the opportunity to address how you’ll make time for research in the future.
Michele suggests implementing an ongoing research program, with time allocated for research on a regular basis. Even blocking off an hour or two a week for research sessions can help teams keep users front and center. This also means your team won’t be able to fall into the “we never have time for research” trap.
5. When you’re going through the motions to prove your solution is right.
Sometimes, people on a team can push for research just to validate their own ideas for the product. Many times, this isn’t malicious or even a conscious choice, it’s just something people do. But if you’re only doing research to validate your ideas, what’s the point of doing research at all?
It’s pretty easy to determine whether or not you’re doing research for the good of the users or to validate your solution. Michele suggests asking everyone on your team why they want to complete this specific research this way. Using the 5 whys, you can uncover more about the goals of your teammates during research, and ensure you’re in it for the right reasons.
6. When you don’t have stakeholder buy-in.
Without stakeholder buy-in, your research won’t get very far. Michele pointed out that, unfortunately, budgets are a thing and without buy-in from stakeholders, research is unlikely to get the resources it needs to do well. She also pointed out that, without stakeholder buy-in, research isn’t as likely to be heard and absorbed by the rest of the team.
If your stakeholders aren’t on board with your research plan, take some time to sit down with them and learn about how you can work together to make research valuable to your organization as a whole. Zach Lamm, UX researcher at SoFi, recently sat down with our team for an interview about stakeholder interviews (very meta). He suggested asking stakeholders three key questions about what they expect from research. Here they are—
What motivated the stakeholder to come to research with this question in mind?
How does answering this question fit into the broader context of the business?
Above all, what problem are we trying to solve for users?
These questions help you learn more about what stakeholders expect from research, and helps you understand what you need to do to win their buy-in. Some stakeholders will have unrealistic expectations of research, which means you’ll need to do a bit of work setting clear boundaries and expectations about what research can and can’t do. Other stakeholders may be skeptical about the value of research, or be confused about what they can expect from it at all. In these cases, you as the researcher can lay out the benefits of each research project and present a clear plan for how other teams can use research to learn and grow.
7. When you haven’t right-sized the question.
Some questions are just too big for user research to answer in the scope of a single project. Questions like, “What do our customers want from their real estate providers?” or “What’s the best way to build a product discovery app?” are too big to be answered in the scope of a single study. That’s why Michele suggests right-sizing your question before starting user research. That means that your research question is actually something you can answer during the scope of your project, with the resources you have available.
We’ve found that the best way to ensure your research question is practical for your project is to make sure it is specific, actionable, and practical. This means, rather than asking “What’s the best way to build a product discovery app?”, you can ask questions like “How do people discover new products right now?”, or, “How often do people search for product discovery platforms online?”. These questions are specific enough for you to know when you’ve found the answer, practical in that they could be answered within the scope of a single study, and actionable in that you could do something with the answer you find.
Right-sizing your research question means you’ve considered not only whether or not you can answer your question, but whether or not you can answer it in the time frame you have with the resources you have available.
For example, to answer the question, “How do people discover new products right now?”, you would likely need to do at least 5 generative interviews with early adopters, which would likely take a day or two to complete. With the project preparation, participant recruitment (which, on average, takes about 3 hours with User Interviews 😄), completing the sessions, and coding your notes.
In contrast, “How often do people search for product discovery platforms online?”, could be answered in an afternoon, with no outside help. You can use tools like ahrefs to find the search volume on a product discovery platforms, compile a list of the most relevant results, and have information compiled by the end of the day.
Still struggling to understand when’s the right time for user research? Check out this little flowchart we made to help you decide when you should start research, and when you should take a step back.
Want the full transcript of the interview? Find it at User Interviews.
