Curiosity Tank

View Original

Do you use numbers when reporting qualitative research findings?

Q: Do you use numbers when reporting qualitative research findings?
A: It depends and here's why.

Qualitative Research gathers data from fewer people compared to Quantitative Research. We are careful whether to include numbers of who did or said what when reporting on qual learnings. For example, stating "5 out of 7 people didn't understand X” could be very misleading.

One concern is that using #s from small sample sets implies a level of generalizability that we cannot achieve with fewer respondents.

Another concern is that people may say "the majority did not understand X.” Again, we cannot make these generalizations confidently. It's a framing issue.

Some teams have super strong opinions about this. If #s are used, the person presenting the data has a responsibly to clarify that they may not be more broadly indicative of the general population.

Saying that most people struggled with X, and the # is meaningful as a signal as an opportunity to do Y, is also okay IMHO.

If you do choose to report on #s, include a disclaimer in your report so someone reading it on their own, without the UXR to explain and prevent misinterpretation, is informed about the caveat.