Curiosity Tank

View Original

Do you pilot?

michele ronsen

Pros pilot all things. Every time.

Test flight, test drive, dress rehearsal, beta test, practice run — no matter what you call it, one of the most important steps in a research study happens before you gather your first bit of real data: piloting.

Piloting is not just for beginning researchers. I've probably conducted over 1500 interviews, and even though I'm very senior at this point and confident in my skills and judgment, I still pilot EVERY SINGLE TIME.

Why pilot?

The only welcome surprises when running a session are participant findings. You do not want to learn that they don’t understand your questions, or that you don’t have time to dig deep enough on the big juicy questions. And even the most experienced moderators benefit from a few pilot run-throughs to get familiar with the content — enabling a more natural conversation with your research participants.

What to pilot: Every element you possibly can!

I don't just pilot live interviews. I pilot unmoderated sessions, surveys, card sorts, click tests — and every other data gathering method in my toolkit. I also road test technology, stimuli, note-taking methods and strategies, and everything else I can possibly simulate ahead of time.

If it’s an interview, literally interview someone. You’ll quickly see whether your questions are provoking responses that tie into your research goals, fit within the timeframe allotted, are understandable, in the right sequence, are not redundant, and more.

If it’s a survey, share it with a few folks first. Then roll it out in waves. Solicit feedback.

For all studies, check your tech. Will you have a reliable internet connection? Does your recording equipment work? Can your participants access the prototype outside your firewall? Does everyone involved know how to use the tech, tools, templates, etc. they’ll need?

Who pilots with you?

Involve your team — fellow researchers, stakeholders and note-takers — in the piloting process if you can. Everyone needs to know their individual roles as well as how you plan to analyze and synthesize the data as a team. Pilot your note-taking strategies and tools in your pilot interviews. Share the chosen frameworks and strategies with your team and make sure they're educated and comfortable with it. And you’ll want to send everyone an end-of-day (and end-of-pilot) recap with highlights and updates.

You’ll also need pilot participants. Match them as closely as possible to your ideal participant qualification criteria for the best feedback. If they are not readily available, over-recruit and consider your first one or two sessions to be “pilots.”

When to pilot

In general, schedule your pilots at least two days before your actual sessions begin. You want to give yourself enough time to make revisions to your discussion guide and stimuli, especially if other people are involved. For example, if you're working with a prototype and something goes awry, you want time to go back to the designer and resolve the problem.

The piloting pay-off

A practice round might seem like something only freshman researchers need, but piloting is actually a pro move. It gives you and your team an opportunity to learn and iterate on what could be improved before investing time and money in a study that’s not as effective as it could be.

Ask Like A Pro students practice piloting during every phase of the research process. Happy to chat about how they go about this, the benefits they reap, and more.

Speak up, get involved, share the love

And that's a wrap!

We try to alternate between a theme and UX/UXR jobs, events, classes, articles, and other happenings every few weeks. Thank you for all of the feedback. Feedback is a gift and we continue to receive very actionable input on how to make Fuel Your Curiosity more meaningful to you.

What do you think? Lmk. We're constantly iterating and would still love to hear your input.

Stay curious,
- Michele and the Curiosity Tank team

PS. Have you checked out our free user research resources section lately? It's bursting with new support and inspiration. Looking for something we don't have listed? Send us a note to let us know!