When conducting usability research, especially with a team new to the practice, make sure they understand that they cannot push out updates to the aspects you are studying when testing in a live environment.

Additional context: I am in the middle of a study with goals to:

1. Ascertain how well a tutorial can quickly acclimate participants who are unfamiliar with the platform, or not

2. Understand the ease/difficulty of use for the X, Y, Z features and functionality

3. Gather feedback on the naming of these features and the nomenclature used within them

4. Identify areas of improvement to improve usability to inform future iterations, specifically in regards to X, Y, Z features and functionality

In the middle of a session yesterday, my participant watched the tutorial and provided helpful feedback. Then logs into the platform and I notice that the most important nav element discussed in the tutorial no longer appears in the product. I ask the participant if we can reschedule this portion of the study and regroup later in the day. Thankfully, he agrees.

NOTE: This nav element is required to start all of the tasks and features we are testing.

Immediately I send a *HELP* email to the stakeholders and learn this main nav element has been renamed and some of the elements within it have been renamed as well. These revisions were pushed live yesterday. These aren’t the same names used in the tutorial, and no one told me these changes were in process or took place. We met as a team the day before and the stakeholders are highly engaged.

While I think this was a disconnect, and the team didn’t realize the impacts on the study, the result is hard to swallow.

Interestingly, last week's Fuel Your Curiosity newsletter focused on 21 reasons why research fails and how to mitigate those situations. I'll be sharing that with my stakeholders when we meet today. We'll also be doing a retro on what worked well, and what didn't in regards to this study to improve our efficiency and effectiveness moving forward.

In my 10 years of conducting UX research, this situation is a first for me. I hope sharing this recent experience helps some of you and your stakeholders.