Curiosity Tank

View Original

"Research Theater" Defined with a Checklist

michele ronsen

What is research theater? How is it identified? Why should I care? What can I do if I suspect it?

Research theater refers to situations where organizations claim to be conducting research but are actually just going through the motions without the necessary expertise, rigor, or fidelity. In sum, the appearance of research is prioritized over the substance.

Original image from Freepik

It’s important to be able to identify the signs when research lacks credibility and may be more for show than for genuine inquiry or improvement.

Image from Freepik

Here are some red flags to watch out for:

  1. Lack of Clear Objectives: Genuine research should have clear, well-defined objectives that outline what the study aims to achieve and why it’s important. If the objectives are vague or unclear, it could indicate that the research is being conducted merely to fulfill a requirement, or check a box, rather than to address a genuine need.

  2. Methodological Weaknesses: Research conducted without proper methodology, such as inadequate sampling, biased survey questions, the without screening criteria, or flawed data analysis, is unlikely to yield reliable results. If the methods used in the research are questionable or poorly executed, it may indicate that the research is not being taken seriously or conducted with rigor.

  3. Limited Transparency: Transparency is essential in research to ensure that the process is open, accountable, and replicable. If an organization is not transparent about its research methods, data collection procedures, or findings, it raises questions about the credibility and integrity of the research.

  4. Conflict of Interest: Research that is influenced by conflicts of interest, such as financial incentives or political agendas, is unlikely to be impartial or unbiased. It’s important to assess whether the organization conducting the research has any vested interests that could compromise the integrity of the findings. (I am not referring to participant incentives here.)

  5. Lack of Expertise or Training: Research requires expertise and training to be conducted effectively. If the individuals responsible for conducting the research lack the necessary skills, knowledge, or experience in research recruiting, methodology, data analysis, it raises concerns about the quality and validity of the research.

Image from Freepik

By being aware of these red flags and critically evaluating the research being conducted, stakeholders can avoid being misled by “research lip sync” efforts and ensure that they rely on credible, evidence-based information to inform decision-making and drive genuine organizational improvement.

Image from Freepik

Here’s a checklist:

This checklist of questions is intended to help ascertain whether an organization or individual is engaging in substantive research or research theater:

  1. Research Objectives:

    • Are the research objectives clearly defined and aligned with organizational goals?

    • Does the research aim to address genuine organizational needs or issues?

  2. Methodology and Design:

    • What research methods are being used (e.g., surveys, interviews, experiments)?

    • Have the research methods been chosen based on their appropriateness for addressing the research objectives?

    • Is there a detailed research plan outlining the study design, data collection procedures, and analysis techniques?

  3. Data Quality and Integrity:

    • Is there transparency around how data is collected, stored, and analyzed?

    • Are measures in place to ensure the integrity and accuracy of the data (e.g., data validation checks, quality control procedures)?

  4. Expertise and Training:

    • Do the individuals conducting the research have the necessary expertise, training, and experience in research methodology and data analysis?

    • Is there access to external expertise or consultation when needed?

  5. Bias and Conflict of Interest:

    • Are steps being taken to minimize bias in the research process (e.g., through random sampling, double-blind procedures)?

    • Are there any potential conflicts of interest that could influence the research findings, and if so, how are they being addressed?

  6. Transparency and Accountability:

    • Is there transparency around the research process, including methods, data, and findings?

    • Are stakeholders involved in the research process, and is there a mechanism for providing feedback and accountability?

  7. Usefulness and Actionability:

    • Are the research findings relevant and actionable for the organization?

    • How are the research findings being used to inform decision-making and drive organizational improvement or change?

  8. Continuous Improvement:

    • Is there a culture of continuous improvement in the organization, where research findings are used to learn and adapt over time?

    • Are there mechanisms in place for evaluating the effectiveness and impact of the research efforts?

Image from Freepik

By evaluating the organization/individual/research against these criteria, stakeholders can gain a better understanding of whether the research being conducted is substantive and credible or merely serving as a form of “research theater” without genuine value or impact.

What to do if you suspect research theater?

Our next Fuel Your Curiosity article will offer several suggestions. Stay tuned!

Speak up, get involved, and share the love!

And that’s a wrap!

We try to alternate between a theme and UX/UXR jobs, events, classes, articles, and other happenings every few weeks. Thank you for all of the feedback. Feedback is a gift, and we continue to receive very actionable input on how to make Fuel Your Curiosity more meaningful to you.

What do you think? We're constantly iterating and would love to hear your input.​

Stay curious,

- Michele and the Curiosity Tank team

PS: How do you respond to research theater? Reply here and share your thoughts with us!