90% of surveys contain one or more significant issues that will result in flawed results

90% of the surveys I see contain one or more significant issues that will result in flawed results. While survey platforms can be easy to use, this ease leads people to believe they will also be able to survey and or screen participants effectively and efficiently. Yet ease and success are not two ends of the same spectrum.

Survey platform ease does not address the more important human factors: asking the right questions, to the right people, in the right way. In surveys and screeners this includes not only how the questions are worded, but also the solicitation to participate, the question types, sequence of questions, response options, termination points, escape hatches and possibly branching and piping.

Authoring consistently sound surveys and screeners takes years of actual practice in a variety of situations. It is anything BUT easy. Proficient survey and screener authors know that the platform is far less important that these other, much more critical aspects.

This screener question is a great example.

• What’s are the fatal flaws here?

• How could these be addressed?

• What is the impact to the results?

• What are the downstream ramifications of acting on flawed results?

• Does it matter that the platform was easy to use, or not, in this instance?

• Is the platform or the author responsible for this issue?

Let’s discuss!