Measuring whether and how brand marketing changes people’s perception of a brand often remains more art than science. Tools to track brand perception can easily cost millions across multiple markets but often come with serious challenges in data quality. One of the areas that can greatly contribute to ambiguity in the data authenticity and accuracy is the quality of the response. 

Up to 40% of survey responses can be fraudulent

Most online research panels give respondents incentives to complete surveys, typically gift cards or cash payments. Offering such rewards helps recruit respondents, ensures a high level of engagement, and motivates respondents to complete longer surveys. It also compensates for the time and effort people dedicate to sharing their feedback.

However, incentivising respondents to complete surveys comes with a significant challenge for data quality: it attracts fraudsters who prioritise collecting rewards over providing honest responses. Common fraudulent behaviours include:

• Straightlining and speeding: selecting the same answer option for all questions or rushing through questions at an unrealistically fast pace to finish quickly.

• Duplication: registering for several online research panels to complete as many surveys as possible can lead to duplication.

• Wrong qualification answers: purposefully providing wrong information in qualification questions to be able to complete more surveys.

• Survey bots: developing bots that automatically complete surveys, a practice that has been supercharged by AI.

Conducting incentivised surveys in our research revealed that bots and fraudulent human responses accounted for an average of 20% of completed surveys. However, industry estimates suggest this figure could reach as high as 40%, significantly undermining the reliability of insights derived from such data.

Designing surveys that people want to answer without a reward

To address the problem of fraud once and for all, we went back to first principles and asked ourselves what a survey would need to look like for people to be willing to answer it without a reward. Inspired by feedback mechanisms in e-commerce, ride-hailing, and food delivery platforms, we completely re-designed the survey experience and adopted a set of internal “rules” to guide our survey development:

1. The 5-second rule:

If it takes longer than five seconds to read, understand and answer a question, drop-off rates increase significantly. To address this, we follow a "5-second rule," ensuring every question can be understood and answered within 5 seconds.

2. No scrolling:

Questions with long lists of answers often confuse users and lead to higher drop-offs. To avoid this, we limit questions to two lines and cap answer options at five, eliminating the need for scrolling.

3. The “street interview” test:

We assess each question by imagining how likely it is that a random person on the street would answer it. If the likelihood is four or five out of five, we keep it. If it’s three, we revise it. If it’s one or two, we remove it.

4. The 10/90 rule:

The first 10 questions need to be able to provide 90% of the information we need. So we developed a dynamic question attribution model that ensures we can get the maximum information with the least possible number of questions.

By radically simplifying the survey structure and experience, we can recruit respondents online to complete surveys without offering any rewards. On average, people who have started one of our surveys answer 11 more questions without a reward, and as many as 20% answer 40 questions or more.

This experience has reshaped how we think about survey design, proving that meaningful engagement doesn’t require monetary incentives - just thoughtful, user-centric design.

Final Thoughts

Survey fraud is increasing at an alarming rate and is becoming more intelligent with the rise of AI. This problem is driven by the motivation for a financial reward for completing a survey, so by removing the incentive and focusing on highly engaging survey design, we can ensure that we have:

1 - Zero fraud

Removing the incentive from our surveys ensures that there is no motivation for bots or poorly behaved respondents to contribute to our answer data. 

2 - Honest feedback

As there is no incentive to persuade disingenuous response behaviour, every response we collect reflects genuine opinions, ensuring data integrity and reliability.