Given contemporary growing concerns about the topic in the market research industry, it’s no surprise that data quality is by far the most important factor when choosing a market research partner or supplier (GRIT Report 2020). Broadly considering the presentation of survey fraud and poor quality data, here are five key principles wise to apply to any quantitative market research study:
1. Fraud evolves over time
The quality assurance (QA) checks that worked well in the past to prevent fraudulent survey responses are no longer sufficient. As panels continue to improve their frontend tools to block fraud, fraudsters also become more sophisticated in their ability to get into our surveys. Evolving past the fraudsters, and developing the right tools and strategies to stop them, is a continuous and critical effort to improve data quality.
2. There is no silver bullet
Alas, there is no single, one-stop solution for detecting and preventing fraud. Instead, we approach the problem by thinking of it in terms of layers of protection that are implemented throughout the research process from design to analysis. Panel sourcing, fraud detection software, programming logic built into the survey, and backend checks work seamlessly to prevent different types of quality issues.
3. Automation alone is not going to improve data quality
Today, there is a fair amount of manual labor involved in identifying suspicious patterns in data. While technology easily spots outliers, human intelligence is still essential in designing robust questionnaires, deciding how to measure fraud, setting appropriate thresholds for suspicious respondents, and identifying systematic issues that are far more obvious to humans than they are to algorithms.
4. Human error is not fraud
People make mistakes, get frustrated, or misread a question, but none of those things invalidates an entire survey. In fact, being reminded that real humans are taking your survey can be reassuring. Systematic fraud, both organized and malicious, is harder to identify and has a much bigger impact on the data.
5. Everyone has a role to play
Everyone involved in the research process has a role to play, starting with the panel providers who ultimately decide who is allowed into our surveys. That said, researchers are best-positioned to assess data quality as they typically know what to expect from the results.
These five key, self-imposed principles keep us researchers on our toes and serve to focus our efforts on the most effective ways to prevent fraud and improve data quality. By acknowledging data quality challenges and striving to keep abreast of issues in the field and new tools and strategies in the industry, we can uphold the data quality standards we hold ourselves to.
