Self-Reporting Limitations and How to Mitigate

…or, “I often pick my nose in public!”

Most market research relies on subjects to tell the truth and respond to questions as honestly as possible. We set the stage by making the survey voluntary, encouraging different opinions in a focus group or letting participants know that there are no wrong answers.

But some questions are harder than others to answer honestly. When a respondent knows that his answer is “wrong” (morally, legally, ethically or socially), he may tend to offer the “right” answer instead. After all, there are no repercussions to him for answering in a normative way. Yet, as great interrogators might say, “We have ways of making you talk!”Right-vs.-Wrong

By its very nature, market research centers on self-reporting. There is evidence that the more personal the format, i.e. an in-person interview, the more likely subjects are to self-report falsely. We have witnessed increased truthfulness in online surveys, which are far less personal.

During the research phase of a past campaign, we asked subjects, over the phone, how often they brush their teeth. Nearly everyone knows that ‘twice a day’ is the right answer, and as a result, an overwhelming majority offered this answer. Interestingly, the survey population has a very high incidence of cavities. Brushing-TeethIf they truly brushed twice a day, they likely wouldn’t experience such a great degree of tooth decay. When we conducted an online survey of the same audience, we noticed significantly fewer ‘twice a day’ answers. The impersonal nature of an online survey allowed respondents to answer more truthfully than they did on the phone.

In another case, we probed into rampant abuse of handicapped parking spots. Here again, subjects knew full well that they were doing the wrong thing. In focus groups, these participants denied ever having parked illegally in a disabled parking spot.images

To get people to open up, we decided to try something new: the moderator admitted to having done it on occasion herself. Once this person in a position of authority confessed to her own counter-normative behavior, the subjects readily recounted their own incidents and motivations.

The most credible market research findings will always be based upon observational studies, especially where stigmatized or socially unacceptable behavior is concerned. But when more in-depth insights are required, consider research tactics that allow subjects to report freely, honestly and guiltlessly.

Leave a Reply

Your email address will not be published. Required fields are marked *