Surveys come with some complicating factors, however, which are specifically perilous in the ethics and compliance context. As Dan Ariely, Francesca Gino, and Max Bazerman have written in recent books on the subject of behavioral ethics, the way that people respond to questions about ethics is strongly affected by a number of non-ethical factors. These include cognitive biases, the need of most people to be seen as a moral person, and external concerns such as job security and the desire to please their managers. In our work at SAI Global Advisory Services Group, we use surveys extensively in Program, Cultural, and Learning Assessments, as well as direct Attitudinal Surveys. Having performed many client surveys, we have experienced firsthand the challenges to developing accurate, relevant, unbiased surveys.
One example of a challenge that afflicts surveys more in the ethics and compliance space than most other fields is that of response bias. Response bias occurs when a respondent answers a question with what they think is the “right” answer, rather than with their actual beliefs. While this is a challenge in the construction of most surveys, in ethics surveys it is compounded by what Dan Ariely calls the Fudge Factor. One of Ariely’s discoveries has been that most people have a need to be seen as moral, ethical, decent people, but they will lie and cheat up to the point where their behavior begins to damage their own ethical self-assessments. This desire to be seen as decent is stronger than the desire just to get the right answer. Therefore, while response bias is already in evidence (and needs to be controlled for) in factual questions (e.g., “how many online training courses have you taken this year?”), it is even stronger for ethical questions (e.g., “If you were to witness racial discrimination in your office, would you report it?”). Far more people who wouldn’t actually report discrimination are going to answer, “yes” to that question than would inflate their online training experience.
Another factor that can bias surveys is employee fear of surveillance, reprisals, or retaliation if they don’t answer the way their employer would like them to. If the relationship between management and rank-and-file employees is one of intimidation, tension, or distrust, many respondents will have reservations about the anonymity of the surveys. In this case, the desire to respond with the “correct” answers isn’t motivated by a need to be seen as ethical but by a much more basic motive: the need to keep one’s job. There are few reasons stronger than that.
While some biases can be controlled for or corrected through statistical means, response bias can be very difficult to measure and/or predict. Additionally, it creates anomalous situations where a very positive aggregate response to some questions can be read as exactly the opposite of a simple reading of the question. Questions on retaliation are particularly sensitive to this sort of effect, wherein if a survey is suggesting that 99% of employees feel that retaliation isn’t a problem at their firm, it may be that the fear of retaliation is what has created the near-unanimity.
The takeaway for compliance officers should be that simply asking employees “do you have a positive view of the ethics program?” or “do you feel we have an ethical company?” is not likely to yield accurate, useful survey data to help identify issues with your firm or plot a roadmap to success. Survey construction is a learned skill in any domain, but the complications to ethics and compliance surveys are particularly challenging.
The good news is that there are ways to avoid, detect, and mitigate those challenges. Through the careful construction of questions (both the content of the questions and the scales used for responses), response bias can be minimized. Additionally, the use of non-quantitative techniques such as structured interviews and focus groups can help verify and triangulate survey findings. Therefore, while we believe that surveys are one of the most important tools in the fact-finding toolkit for compliance officers, we also urge our clients to take a sophisticated, nuanced approach to surveys that takes into account the peculiar difficulties that complicate ethics and compliance surveys.