Breaking The Brand is a volunteer organisation that doesn’t currently have the funds to do substantial evaluations. But that doesn’t mean that we haven’t tried to evaluate the response to our pilot campaign in Viet Nam. In early 2014, long before our pilot campaign was launched, we approached an international company with offices in Viet Nam to ask if we could interview senior managers in Hanoi and Ho Chi Minh City. To clarify:
- The managers were Vietnamese citizens
- The interviews would be voluntary and no personal information would be collected
- In volunteering for the interviews, this in no way indicated that the interviewee was a rhino horn user, simply that they were part of the demographic group that could afford genuine rhino horn
- Even though the company supported the interviews, the evaluation would not formally be linked to the company
- We wanted to interview at least 30 people
We know from our 15 years’ experience in executive coaching that the quality of the information you get from interviews with people that you have never met before and who may have good reason not to trust you, especially if they didn’t initiate the coaching themselves, depends entirely on how you approach the interview.
The key elements are mutuality, vulnerability and genuine interest in the other person. These are usually not in place when interviews are conducted by market research agencies or university researchers. They tend to approach interviews either from a position of (expert) superiority or maintain their distance to be an ‘objective observer’. Neither approach has a great chance of success in getting people to truthfully talk about behaviours that may be illegal, subject to social stigma or contrary to established norms.
What works is the opposite. As quickly as possible establish mutuality – find out the level of thinking and decision making the person operates from and talk in their language. This is not a skill we learned overnight, but it is a skill that can be taught and learned through practice. It goes hand-in-hand with showing vulnerability. Instead of our egos dominating the conversation we decide to leave our egos outside the door before we enter and greet the person. That ensures the conversation is about them, not us, that they are the most important person in the frame and that we can be playful, humble, curious and challenging all at the same time. We can play dumb and ask seemingly trivial questions to verify assumptions we are making about the person and how they see the world. We can jokingly throw suggestions into the mix and observe the response to judge if we have hit a nerve. We have the sensory acuity to see when the interviewee is becoming a bit nervous because they feel they have revealed too much and do what is necessary to neutralise their concern so they become comfortable again and feel safe to tell us more. And so on.
This was the process used as I interviewed the users of ‘genuine’ rhino horn in mid-2013. The interviews were conducted by phone. This level of anonymity, even if I knew the person’s name, meant a few were comfortable to talk about their personal use of rhino horn straight away. For most the conversation started by me asking questions about people in their demographic group that may use rhino horn, and then you peel back the layers:
- These are stories that you have heard from other people?
- This is what you have been told directly from the users of genuine rhino horn you know?
Once the conversation had warmed (which takes approximately 15 minutes)
- Would you mind if I ask – Have you ever used rhino horn in the past?
- Would you mind if I ask and please if you would rather not answer – Are you currently using rhino horn?
- Once they have admitted they are, then you can explore why, what, when, where, who with, how etc using the techniques described above to keep them open and talking.
This shows we tend to take a while before we ‘get to the point’. We never mechanically go through a series of predetermined questions. We rarely takes notes, as this can raise suspicion. We have learned to remember everything being said in the interview and write it down afterwards. Most importantly, we display unwavering attention to everything the person says and our questions and comments demonstrate genuine interest in them at all times. Even if it is the two hundredth person we are interviewing that time, we still maintain the same level of genuine interest. Over the years I have lost count of how many clients when they have contacted me at the end of the day have said “No matter how late I ring you in the day, you always make me feel like I am the first person you have heard from today, your energy is always fresh”. The interest is based on our desire to understand the person and the behaviour, to understand their model of the world.
In the end the international company didn’t want to go ahead; Breaking The Brand’s request was taken to the lawyers who manage the company’s risk and they said no. But we are not giving up and will try this company again and others that have a base in Vietnam. We would recommend that the large conservation organisations who have strong corporate links ask these supporters for access to employees, who fit the right demographic group, as part of an evaluation strategy for campaigns.
Now I know when I talk about interviewing 30 people the number seems small. But remember in a recent article WildAid Managing Director John Baker said “Of 400 people interviewed in a survey commissioned by WildAid in late 2014, no one admitted having purchased rhino horn before. Either they will not admit it or the number of people surveyed did not get the right individuals.” In another survey carried out by one of the large conservation organisations I was told by an employee that in reality of the several hundred people who were part of the survey it was estimated that between 3 and 8 where users of rhino horn.
So what do you do when it is not practical or possible to interview a large number of people and quantitative information is required for analysis? In the case of illicit or other behaviours people don’t like to disclose honestly, social scientists have developed a number of so-called de-jeopardising techniques that give better, more honest results than direct questioning or normal questionnaires. Breaking The Brand is using these de-jeopardising techniques to develop evaluations that will be used once we are able to roll out our campaigns on a larger scale.
The Randomised Response Technique (RRT) has been used extensively in the social sciences since the 1970s. In this technique a randomisation method (like throwing a pair of dice) is used to determine if the person should answer honestly or give a fixed answer (irrespective of whether they engage in the behaviour or not). Because the interviewer cannot see the result of the dice throw, they don’t know if the answer is pre-determined or an honest answer, this information can only be extracted statistically.
The RRT has been applied in the conservation industry, mainly in face-to-face interviews. For example, Solomon et al. (2007) reported that 39% of respondents admitted to hunting illegally inside Kibale National Park, Uganda, when asked via RRT, whilst only 1.7% of respondents admitted to this behaviour when asked directly. So this technique is immensely valuable in getting people to disclose illegal behaviour.
Since around 2005 a number of researchers have investigated if the RRT works equally well when used in online surveys. There is still disagreement on the results, but overall it would appear that people in online surveys:
- Don’t trust the randomisation technique (electronic dice throw vs. using physical dice); and
- People not engaging in the behaviour prefer to answer ‘No’, even if the technique compels them to answer ‘Yes’ because of the dice numbers generated
Therefore an alternative technique called the Crosswise Model was proposed by Chinese researchers in 2008 and has been studied empirically since 2012. In this model online survey participants are presented with two questions, one ‘inoffensive’ question with a known distribution of answers (‘What is the starting digit of your parents’ house number?’) and one ‘offensive’ question with an unknown distribution. The unknown distribution can be extracted from the complete data sample using statistical techniques. This method has been shown to be superior to RRT in online surveys, especially if the answers to the inoffensive question follow a so-called Benford Law (like the starting digit of house numbers).
Whilst this article illustrates that both sophisticated interview and quantitative survey techniques are available for collecting information on the behaviour of consumers of illegal wildlife products, we must stress it is not always about the measurement. Currently we place too much importance on measuring and too little importance on understanding. The reason the two don’t yield the same insights is that we will measure what can easily be measured, and not necessarily what is relevant. This is in no way unique to the conservation industry, the same applies in the business sector, government and in academia.
These are the views of writer: Dr. Lynn Johnson, Founder, Breaking the Brand