Why stakeholder surveys are different
In recent years, we have been very fortunate to be awarded some really interesting stakeholder survey projects. We love it when we win these, because they are always challenging and take a lot of collaborative effort to work through.
The more we have conducted, the clearer it has become that we need to design very different research for stakeholders than we do for research which is among discrete samples of consumers, employees or audiences.
The challenges peculiar to stakeholder surveys are that:
- Different stakeholders relate and interact differently with the organisation.
- Some stakeholders may be more important to the organisation than others.
- Some may want different outcomes. It is not unusual for one stakeholder type - customers say - to want lower prices, compared with another stakeholder type - such as shareholders - who hope for higher profitability.
- Some stakeholder segments are much easier to research than others, because we have better lists or there are more of them, or they are more accessible, for example.
The must do's
We have found it is essential to:
- Conduct upfront qualitative with each key stakeholder group. Always.
- Pilot the survey with members of each key stakeholder group. Always.
- Use appropriate methods for different stakeholders. Although this involves more effort, it pays off every time.
- Work hard to achieve the response rate we want for all segments. If we want the CEOs of the ASX100 in our sample, we obviously need to treat them according to their status, even though it may be a more intensive process.
- Select sample sizes sufficient for analysis 'by stakeholder segment', when practical.
- Have broad internal stakeholder consultation as part of the research design process.
Things can get political
Imagine a survey that was designed to have equal sample sizes for each stakeholder group. Imagine now that the results are in, and one numerically large stakeholder group (say retail shareholders) rates their engagement with the organisation more positively than does a smaller stakeholder group. Should we weight the data to reflect their proportion in the population, knowing that the effect of weighting will be to change the overall result?
Weighting is often very tricky, because there is not necessarily any scientific basis for working out what the relative weights should be, but it can look appealing. Arguably, this kind of decision should be made at the outset of the survey, not when the results are in.
Here is a different example. Let's say we unexpectedly achieved:
- a great response to our employee survey, because the lists were up to date, but
- a poor response to the customer survey because their contact details turned out to be out of date.
This kind of response pattern is very common in stakeholder surveys, because we can get better lists for some stakeholder types than others, and we can achieve higher response rates for some than others.
Again, should we just accept the sample sizes we achieved and just aggregate the results, or should we weight them to reflect their 'real' proportion (if we know what that is)? Does it even make sense to add stakeholder groups together?
What is right for this survey for this client
The answer to these questions is the answer that is right for this survey for this client. In our view, stakeholder surveys should be in the hands of research agencies (like us) who are prepared to take all of these issues into account and who are happy to work collaboratively with clients. They will make sure that your stakeholder assessment is meaningful, useful and credible.
Tags: Stakeholder Surveys