Charities have a new big brother. A new set of what is right and what is wrong about how they should think and behave. A new set of permitted words and phrases that are allowed and those that aren't. A new set of jargon and acronyms that serve only to confuse and emasculate.
The source of all this new oppression? Its the attempt to make charities to measure and communicate their impact.
Don't get me wrong – charities should absolutely be able to measure their impact – its one of the most important challenges they face.
I have even described the non-profit organisations that can successfully measure and communicate their impact as being like our sector's equivalent to Google or Amazon: they will change the way the sector works.
But the way to do this is to let organisations try their own ideas. Let the free market of ideas and approaches blossom. This will let the best ideas flourish and the poor ones wither. Unfortunately some people just can't let charities do this.
For example, try measuring your impact in outputs and the thought police will be on you: "That's just measuring bums on seats, not impact". That's because the impact-thought-police want you to measure outcomes – the difference that a charity has made. The problem is that outputs are easy to measure whereas outcomes are hellishly difficult.
I'd rather a charity wore its outputs on its sleeve than did nothing. As Voltaire said, "perfect is the enemy of better". I'd like a charity to brag about its outputs: how many meals it has served to homeless people, or how many people have called its helpline, or how many people used its website. If we wait for them to brag about their outcomes we will all be grey and old.
Outputs and outcomes are just one example of how the ideas orthodoxy is suffocating the development of impact measurement in practice. Another is SROI or social return on investment. SROI is a fantastic idea – monetising impact so that the difference a charity is making can be calculated in a financial format. However calculating SROI is difficult. We've did an SROI calculation for three projects at nfpSynergy which took 50 pages of calculations and explanations - and even then, we got a three page email from the thought police telling us where our calculations were wrong. A whole set of SROI standards is now being set up.
It is small charities in particular who are disadvantaged by the orthodoxy of SROI and impact measurement. With a thousand pressures at any time, making impact measurement harder will only make it less likely to happen. It is only the largest charities who can afford the costs of hiring external consultants to carry out impact work on their behalf.
I would be just a discordant voice in a cup final crowd if charities were taking to impact like ducks to water. But they aren't. Pause a moment and tell me which charities are doing a great job. Which charities can you name, who you know deliver impact? It's a tough task.
Charities are not good at their impact measurement let alone their impact communication. And the more we raise the bar of acceptability, the more we create an orthodoxy on what is acceptable and what isn't, the more we will extinguish the green shoots of organisations who are trying and finding their own ways to measure their value to the world. In particular we will provide another way in which smaller charities on limited resources are at a disadvantage.
Good article. I was beginning
Good article. I was beginning to think that I was alone in having difficulty with the nostrums of the high priests of measurement. I have a letter published in Third Sector this week on the subject and have put my thoughts on our website - link below. The debate is too one sided; in fact there is no debate!
http://www.shipwreckedmariners.org.uk/Home/MediaCentre.aspx?year=2012&a…
I know it's heretical but it
I know it's heretical but it might just not be possible to measure impact. It may just be too time-consuming and too uncertain. It is easy to measure a business, it's all there on the account sheet but measuring social impact is always going to be not just complicated but complex and we need to accept this rather than wishing it were some other way. I suspect all that the concern for measuring impact does is create new ways for people to cheat and deceive. The key issue is perhaps that what the government is interested in is charities speaking a language they can understand. Whether that language has anything to do with reality is not really what they are concerned about, because they're not interested in complexity but just in straightforward and linear solutions which sell well in our 'kind of democracy'.
Totally agree with the
Totally agree with the thought behind this. We have to be able to get to grips with the simple stuff and build from there, instead of being put off and berated for not having the perfect methods immediately. It has to be said also that there is a key role for trust.
Hi!
Hi!
I have an interest to declare: I work with FSG, an international social enterprise that does consultancy and research on social impact. A big part of our work relates to strategic measurement and evaluation.
I agree with some of the basis of where you're coming from. Impact is indeed fiendishly difficult to measure, and trying to attribute impact to specific interventions -- when we all know that social change occurs because multiple things happened at the same time -- can be a bit of a fool's errand. SROI is also hugely problematic (imagine trying to express the profit and loss of a company in terms the change in life expectancy associated with the contribution to national GDP, which is essentially an equivalent calculation in reverse).
But just focusing on outputs can be equally unhelpful and misleading, as my colleague, Val Bockstette, recently articulated on FSG's blog. You can read her post here: http://www.fsg.org/KnowledgeExchange/Blogs/StrategicEvaluation/PostID/3…
Mike
If the first duty of a
If the first duty of a charity is value for money for its beneficiaries, then charities far from the point of crisis will need to be careful that their impact measurement is a nett gain in helpfulness and transparency. It's ironic that a charity where impact measurement is very clear, like the RNLI saving lives say, finds it easy to measure impact, whereas lighthouses preventing the shipwreck in the first place, would find it hard.
Great article. It's good that
Great article. It's good that someone finally says this! It would be better to accept the fact that there is no perfect way of proving impact rather than trying to bang our heads against a brick wall. On the other hand, reality exists and that is hugely shaped by tools invented by the New Economics Foundation. It seems to me that funders, commissioners and some charities are really grateful for these tools, because they appear to be able to square the circle and provide an approach of figuring out where money is allocated more efficiently. But I think charities and other VCS organisations have to be proactive and confident about educating both funders and commissioners about what is a realistic and feasible way for them to provide information about their impact and the social value they might be generating. That would be better than adopting costly approaches just to get brownie points.