Last week CAF released a report saying that the public has given an extra £800m during the first six months of this year, compared to last. Charities have seen a massive loss of fundraising income this year and many charities are threatened with closure or reduction in services due to the decline in income. CAF’s research flows against the tide of almost every other research carried out in Covid. This blog looks at the reasons why CAF’s research should be treated with an abundance of caution (something CAF notably fail to do):

1. No corroborating evidence with this research

Whenever research comes up with results that are counter-intuitive it's worth asking the question: what corroborating evidence is there? The simple answer is that there is none that I am aware for the CAF research. The evidence from all the Sector bodies, from Pro-bono Economics, and much other research (including our own) shows that the fundraised income has fallen. Our research shows the number of people giving (and volunteering) has fallen since Covid started (while for CAF its stable). It’s amazing that CAF don’t even appear to question the likelihood of their own research being an outlier or even rogue result. CAF habitually produce research which goes against other available evidence. For example, CAF had the total giving in 2013 as £10.8 billion and in £2018 as £10.1 billion. In contrast Charity Financials had the fundraised income of the top 5000 charities (based on their accounts) as £13.3 billion in 2013 and £16.9 billion in 2018. So, CAF say giving has gone down by 7%, 2013 to 2018 while the accounts of charities say it has gone up by over 25%.

2. Social desirability bias is a challenge in measuring giving

If you ask somebody if they have given to charity many people will say yes. If you ask them in the middle of a pandemic, if they give to charity, they may be even more likely to say yes. Researchers need to be really careful when measuring giving that they aren’t measuring what people think they should say, rather than what they actually do. The problem also exists in researching people’s sex lives and their alcohol intake. What people say and what they do can be very different in these areas of life as well. CAF make no reference to how they address social desirability bias, but in the extraordinary situation of a global pandemic, it is unlikely that people react the way they would in normal times. But we don’t know.

3. The confidence limits are pretty wide for this research.

Forgive me people for getting technical but the total amount that people give is the product of two variables: the percentage who give and the amount they give. This means that the confidence intervals for two variables are not added but squared. We don’t know enough about the CAF data to know what the confidence intervals would be. In our report ‘Give and Take’ we had a worked example which shows that for our own data on giving, the confidence intervals of 95% would have a range of £7.5 billion to £8.4 billion. This means that a variation of several hundred million pounds is well within the confidence intervals. CAF don’t say that the increase of £800 million is statistically significant, or significant, so I presume it isn’t in their view (while they do comment on the significance of the increase or decrease of several other changes).

4. Only two of CAF’s months actually show a substantial increase in total income

Digging into CAF’s figures only two months actually show an increase of any substance – April, and May. According to the CAF data these two months alone could account for all the additional funds. Indeed, March 2020 shows a substantial decrease of 15% compared to 2019. It’s worth saying that we find in our research considerable random variability across a year in how much people say they have given, and this is why we don’t use it as a mechanism for calculating total giving.

5. The percentage giving in the CAF data hasn’t changed at all

Any calculation of giving is made up of two factors, the number of people who give, and how much they give. For the CAF data, the average over the first six months of 2019 is 31% and the average for the first six months of 2020 is 31%. In other words, no change year on year. The implications of this are that from the CAF data any change in the total amount given by the public is about the same number of people giving, but those people giving more.

6. If there is an increase it’s a small number of donors giving a lot extra

The CAF data makes clear that it is the amount that people gave that has changed. The average donation in 2019 was £41.33 and £50.17 in 2020. In the two key months of April and May 2020, the average donation was £61 and £62 (compared to £37 and £44 in 2019). The median donation in 2020 was only slightly raised, £22 in April and £30 in May, compared to the CAF norm of £20 a month. In other words, if there is a raised amount given it is not an outpouring of public generosity overall, but a very small number people of giving a lot more.

7. Their analysis says median donation is more important – but the total donation figures are based on average donations

The narrative in the CAF report makes clear that they think that median is a more useful way of measuring giving levels than average donation. For those who don’t remember their school maths, average is the total value of all gifts divided by the number of gifts, while median is the middle value if all donations are put sequentially. A single donation of £1 million dramatically skews the average but not the median. CAF is probably right that the median is a more useful measure of giving levels than the average: the problem is that their calculations for the total amount given are based not on the median but on the average donation. Strange really to praise median values but then to use averages for the calculation.

8. Incredibly for most of the period of this research animals are the most popular cause given to!!

The CAF report makes a play of how giving to ‘hospitals and hospices’ are among the main recipients of the public largesse. This makes intuitive sense. Major Tom was doing his stuff and the pressures on the NHS were all too clear. What they fail to mention is that for all but two months of the period January to August 2020 ‘animal welfare’ was the most popular cause given to. Perhaps I missed those news articles about the terrible impact of Covid on the animal population. The only two months where animals weren’t top were, yes you’ve guessed it, April and May, where hospitals and hospices can top. In their rush to create a neat narrative CAF has conveniently ignored results which don't fit the picture they want to paint.

It would be easy for us to ignore CAF publishing this kind of research showing extraordinary, and counter-intuitive results. They singularly fail to add in any of the caveats or footnotes about how the research results should be viewed. So consider this blog the footnotes that CAF have chosen not to add to their own research. Equally important, we would hate the government to think they don’t need to help charities because the public are giving more: not least when the research is of uncertain quality and shows an increase only in two months of this year.

If you want to read more about the challenges of measuring giving please do read our whole report here: https://nfpsynergy.net/free-report/give-and-take-challenges-measuring-p…