COVID death data can be shared to make it look like vaccines don’t work, or worse – but that’s not the whole picture

Social media posts commenting on data from the UK, Israel and South Africa, among others, claim deaths from COVID (or all deaths) are now higher in vaccinated than unvaccinated citizens. Others make the more moderate claim vaccines do nothing to prevent death from COVID.

These reports appear intimidating, because they usually utilise real data or statistics. Many of the raw numbers presented are indeed correct, though not complete.

But people – including Clive Palmer who said this week vaccines don’t work and Craig Kelly who spread vaccine misinformation via text message – should ask whether they have understood the context, analysed the data properly and interpreted the results accurately.

What counts as ‘vaccinated’? When comparing studies or statistics, a lot hinges on how data providers define “vaccinated”.

Some vaccines are single-dose, others are double-dose regimens. Most jurisdictions define “fully vaccinated” as two weeks after the last required dose, but some social media posts like this one lump together anyone who has had any dose.

Some jurisdictions track patients in the interim period, others more crudely lump them with the partially vaccinated. We also have to compare patients who have received different vaccines, or mixed vaccine doses. Soon we will have to track those who have received booster shots.

To complicate things further, not all jurisdictions publish the necessary level of detail. Thankfully, we do have some good data at our fingertips: NSW is on track to become one of the most highly vaccinated jurisdictions in the world, and has published very detailed data with comparisons between vaccinated and unvaccinated cases that show vaccination is highly effective.

Not all numbers are equal Focusing on an absolute number – how far it is from zero, regardless of its relative value in the real world – can be very misleading.

Let’s say 95% of NSW’s entire population of approximately 8,000,000 was fully vaccinated. That’s 7.6 million people. Imagine 0.05% of fully vaccinated people needed to be hospitalised with COVID. That’s just shy of 4,000 people.

Let’s also say 1% of the remaining 400,000 unvaccinated people were hospitalised with COVID. That’s 4,000 people. So the absolute number of hospitalised vaccinated people would be similar to the hospitalised unvaccinated people.

But, less than 0.05% of fully vaccinated people are in hospital, compared to 1% of unvaccinated people – that’s 20 times higher! The same scenario could be applied to absolute numbers of COVID deaths.

Another way to explain this relates to what’s known as Simpon’s paradox: when a very large proportion of the population is vaccinated, it is quite likely that a larger absolute number of cases will be vaccinated rather than unvaccinated. To compare effects, one must instead focus on the rates of cases and death from COVID.

COVID doesn’t affect everyone in the same way COVID hospitalisation and death rates are skewed sharply by sex, pre-existing health conditions like metabolic syndrome, and age. So, men are more likely to die than women and the same is true for older people and those with other health problems.

And almost all countries vaccinated the most at-risk populations first. Some, like the elderly, already displayed higher death rates. Given the most likely to die are the first vaccinated, then we may see more deaths in the group vaccinated first. The vaccine will have lessened the deaths, but at-risk people were always more likely to die.

To truly discern the effect of the vaccine, then, one has to compare rates across matched levels of risk, especially age. A very good analysis matches different levels of age risk on the Israeli data and shows that even though the absolute number of vaccinated hospitalised cases was more than the unvaccinated hospitalisations, the Pfizer vaccine still showed good effectiveness against severe disease.

For the 10–59 age group, the vaccinated population will on average be older than the unvaccinated population … As mortality rates are higher for older people, this will increase the mortality rates for the vaccinated population compared to the unvaccinated population.

Making sense of all the numbers We know there are “lies, damned lies, and statistics”. And yet, some spurious social media claims rely on real numbers.

So the question isn’t: should you believe the statistics? Rather: which statistics should you believe? The best advice is to step back and look at the bigger picture.

When this pandemic winds down, there could be a day when there is only one hospitalised COVID patient. If that patient were to be vaccinated, would we say that 100% of hospitalised patients are vaccinated, and therefore the vaccines don’t work?