‘Most of the trouble in life comes from misunderstanding’ as Willy Loman said in Oscar Wilde’s masterpiece An Inspector Calls. Grating isn’t it? But also a reasonable facsimile of how science reporting often reads, at least to those of us self identifying as scientists. Pre-Covidit was so. With Covid the problem has become, well, an epidemic. Recent reporting about the efficacy of masks is a classic case in point.

You might remember thestudy in question: it certainly made headlines when it came out. ‘Mask-wearing cuts Covid incidence by 53 per cent, says global study’, stated the Guardian: ‘results from more than 30 studies from around the world were analysed in detail’. The NewScientist agreed: ‘Covid-19 news:Mask wearing cuts infections by 53 per cent’, but instead referenced a greater number of evidence bases: ‘using data from 72 studies’. A more cautious Timesreported the same results as ‘Masks more effective than distancing and handwashing’. It goes for ‘more than 30 studies from around the world’ which ‘were analysed in detail’. Bloomberg claimed ‘Masks Cut Covid Risk in Half’, now based on a far-lower number: eight studies.

Let’s leave aside the nitpickingdetails (infections and risk are not the same thing, for example) — surely these reports are usefully communicating the gist of the new evidence, are they not? I’m afraid not.

The BMJ paper at the centreof these reports is a meta-analysis. That is to say: a summary, an attempt to combine the evidence containedin several scientific studies to arrive at an objective assessment of the state of knowledge on a topic.

The analysisidentified 72 studies that might potentiallyhave provided evidence on the effectiveness of masks, social distancing and hand washing. Of those, just six (not eight, 30 or 72) were sufficiently relevant — and of sufficient quality — that they could provide any useful information on mask efficacy. And howreliable were the six? Four were assessed to have a moderate risk of bias, and two to have a serious or critical risk.

Let’s start with the moderatefour.

One(Lio et al. 2021) detected a reduced risk in subjects who always wore a mask when outdoors.The problem is that the subjects chose to do this, and there is plenty of decent evidence that outdoor transmission risk is tiny. What has most likely been detected is that being extremely cautious about Covid — to the point that you wear a mask even when outdoors — reduces your risk of catching it. The mask is as much an effect of the caution as the reduced risk.

Moving on we have Wanget al. (2020), who looked at transmission only within the same household. If at least one householdmember was always wearing a mask at home before anyone in the house got sick, then there was a reduced risk of transmission. So the ultra-cautious are at lower risk of something irrelevant to the discussion of public mask-wearing.

Xu et al. (2020) was basedon an online survey, but I could not relate the results in the paper to those given in Talic et al.

Fourthly we have Bundgaard(2020), the famously inconclusive ‘Danmask’ study of 6,024 Danish citizens and the only oneof the six based on the sort of randomised trial that usually counts as evidence for a medical intervention.

Finally, we come to thestudies deemed to be at risk of serious or critical bias:

Krishnamachariet al. (2021) comparethe timing of mask mandates in different areas of the US and contrast subsequent case rates. It is unclear how such a study provides a measure of relative transmission risk that can be combined with the other five.

Doung-ngernet al. (2020),on the other hand, did attempt to look at the actual risk of transmission: to the contacts of people who subsequently turned out to be infected. If the contact wore a mask throughout the interaction then they were at reduced risk of picking up the infection. Thismight seem compelling until you think about the sort of interactions during which you do not remove your mask, as compared to those in which you do.

You should form your ownjudgement of how much evidence all this can provide for the efficacy of public mask-wearing as a Covid mitigation strategy, but it is certainly not as clear-cut as generally reported. It’s also interesting that the fact-checkers, who were so quick to crackdown on reports suggesting low mask efficacy, especially the Danmask study, seem to have beenless worried this time around.

My view is that the misunderstandingand unbalanced reporting of science leads to a great deal of trouble. Masks are the least of it (it’s Anne of the Island, L.M. Montgomery, by the way).