FALSE POSITIVES

© MMXX V.1.0.0
by Morley Evans

READ ORIGINAL

Test, test, test said the WHO. And globally, that’s what everyone did: tests have detected more than 14 million cases of Sars-CoV-2 so far. The thinking goes: turn up, have your test, and if positive, you must have the disease. But that’s far from the truth. When virus levels in the population are very low, the chances of a test accurately detecting Covid-19 could be even less than 50 per cent – for reasons that are not widely understood.

There are two issues about tests to get your head around. The first is the sensitivity of the test: the proportion of people who test positive, out of the population who have the virus. The second measure, specificity, is about the proportion of people who test negative, out of the population who should have tested negative. Finding out the actual values of these two measures is tricky. The Office for National Statistics admits they do not ‘know the true sensitivity and specificity of the test because Covid-19 is a new virus’.

Estimates suggest that roughly 80 per cent of infected people will have a positive test (the sensitivity). Based on the latest data, specificity may be as high as 99.9 per cent for those who test negative. I think this is a bit high, but let’s run with it for now.

There are two issues about tests to get your head around: sensitivity and specificity

To unravel the confusion, let’s think about what happens when the virus level is low – which it is in Britain at the moment. The latest ONS estimate is that about 0.04 per cent, or one in 2,300 people, had the virus at any point between 6 and 12 July. But for ease of calculation, let’s imagine the real infection level is higher: that 1 in 1,000 of us have the virus. Or 0.1 per cent.

And let’s imagine again that, in this scenario, 10,000 random people go for a Covid-19 test. With the infection level at 0.1 per cent, just ten people will have Sars-CoV-2 and 9,990 will not. Of the ten who turn up with an infection, 80 per cent will test positive, meaning eight people will be correctly identified while two walk away with a false negative.

And of the 9,990 not infected, all but ten will be correctly diagnosed as negative: hence the success rate of 99.9 per cent (the specificity). But ten will be told they have Covid-19, when in fact they don’t. That leaves us with 18 positive tests: eight from people who genuinely had the virus and ten who did not. So only eight out of 18 (44 per cent) of the infections are real. That’s where the chance of accurately detecting the disease being less than 50 per cent comes from.

But the above is not so hypothetical, given that the latest figures show the virus at lower levels than the scenario considered above, with tens of thousands of tests carried out every day. The academic study accompanying the ONS Infection Survey from 26 April to 28 June makes this point: ‘Even in a purely hypothetical situation that the virus is not circulating, a test specificity of 99.9 per cent would be associated with an expected number of positive tests that is approximately equal to what we observed over the entire study period.’

Problems with test accuracy are likely to be more of an issue globally. The current US Centers for Disease Control test kits can generate up to 30 per cent false positives even in their best laboratories. Highly accurate tests can prove costly – more than £100 per test. So, we shouldn’t be surprised that in poorer countries, highly questionable cheaper alternative tests, which cost less than £3, have been distributed and used. A recent BMJ review reported that the specificity of PCR tests could be as low as 95 per cent, as PCR test performance can be much worse in low prevalence community settings. This would mean that, in our hypothetical of 10,000 tests, we’d have 500 false positives amongst the eight genuine positives. So the hundreds of false-positive Covid-19 results would dwarf the genuine results – meaning an apparent surge in infections that is not followed by a corresponding surge in hospital admissions or deaths.

At very low prevalence, the proportion of people with infection falls and the numbers of falsely misdiagnosed increases. If Covid-19 completely disappears, then of our 10,000, no one will be infected. If you have followed the reasoning so far, you will have worked out this means that ten people would still be wrongly diagnosed as positive and the official data would show a national Covid-19 prevalence of 0.1 per cent. This is why understanding the accuracy of tests in the population that they are applied to matters: going off current testing practices and results, Covid-19 might never be shown to disappear.

Professor Carl Heneghan is director of Oxford’s Centre for Evidence-Based Medicine.