How to manage a drug-resistant pandemic (1) - estimating the global burden

For some time we’ve been struggling to figure out how the WHO has been reporting that the proportion of MDR-TB hasn’t been rising (or at least that it has “changed little in recent years”). This isn’t just for the last couple of years, by the way, it’s been reported it this way for about a decade - in fact for almost half of the period during which TB has been an official Global Emergency. In 1993, when the emergency was first declared, MDR-TB was hardly being taken seriously at all – but it’s not always been that way because thirteen years later it was – as the WHO/StopTB’s own graphic from 2006 (shown below) clearly demonstrates:

WHO graphic from 2006 identifying the global burden of MDR-TB

The WHO’s most recent estimated number of new cases (for 2014) was 480,000 (i.e. 9,000 less), with the death toll estimated at 180,000 (i.e. 60,000 more, probably because the success rate of treating MDR-TB has been significantly downgraded in the past decade). Meanwhile the number of XDR-TB cases was estimated last year at about 46,560, and the death toll from XDR-TB wasn’t even guessed at (which is hardly surprising since the infrastructure required to diagnose (let alone treat) XDR-TB is almost invisible in TB endemic countries).

Let’s put our cards on the table. We’re sure that the real drug-resistant numbers are much higher today than the WHO estimates that are being published and here are just a few reasons why:

- Between 2006 and 2015 the number of countries reporting at least a single case of XDR-TB has more than doubled from 49 to 105.

In 2008 MDR-TB rates approached “25% in several Eastern European countries; now some are over 50%.

- The recorded success rates of treating MDR-TB has dropped 2006-15(from 67% to 50%).

- MDR-TB is believed to be just as infectious as ‘normal’ TB (but with 'normal' TB being successfully treated in 87% of cases, but MDR only successfully treated 50% of the time).

- The relative case detection rates of drug-resistant disease is still (at the very best) just 26% as opposed to 63% for all-TB.

- Meanwhile the latest outcome data for MDR-TB patients (for the 2011 global cohort) comprised just 25,319 patients (or less than 6% of the WHO's estimated new MDR cases for that year) suggesting that the data that’s available is extremely poor.

- And, as we’ll see below, the way that these estimates of new MDR cases are prepared for most countries where TB is endemic is flawed.

Everything tells us that the proportion of MDR-TB must have been increasing in the last decade, although bafflingly no-one seems keen to challenge the WHO’s suggestion that it isn’t. Well we’ve taken our own look and we think we’ve unearthed a couple of reasons why the WHO’s numbers keep spitting out the way they do. It’s a little complicated unfortunately so we ask you to bear with us while we do our best to explain and expose why this is so.

The background

The first global and country-specific estimates of the drug-resistant disease burden associated with TB were published by WHO in 2001. It was estimated then that there'd been 272,906 new cases of MDR-TB globally the previous year (in 2000). The number estimated for 2004 rose to 424,203, a rise of 55% in four years - although this was partly explained as being because this estimate included incident cases of MDR-TB “among previously treated TB cases as well as those diagnosed for the first time”. (It’s not clear exactly, however, why these cases weren’t included in the earlier numbers because clearly they should have been). As we can see in the graphic above, however, the number estimated for 2006 was 489,000 – so there was estimated to have been a further 15% rise in the next two years. So we think that there’s more than reasonable evidence within the WHO’s own earlier numbers that MDR numbers were rising pretty fast, at least by 7% each year. Since then, however, something appears to have changed for no logical reason because, during this last decade, these estimated numbers first dropped (the estimate issued for 2008 was 440,000) and since then they've barely risen in spite of all of those simultaneously-published poor detection rates, the consistently poor treatment roll-outs and the dropping success rates, all of which normally would contribute to a growing pandemic. And the WHO reckons that the state of the MDR pandemic is “essentially unchanged from previous years”.

So how the numbers are actually being estimated?

Ideally, each country’s burden of MDR-TB should be being estimated from national drug-resistant surveys (DRS). Unfortunately resources dictate otherwise (and in the next blog in this sequence we’ll be looking at what can happen when one of them does have data from a DRS).

The reality is that most TB endemic countries have no valid data on MDR-TB, and for them the WHO have two ways of estimating MDR. They then put the two methods together using each to validate the other. (This can all be found in detail on http://www.stoptb.org/wg/mdrtb/assets/documents/MDR_diseaseburden_backgrounddocument_20140414.pdf , a document which was published by the WHO in 2014). By combining these two estimates the WHO establishes its annual global estimate.

In the example that’s given in this document itself these two methods do roughly correlate with each other so at first glance it looks like a useful technique: one method came in with a global total of 450,000, the other 420,000. But should we accept that the estimates which this methodology produce are correct given the anomalous implication of what this implies (that this pandemic is stagnating when logic suggests it must be on the rise)? Sure, if this way of developing the estimate is indeed correct then the disease must indeed be stagnating and we have less to fear from it. But if it’s wrong then the problem may be far more serious than is reported.

So what exactly do these methods consist of?

Well, the first method approximates the total estimated number MDR-TB incident cases (new cases) by first adding together the number of notified MDR-TB cases from among the three distinct types of TB case notifications (‘new all forms’, ‘relapse all forms’, and ‘all retreatments that are not relapse forms’). Such case notifications are those which are diagnosed and notified by national TB programs, however, so they can't comprise any estimation of all new MDR cases. In order to approximate a more realistic grand total, therefore, this initial sum is then “inflated upwards by the estimated amount of incident cases not identified by the surveillance system, for all forms of TB”. We enter this as a direct quote from the document referred to above not just so that we leave no doubt about it, but also because the last five words (in bold) exposes the invalidity of the method itself. As anyone who knows anything about TB knows, the number of infectious TB cases “not identified by the surveillance system” vary dramatically between the different forms of TB. For ‘all-TB’, for instance, it’s most recently been pegged at 63%. For MDR-TB, however it’s reported as being about 26% (and actually we’d suggest it’s almost certainly a lot less than this). But the document is unembarrassed about this anomaly explicitly stating that: “The number of incident MDR-TB cases missed by the surveillance system is assumed to be the same as that for drug susceptible TB, and does not vary according to retreatment categories” with this assumption made in full knowledge of the vast discrepancy between the two estimated case detection rates.

To give an idea of what this means, let’s run some numbers. Let’s say we found a total of 100 new cases of MDR-TB and notified them to the WHO, using the method above we would then divide it by 63 (for the 63% global detection rate) and then multiply it by 100 to ‘inflate it upwards’ to the final figure. This makes for 159 estimated new cases. If we used the 26% case detection rate, however, the same calculation would be: 100 divided by 26, and then multiplied by 100. This makes a total of 384. We can see from this exercise that factoring the number of notified cases by “the estimated amount of incident cases not identified by the surveillance system for all forms of TB” thus makes a total that is only about 40% of what it might really be. The implication of this is immense – it could mean that a more accurate estimate of new cases of MDR-TB each year isn’t half a million – it’s actually nearer 1.2 million, and it would certainly mean that this pandemic is on the rise.

So much for method I... But we already know that the two methods apparently corroborate each other, so maybe we’re wrong about this. So what about the second method?

Method II estimates MDR-TB incidence as “the ratio of the number of MDR-TB deaths divided by the MDR-TB case fatality rate (expressed as a proportion)”. The problem, of course, is that there still aren’t any decent measurements of MDR-TB deaths because there aren’t any reliable numbers of MDR cases being reported in the first place. Because of this a more indirect method of assessing this has total to be used instead. The WHO’s alternative number-crunching is therefore done as follows: Method II total is estimated by using the “ overall deaths from TB, [multiplied by] the overall proportion of TB cases that have MDR-TB (approximated by the weighted average of the proportion of new and retreated cases that have MDR-TB, as calculated from DR surveys or continuous surveillance), and [in turn multiplied by] the relative risk of dying among people with MDR-TB compared with those without MDR-TB (estimated from a systematic review that included data from twenty-five studies)”.

This is all a bit of a mouthful, but we can see from the parts we’ve highlighted in bold text is that the first factor that this second estimate is fundamentally dependent upon is the “proportion of cases that have MDR”. And of course this proportion was a part of Method I which means that any factor of error could be the same! In other words, the flaw in the first method is passed on to become the flaw in the second. The second method could similarly spit our an estimate that is just 40% of reality.

But there is another flaw with this second method because this calculation also includes the incorporation of ‘ICD-10’. ICD-10 is the international classification of diseases which was adopted by the WHO in 1992 in relation to reporting mortality statistics. It’s used by statisticians to code the underlying cause of every death with specific concern for matters relating to public health. Basically this means that a death can only ever be attributed (as far as this coding is concerned) to one disease – and, because of the threat of the HIV pandemic at the time the code was finalised, it was determined back in the 90s that if an individual died HIV-positive then their death should be coded only to HIV/AIDS whatever might have been the actual the cause of their death. By using ICD-10 the real reductions that have recently been recorded in TB deaths have been inevitably exaggerated (suggesting, for example, that the targets for reducing TB deaths by 2015 were nearly met, when they were actually far adrift of it because they didn’t include HIV/TB deaths).

In similar fashion the document states that its calculations of MDR-TB mortalities use TB mortality numbers “defined as the number of deaths caused by TB in HIV-negative people, according to the latest revision of the International classification of diseases(ICD-10).” Because of this the resulting calculations don’t include any TB deaths in individuals who are co-infected with HIV which inevitably further skews the final figures, especially where there are high rates of co-infection (i.e. in the African region where currently rates of MDR-TB are, perhaps unsurprisingly, being estimated to be so unexpectedly low...).

Actually, when classified using ICD-10, 26% of all estimated TB deaths worldwide must fall into this category of ‘HIV deaths’ (and most will be African). This amounts to 400,000 annual deaths from TB which don’t get coded as TB deaths at all – and so definitely won’t get counted as TB deaths in this second method described above.

There's one further issue relating to ICD-10, unfortunately. Using ICD-10 a death is only coded as one if the casualty was bacteriologically confirmed as sputum positive before death. This makes a TB coded death harder to maker than a TB diagnosis, simply because about half of all notified cases are diagnosed despite being sputum negative because of symptoms - because the sputum test is known to be so unreliable. Once again, we can predict a skewing of the numbers.

So can we conclude that the WHO’s suggestion that it has “no evidence” that the proportion of the pandemic that is MDR is on the rise is fundamenetally questionable? We think we can because it's based on a system of reporting and estimating that is fundamentally flawed and which needs to be urgently revised if this pandemic is to be taken seriously.

The solution, of course, is to implement national drug-resistant surveys, though these are expensive and are, in any case, sometimes also far from as reliable as might be wished.

In the next blog we will explore exactly this. We’ll discuss a case study, reviewing the MDR-TB numbers that are both estimated and reported for South Africa – and we'll show show how terribly wrong things can go if they’re allowed to – even with the help of a national drug resistance survey.