Background: Xpert MTB/RIF Ultra (Ultra) is a new test for tuberculosis undergoing global roll-out. We assessed the performance of Ultra compared with Xpert MTB/RIF (Xpert) in an HIV-endemic setting where previous tuberculosis is frequent and current test performance is suboptimal. Methods: In this two-cohort diagnostic accuracy study, we used sputum samples from patients in South Africa to evaluate the accuracy of Ultra and Xpert against a single culture reference standard. For the first cohort (cohort A), we recruited adults (aged ≥18 years) with symptoms of presumptive tuberculosis at Scottsdene clinic in Cape Town, South Africa. We collected three sputum samples from each patient in cohort A, two at the first visit of which one was tested using Xpert and the other was tested using culture, and one sample the next morning which was tested using Ultra. In a separate cohort of patients with presumptive tuberculosis and recent previous tuberculosis (≤2 years) who had submitted sputum samples to the National Health Laboratory Services (cohort B), decontaminated sediments were, after processing, randomly allocated (1:1) for testing with Ultra or Xpert. For both cohorts we calculated the sensitivity and specificity of Ultra and Xpert and evaluated the effects of different methods of interpreting Ultra trace results. Findings: Between Feb 6, 2016, and Feb 2, 2018, we recruited 302 people into cohort A, all of whom provided sputum samples and 239 were included in the head-to-head analyses of Ultra and Xpert. For cohort B, we collected sputum samples from eligible patients who had submitted samples between Dec 6, 2016, and Dec 21, 2017, to give a cohort of 831 samples, of which 352 were eligible for inclusion in analyses and randomly assigned to Ultra (n=173) or Xpert (n=179). In cohort A, Ultra gave more non-actionable results (not positive or negative) than did Xpert (28 [10%] 275 vs 14 [5%] 301; p=0·011). In the head-to-head analysis, in smear-negative patients, sensitivity of Ultra was 80% (95% CI 64–90) and of Xpert was 73% (57–85; p=0·45). Overall, specificity of Ultra was lower than that of Xpert (90% [84–94] vs 99% [95–100]; p=0·001). In cohort B, overall sensitivity was 92% (81–98) for Xpert versus 86% (73–95; p=0·36) for Ultra and overall specificity was 69% (60–77) for Ultra versus 84% (78–91; p=0·005) for Xpert. Ultra specificity estimates improved after reclassification of results with the lowest Ultra-positive semiquantitation category (trace) to negative (15% [8–22]). In cohort A, the positive predictive value (PPV) for Ultra was 78% (67–87) and for Xpert was 96% (87–99; p=0·004); in cohort B, the PPV for Ultra was 50% (43–57) and for Xpert was 70% (61–78; p=0·014). Ultra PPV estimates in previously treated patients were low: at 15% tuberculosis prevalence, half of Ultra-positive patients with presumptive tuberculosis would be culture negative, increasing to approximately 70% in patients with recent previous tuberculosis. In cohort B, 21 (28%) of 76 samples that were Ultra positive were rifampicin indeterminate (all trace) and, like cohort A, most were culture negative (19 [90%] of 21). Interpretation: In a setting with a high burden of previous tuberculosis, Ultra generated more non-actionable results and had diminished specificity compared with Xpert. In patients with recent previous tuberculosis, a quarter of Ultra-positive samples were indeterminate for rifampicin resistance and culture negative, suggesting that additional drug-resistance testing will probably be unsuccessful. Our data have implications for the handling of Ultra-positive results in patients with previous tuberculosis in high burden settings. Funding: South African Medical Research Council, the EDCTP2 program, and the Faculty of Medicine and Health Sciences, Stellenbosch University.
