Abstract
Although anaesthesiologists strive to avoid hypoxaemia during surgery, reliably predicting future intraoperative hypoxaemia is not possible at present. Here, we report the development and testing of a machine-learning-based system that predicts the risk of hypoxaemia and provides explanations of the risk factors in real time during general anaesthesia. The system, which was trained on minute-by-minute data from the electronic medical records of over 50,000 surgeries, improved the performance of anaesthesiologists by providing interpretable hypoxaemia risks and contributing factors. The explanations for the predictions are broadly consistent with the literature and with prior knowledge from anaesthesiologists. Our results suggest that if anaesthesiologists currently anticipate 15% of hypoxaemia events, with the assistance of this system they could anticipate 30%, a large portion of which may benefit from early intervention because they are associated with modifiable factors. The system can help improve the clinical understanding of hypoxaemia risk during anaesthesia care by providing general insights into the exact changes in risk induced by certain characteristics of the patient or procedure.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$99.00 per year
only $8.25 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
Owing to patient-privacy considerations, the operating-room datasets from participating hospitals are not publicly available. The raw data from the anaesthesiologist comparisons in Fig. 3 are available in Supplementary Tables 6 and 7, and data from Fig. 5 are available in Supplementary Tables 8 and 9.
References
Weiser, T. G. et al. Estimate of the global volume of surgery in 2012: an assessment supporting improved health outcomes. Lancet 385, S11 (2015).
Gawande, A. A., Thomas, E. J., Zinner, M. J. & Brennan, T. A. The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery 126, 66–75 (1999).
Kable, A. K., Gibberd, R. W. & Spigelman, A. D. Adverse events in surgical patients in Australia. Int. J. Qual. Health Care 14, 269–276 (2002).
Nair, B. G., Gabel, E., Hofer, I., Schwid, H. A. & Cannesson, M. Intraoperative clinical decision support for anesthesia. Anesth. Analg. 124, 603–617 (2017).
Maier-Hein, L. et al. Surgical data science for next-generation interventions. Nat. Biomed. Eng. 1, 691–696 (2017).
Dunham, C. M., Hileman, B. M., Hutchinson, A. E., Chance, E. A. & Huang, G. S. Perioperative hypoxemia is common with horizontal positioning during general anesthesia and is associated with major adverse outcomes: a retrospective study of consecutive patients. BMC Anesthesiol. 14, 43 (2014).
Strachan, L. & Noble, D. W. Hypoxia and surgical patients--prevention and treatment of an unnecessary cause of morbidity and mortality. J. R. Coll. Surg. Edinb. 46, 297–302 (2001).
Ehrenfeld, J. M. et al. The incidence of hypoxemia during surgery: evidence from two institutions. Can. J. Anaesth. 57, 888–897 (2010).
Kooij, F. O., Klok, T., Hollmann, M. W. & Kal, J. E. Decision support increases guideline adherence for prescribing postoperative nausea and vomiting prophylaxis. Anesth. Analg. 106, 893–898 (2008).
Garg, A. X. et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 293, 1223–1238 (2005).
ElMoaqet, H., Tilbury, D. M. & Ramachandran, S. K. Multi-step ahead predictions for critical levels in physiological time series. IEEE Trans. Cybern. 46, 1704–1714 (2016).
Lipton, Z. C., Kale, D. C. & Wetzell, R. C. Phenotyping of clinical time series with LSTM recurrent neural networks. Preprint at http://arxiv.org/abs/1510.07641 (2015).
Henry, K. E., Hager, D. N., Pronovost, P. J. & Saria, S. A targeted real-time early warning score (TREWScore) for septic shock. Sci. Transl. Med. 7, 299ra122 (2015).
Saria, S., Rajani, A. K., Gould, J., Koller, D. & Penn, A. A. Integration of early physiological responses predicts later illness severity in preterm infants. Sci. Transl. Med. 2, 48ra65 (2010).
Caruana, R. et al. Intelligible models for healthcare: predicting pneumonia risk and hospital 30-day readmission. In Proc. 21st ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. 1721–1730 (ACM, 2015).
Deo, R. C. Machine learning in medicine. Circulation 132, 1920–1930 (2015).
Memarian, N., Kim, S., Dewar, S., Engel, J. & Staba, R. J. Multimodal data and machine learning for surgery outcome prediction in complicated cases of mesial temporal lobe epilepsy. Comput. Biol. Med. 64, 67–78 (2015).
Štrumbelj, E. & Kononenko, I. Explaining prediction models and individual predictions with feature contributions. Knowl. Inf. Syst. 41, 647–665 (2014).
Ribeiro, M. T., Singh, S. & Guestrin, C. “Why should I trust you?” Explaining the predictions of any classifier. In Proc. 22nd ACM SIGKDD Int. Conf. Know. Disc. Data Min. 1135–1144 (ACM, 2016).
Lundberg, S. & Lee, S.-I. A unified approach to interpreting model predictions. In Adv. Neural Information Processing 4765–4774 (Curran Associates, 2017).
Lundberg, S. M., Erion, G. G. & Lee, S.-I. Consistent individualized feature attribution for tree ensembles. Preprint at http://arxiv.org/abs/1802.03888 (2018).
Tarassenko, L., Hann, A. & Young, D. Integrated monitoring and analysis for early warning of patient deterioration. Br. J. Anaesth. 97, 64–68 (2006).
Summers, R. L., Pipke, M., Wegerich, S., Conkright, G. & Isom, K. C. Functionality of empirical model-based predictive analytics for the early detection of hemodynamic instabilty. Biomed. Sci. Instrum. 50, 219–224 (2014).
Current Procedural Terminology: CPT (American Medical Association, 2007).
Friedman, J. H. Greedy function approximation: a gradient boosting machine. Ann. Stat. 29, 1189–1232 (2001).
Chen, T. & Guestrin, C. XGBoost: a scalable tree boosting system. In Proc. 22nd ACM SIGKDD Int. Conf. Know. Disc. Data Min. 785–794 (ACM, 2016).
Lumachi, F. et al. Relationship between body mass index, age and hypoxemia in patients with extremely severe obesity undergoing bariatric surgery. In Vivo 24, 775–777 (2010).
Kendale, S. M. & Blitz, J. D. Increasing body mass index and the incidence of intraoperative hypoxemia. J. Clin. Anesth. 33, 97–104 (2016).
Defining Adult Overweight and Obesity (Centers for Disease Control and Prevention, 2016); https://www.cdc.gov/obesity/adult/defining.html
Myles, P. S., Leslie, K., McNeil, J., Forbes, A. & Chan, M. T. V. Bispectral index monitoring to prevent awareness during anaesthesia: the B-Aware randomised controlled trial. Lancet 363, 1757–1763 (2004).
Avidan, M. S. et al. Anesthesia awareness and the bispectral index. N. Engl. J. Med. 358, 1097–1108 (2008).
Epstein, R. H., Dexter, F. & Patel, N. Influencing anesthesia provider behavior using anesthesia information management system data for near real-time alerts and post hoc reports. Anesth. Analg. 121, 678–692 (2015).
Guay, J. & Ochroch, E. A. Intraoperative use of low volume ventilation to decrease postoperative mortality, mechanical ventilation, lengths of stay and lung injury in patients without acute lung injury. Cochrane Datab. Syst. Rev. J. 2018, CD011151 (2018).
Pulse Oximetry Training Manual (World Health Organization, 2011).
Dyagilev, K. & Saria, S. Learning (predictive) risk scores in the presence of censoring due to interventions. Mach. Learn. 102, 323–348 (2016).
Roth, A. E. (ed.) The Shapley Value: Essays in Honor of Llloyd S. Shapley (Cambridge Univ. Press, Cambridge, 1988).
Health, United States, 2016: With Chartbook on Long-term Trends in Health 314–317 (National Center for Health Statistics, Hyattsville, 2017).
Acknowledgements
We thank G. Erion, M. T. Ribeiro, J. Schreiber and members of the Lee laboratory for feedback and suggestions that improved the manuscript and experiments. This work was supported by National Science Foundation grant nos. DBI-135589 and DBI-1552309, National Institutes of Health grant no. 1R35GM128638, NSF Graduate Research Fellowship grant no. DGE-1256082 and a UW eScience/ITHS seed grant Machine Learning in Operating Rooms.
Author information
Authors and Affiliations
Contributions
S.-I.L., S.M.L., B.N. and J.K. initiated the study. S.-I.L. and S.M.L. developed the Prescience algorithms and designed data analyses and experiments. S.M.L. performed data analyses, experiments and data preprocessing. B.N. and S.-F.N. provided the electronic medical record data. J.K. recruited anaesthesiologists and helped design the anaesthesiologist test and survey. M.H., M.J.E., T.A., D.E.L. and D.K.-W.L. performed the web-based anaesthesiologist experiments and provided survey data. M.S.V. provided clinical assessment, interpretation of feature importances and connections with anaesthesiologists’ workflow. S.-I L. and S.M.L. wrote the paper in conjunction with B.N., J.K. and M.S.V. who wrote the sections on clinical interpretation and integration with current practices. M.H. provided manuscript feedback.
Corresponding author
Ethics declarations
Competing interests
B.N. is an advisor for Perimatics LLC and holds equity in the company. D.K.-W.L. is a Chief Medical Officer for MDmetrix, Inc. The other authors declare no competing interests.
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
Supplementary Figures 1–12, Supplementary Tables 1–3 and Supplementary References 1–2.
Supplementary Table 4
Initial features used by Prescience. An enumeration of all the 3,797 features used for preoperative predictions.
Supplementary Table 5
Intraoperative features used by Prescience. An enumeration of all the 3,905 features used for intraoperative predictions.
Supplementary Table 6
Data for Fig. 3a.
Supplementary Table 7
Data for Fig. 3b.
Supplementary Table 8
Data for Fig. 5a.
Supplementary Table 9
Data for Fig. 5b.
Rights and permissions
About this article
Cite this article
Lundberg, S.M., Nair, B., Vavilala, M.S. et al. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat Biomed Eng 2, 749–760 (2018). https://doi.org/10.1038/s41551-018-0304-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41551-018-0304-0
This article is cited by
-
Prediction models for postoperative recurrence of non-lactating mastitis based on machine learning
BMC Medical Informatics and Decision Making (2024)
-
Using time-series chest radiographs and laboratory data by machine learning for identifying pulmonary infection and colonization of Acinetobacter baumannii
Respiratory Research (2024)
-
CT-based radiomics: predicting early outcomes after percutaneous transluminal renal angioplasty in patients with severe atherosclerotic renal artery stenosis
Visual Computing for Industry, Biomedicine, and Art (2024)
-
Predicting sepsis in-hospital mortality with machine learning: a multi-center study using clinical and inflammatory biomarkers
European Journal of Medical Research (2024)
-
Comprehensive machine learning-based preoperative blood features predict the prognosis for ovarian cancer
BMC Cancer (2024)