Inci M. Baytas
ABSTRACT
In the study of various diseases, heterogeneity among patients usually leads to different progression patterns and may require different types of therapeutic intervention. Therefore, it is important to study patient subtyping, which is grouping of patients into disease characterizing subtypes. Subtyping from complex patient data is challenging because of the information heterogeneity and temporal dynamics. Long-Short Term Memory (LSTM) has been successfully used in many domains for processing sequential data, and recently applied for analyzing longitudinal patient records. The LSTM units are designed to handle data with constant elapsed times between consecutive elements of a sequence. Given that time lapse between successive elements in patient records can vary from days to months, the design of traditional LSTM may lead to suboptimal performance. In this paper, we propose a novel LSTM unit called Time-Aware LSTM (T-LSTM) to handle irregular time intervals in longitudinal patient records. We learn a subspace decomposition of the cell memory which enables time decay to discount the memory content according to the elapsed time. We propose a patient subtyping model that leverages the proposed T-LSTM in an auto-encoder to learn a powerful single representation for sequential records of patients, which are then used to cluster patients into clinical subtypes. Experiments on synthetic and real world datasets show that the proposed T-LSTM architecture captures the underlying structures in the sequences with time irregularities.
Supplemental Material
- Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2014. Representation Learning: A Review and New Perspectives. arXiv:1206.5538v3[cs.LG] (2014). https://arxiv.org/abs/1206.5538Google Scholar
- Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning Long-Term Dependencies with Gradient Descent is Difficult. IEEE Transactions on Neural Networks Vol. 5, 2 (March 1994), 157--166. Google ScholarDigital Library
- Chao Che, Cao Xiao, Jian Liang, Bo Jin, Jiayu Zhou, and Fei Wang. 2017. An RNN Architecture with Dynamic Temporal Matching for Personalized Predictions of Parkinson's Disease. In Proceedings of the 2017 SIAM International Conference on Data Mining. Google ScholarCross Ref
- Zhengping Che, David Kale, Wenzhe Li, Mohammad Taha Bahadori, and Yan Liu 2015. Deep Computational Phenotyping. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 507--516. Google ScholarDigital Library
- Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, and Yan Liu 2016. Recurrent Neural Networks for Multivariate Time Series with Missing Values. arXiv preprint arXiv:1606.01865 (2016).Google Scholar
- Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, and Dzmitry Bahdanau et al. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv:1406.1078v3[cs.CL] (2014). https://arxiv.org/pdf/1406.1078v3Google Scholar
- Edward Choi, Mohammad Taha Bahadori, Andy Schuetzy, Walter F. Stewarty, and Jimeng Sun. 2016. Doctor AI: Predicting Clinical Events via Recurrent Neural Networks. arXiv:1511.05942v11 [cs.LG] (2016). https://arxiv.org/pdf/1511.05942v11.pdfGoogle Scholar
- Edward Choi, Mohammad Taha Bahadori, Andy Schuetzy, Walter F. Stewarty, and Jimeng Sun. 2016natexlabb. RETAIN: Interpretable Predictive Model in Healthcare using Reverse Time Attention Mechanism. arXiv:1608.05745v3 [cs.LG] (2016). https://arxiv.org/pdf/1608.05745v3.pdfGoogle Scholar
- Edward Choi, Mohammad Taha Bahadori, Elizabeth Searles, Catherine Coffey, Michael Thompson, James Bost, Javier Tejedor-Sojo, and Jimeng Sun. 2016. Multi-layer Representation Learning for Medical Concepts Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining KDD16. Association for Computing Machinery (ACM). http://dx.doi.org/10.1145/2939672.2939823 Google ScholarDigital Library
- Ivo D. Dinov, Ben Heavner, Ming Tang, Gustavo Glusman, Kyle Chard, and Mike Darcy et al. 2016. Predictive Big Data Analytics: A Study of Parkinson's Disease Using Large, Complex, Heterogeneous, Incongruent, Multi-Source and Incomplete Observations. PLoS ONE, Vol. 11, (8):e0157077 (August 2016). Google ScholarCross Ref
- Jeff Donahue, Lisa Anne Hendricks, Marcus Rohrbach, Subhashini Venugopalan, Sergio Guadarrama, Kate Saenko, and Trevor Darrell. 2016. Long-term Recurrent Convolutional Networks for Visual Recognition and Description. arXiv:1411.4389v4[cs.CV] (2016). https://arxiv.org/pdf/1411.4389.pdfGoogle Scholar
- Cristobal Esteban, Oliver Staeck, Yinchong Yang, and Volker Tresp 2016. Predicting Clinical Events by Combining Static and Dynamic Information Using Recurrent Neural Networks. arXiv:1602.02685v1 [cs.LG] (2016). https://arxiv.org/pdf/1602.02685v1.pdfGoogle Scholar
- Seyed-Mohammad Fereshtehnejad, Silvia Ríos-Romenets, Julius B. M. Anang, and Ronald B. Postuma. 2015. New Clinical Subtypes of Parkinson Disease and Their Longitudinal Progression: A Prospective Cohort Comparison With Other Phenotypes. JAMA Neurol, Vol. 72, 8 (2015), 863--873. Google ScholarCross Ref
- Alex Graves, Abdel rahman Mohamed, and Geoffrey Hinton. 2013. Speech Recognition with Deep Recurrent Neural Networks. arXiv:1303.5778[cs.NE] (2013). https://arxiv.org/abs/1303.5778Google Scholar
- Joyce C Ho, Joydeep Ghosh, and Jimeng Sun 2014. Marble: High-Throughput Phenotyping from Electronic Health Records via Sparse Nonnegative Tensor Factorization. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 115--124.Google ScholarDigital Library
- Sepp Hochreiter and Jürgen Schmidhuber 1997. Long Short-Term Memory. Neural computation, Vol. 9, 8 (1997), 1735--1780. Google ScholarDigital Library
- Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. 2016. MIMIC-III, A Freely Accessible Critical Care Database. Scientific Data Vol. 3 (2016).Google Scholar
- Uri Kartoun. 2016. A Methodology to Generate Virtual Patient Repositories. arXiv:1608.00570 [cs.CY] (2016). https://arxiv.org/ftp/arxiv/papers/1608/1608.00570.pdfGoogle Scholar
- Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent Convolutional Neural Networks for Text Classification Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence.Google Scholar
- Zachary C. Lipton, David C. Kale, Charles Elkan, and Randall" Wetzell 2016. Learning to Diagnose with LSTM Recurrent Neural Networks. arXiv:1511.03677v6 [cs.LG] (2016). https://arxiv.org/pdf/1511.03677v6.pdfGoogle Scholar
- Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schutze 2008. Introduction to Information Retrieval. Cambridge University Press. Google ScholarDigital Library
- Benjamin M. Marlin, David C. Kale, Robinder G. Khemani, and Randall C. Wetzel 2012. Unsupervised Pattern Discovery in Electronic Health Care Data using Probabilistic Clustering Models Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium. ACM, 389--398.Google Scholar
- Trang Pham, Truyen Tran, Dinh Phung, and Svetha Vankatesh. 2016. DeepCare: A Deep Dynamic Memory Model for Predictive Medicine. arxiv:1602.00357v1 [stat.ML] (February 2016). https://arxiv.org/pdf/1602.00357v1.pdfGoogle Scholar
- Nitish Srivastava, Elman Mansimov, and Ruslan Salakhutdinov. 2016. Unsupervised Learning of Video Representations using LSTM. arXiv:1502.04681v3[cs.LG] (2016). https://arxiv.org/abs/1502.04681Google Scholar
- Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. 2015. Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1711--1721.Google Scholar
- Ting Xiang, Debajyoti Ray, Terry Lohrenz, Peter Dayan, and P Read Montague 2012. Computational Phenotyping of Two-person Interactions Reveals Differential Neural Response to Depth-of-thought. PLoS Comput Biol, Vol. 8, 12 (2012), e1002841. Google ScholarCross Ref
- Yu Zhang, I-Wei Wu, Duygu Tosun, Eric Foster, and Norbert Schuff 2016. Progression of Regional Microstructural Degeneration in Parkinson's Disease: A Multicenter Diffusion Tensor Imaging Study. PLOS ONE (2016).Google Scholar
- Jiayu Zhou, Zhaosong Lu, Jimeng Sun, Lei Yuan, Fei Wang, and Jieping Ye. 2013. Feafiner: Biomarker Identification from Medical Data Through Feature Generalization and Selection. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1034--1042. Google ScholarDigital Library
- Jiayu Zhou, Fei Wang, Jianying Hu, and Jieping Ye. 2014. From Micro to Macro: Data Driven Phenotyping by Densification of Longitudinal Electronic Medical Records. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 135--144. Google ScholarDigital Library
- Jiayu Zhou, Lei Yuan, Jun Liu, and Jieping Ye. 2017. A Multi-Task Learning Formulation for Predicting Disease Progression Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 814--822.Google Scholar
- Xiaoqiang Zhou, Baotian Hu, Qingcai Chen, and Xiaolong Wang. 2015. An Auto-Encoder for Learning Conversation Representation Using LSTM Proceedings of the 22nd International Conference on Neural Information Processing, ICONIP 2015. 310--317. http://dx.doi.org/10.1007/978-3-319-26532-2_34 Google ScholarCross Ref
Index Terms
- Patient Subtyping via Time-Aware LSTM Networks
Recommendations
Software failure time series prediction with RBF, GRNN, and LSTM neural networks
AbstractThe important task of software quality assurance is failure prediction. Time series forecasting methods can be successfully used for this purpose. This paper aims to study and compare the effectiveness of software failure prediction using ...
Evaluating CNN and LSTM for Web Attack Detection
ICMLC '18: Proceedings of the 2018 10th International Conference on Machine Learning and ComputingWeb attack detection is the key task for network security. To tackle this hard problem, this paper explores the deep learning methods, and evaluates convolutional neural network, long-short term memory and their combination method. By comparing with the ...
Identifying Sepsis Subphenotypes via Time-Aware Multi-Modal Auto-Encoder
KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data MiningSepsis is a heterogeneous clinical syndrome that is the leading cause of mortality in hospital intensive care units (ICUs). Identification of sepsis subphenotypes may allow for more precise treatments and lead to more targeted clinical interventions. ...
Comments