nach oben

Erschienen in:

2024 | OriginalPaper | Buchkapitel

Neural Graph Revealers

verfasst von : Harsh Shrivastava, Urszula Chajewska

Erschienen in: Machine Learning for Multimodal Healthcare Data

Verlag: Springer Nature Switzerland

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Sparse graph recovery methods work well where the data follows their assumptions, however, they are not always designed for doing downstream probabilistic queries. This limits their adoption to only identifying connections among domain variables. On the other hand, Probabilistic Graphical Models (PGMs) learn an underlying base graph together with a distribution over the variables (nodes). PGM design choices are carefully made such that the inference and sampling algorithms are efficient. This results in certain restrictions and simplifying assumptions. In this work, we propose Neural Graph Revealers (NGRs) which attempt to efficiently merge the sparse graph recovery methods with PGMs into a single flow. The task is to recover a sparse graph showing connections between the features and learn a probability distribution over them at the same time. NGRs use a neural network as a multitask learning framework. We introduce graph-constrained path norm that NGRs leverage to learn a graphical model that captures complex non-linear functional dependencies between features in the form of an undirected sparse graph. NGRs can handle multimodal inputs like images, text, categorical data, embeddings etc. which are not straightforward to incorporate in the existing methods. We show experimental results on data from Gaussian graphical models and a multimodal infant mortality dataset by CDC (Software: https://github.com/harshs27/neural-graph-revealers).

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Death Prediction by Race in Colorectal Cancer Patients Using Machine Learning Approaches

Nächstes Kapitel Multi-modal Biomarker Extraction Framework for Therapy Monitoring of Social Anxiety and Depression Using Audio and Video

Aluru, M., Shrivastava, H., Chockalingam, S.P., Shivakumar, S., Aluru, S.: EnGRaiN: a supervised ensemble learning method for recovery of large-scale gene regulatory networks. Bioinformatics 38, 1312–1319 (2021)CrossRef

Banerjee, O., Ghaoui, L.E., d’Aspremont, A.: Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. Rese. 9, 485–516 (2008)MathSciNetMATH

Belilovsky, E., Kastner, K., Varoquaux, G., Blaschko, M.B.: Learning to discover sparse graphical models. In: International Conference on Machine Learning, pp. 440–448. PMLR (2017)

Bhattacharya, S., Rajan, V., Shrivastava, H.: ICU mortality prediction: a classification algorithm for imbalanced datasets. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)

Bhattacharya, S., Rajan, V., Shrivastava, H.: Methods and systems for predicting mortality of a patient, US Patent 10,463,312, 5 November 2019

Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., Elhadad, N.: Intelligible models for healthcare: predicting pneumonia risk and hospital 30-day readmission. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1721–1730. ACM (2015)

Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)CrossRefMATH

Chickering, D.M.: Learning Bayesian networks is NP-complete. In: Fisher, D., Lenz, HJ. (eds.) Learning from Data. Lecture Notes in Statistics, vol. 112, pp. 121–130. Springer, New York (1996). https://doi.org/10.1007/978-1-4612-2404-4_12

Fernández, A., Garcia, S., Herrera, F., Chawla, N.V.: SMOTE for learning from imbalanced data: progress and challenges, marking the 15-year anniversary. J. Artif. Intell. Res. 61, 863–905 (2018)MathSciNetCrossRefMATH

10.

Frankle, J., Carbin, M.: The lottery ticket hypothesis: finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018)

11.

Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)CrossRefMATH

12.

Gallicchio, C., Scardapane, S.: Deep randomized neural networks. In: Oneto, L., Navarin, N., Sperduti, A., Anguita, D. (eds.) Recent Trends in Learning From Data. SCI, vol. 896, pp. 43–68. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-43883-8_3CrossRef

13.

Gogate, V., Webb, W., Domingos, P.: Learning efficient Markov networks. In: Advances in Neural Information Processing Systems, vol. 23 (2010)

14.

Greenewald, K., Zhou, S., Hero, A., III.: Tensor graphical lasso (TeraLasso). J. R. Stat. Soc. Ser. B (Stat. Methodol.) 81(5), 901–931 (2019)MathSciNetCrossRefMATH

15.

Hallac, D., Park, Y., Boyd, S., Leskovec, J.: Network inference via the time-varying graphical lasso. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 205–213 (2017)

16.

Haury, A.C., Mordelet, F., Vera-Licona, P., Vert, J.P.: TIGRESS: trustful inference of gene regulation using stability selection. BMC Syst. Biol. 6(1), 145 (2012)CrossRef

17.

Heckerman, D., Chickering, D.M., Meek, C., Rounthwaite, R., Kadie, C.: Dependency networks for inference, collaborative filtering, and data visualization. J. Mach. Learn. Res. 1, 49–75 (2001). https://doi.org/10.1162/153244301753344614CrossRefMATH

18.

Heckerman, D., Geiger, D., Chickering, D.M.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20(3), 197–243 (1995)CrossRefMATH

19.

Hsieh, C.J., Sustik, M.A., Dhillon, I.S., Ravikumar, P., et al.: QUIC: quadratic approximation for sparse inverse covariance estimation. J. Mach. Learn. Res. 15(1), 2911–2947 (2014)MathSciNetMATH

20.

Imani, S., Shrivastava, H.: tGLAD: a sparse graph recovery based approach for multivariate time series segmentation. In: 8th Workshop on Advanced Analytics and Learning on Temporal Data (AALTD) at ECML-PKDD (2023). https://doi.org/10.48550/arXiv.2303.11647

21.

Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press (2009)

22.

Lee, S.I., Ganapathi, V., Koller, D.: Efficient structure learning of Markov networks using \(l_1\)-regularization. In: Advances in Neural Information Processing Systems, vol. 19 (2006)

23.

Lou, Y., Caruana, R., Gehrke, J., Hooker, G.: Accurate intelligible models with pairwise interactions. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 623–631. ACM (2013)

24.

Margolin, A.A., et al.: ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context. BMC Bioinf. 7, 1–15 (2006)CrossRef

25.

Moerman, T., et al.: GRNBoost2 and Arboreto: efficient and scalable inference of gene regulatory networks. Bioinformatics 35(12), 2159–2161 (2019)CrossRef

26.

Pearl, J., Mackenzie, D.: The Book of Why: The New Science of Cause and Effect. Basic Books (2018)

27.

Pu, X., Cao, T., Zhang, X., Dong, X., Chen, S.: Learning to learn graph topologies. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

28.

Rajbhandari, S., Shrivastava, H., He, Y.: AntMan: sparse low-rank compression to accelerate RNN inference. arXiv preprint arXiv:1910.01740 (2019)

29.

Ramanujan, V., Wortsman, M., Kembhavi, A., Farhadi, A., Rastegari, M.: What’s hidden in a randomly weighted neural network? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11893–11902 (2020)

30.

Rolfs, B., Rajaratnam, B., Guillot, D., Wong, I., Maleki, A.: Iterative thresholding algorithm for sparse inverse covariance estimation. In: Advances in Neural Information Processing Systems, vol. 25, pp. 1574–1582 (2012)

31.

Van de Sande, B., et al.: A scalable scenic workflow for single-cell gene regulatory network analysis. Nat. Protoc. 15(7), 2247–2276 (2020)CrossRef

32.

Scutari, M.: Learning Bayesian networks with the bnlearn R package. J. Stat. Softw. 35(3), 1–22 (2010)MathSciNetCrossRef

33.

Shrivastava, H.: On using inductive biases for designing deep learning architectures. Ph.D. thesis, Georgia Institute of Technology (2020)

34.

Shrivastava, H., Bart, E., Price, B., Dai, H., Dai, B., Aluru, S.: Cooperative neural networks (CoNN): exploiting prior independence structure for improved classification. arXiv preprint arXiv:1906.00291 (2019)

35.

Shrivastava, H., Chajewska, U.: Methods for recovering conditional independence graphs: a survey. arXiv preprint arXiv:2211.06829 (2022)

36.

Shrivastava, H., Chajewska, U.: Neural graphical models. In: Proceedings of the 17th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU), August 2023. https://doi.org/10.48550/arXiv.2210.00453

37.

Shrivastava, H., Chajewska, U., Abraham, R., Chen, X.: A deep learning approach to recover conditional independence graphs. In: NeurIPS 2022 Workshop: New Frontiers in Graph Learning (2022). https://openreview.net/forum?id=kEwzoI3Am4c

38.

Shrivastava, H., Chajewska, U., Abraham, R., Chen, X.: uGLAD: sparse graph recovery by optimizing deep unrolled networks. arXiv preprint arXiv:2205.11610 (2022)

39.

Shrivastava, H., et al.: GLAD: learning sparse graph recovery. arXiv preprint arXiv:1906.00271 (2019)

40.

Shrivastava, H., Garg, A., Cao, Y., Zhang, Y., Sainath, T.: Echo state speech recognition. In: 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2021, pp. 5669–5673. IEEE (2021)

41.

Shrivastava, H., Huddar, V., Bhattacharya, S., Rajan, V.: Classification with imbalance: a similarity-based method for predicting respiratory failure. In: 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 707–714. IEEE (2015)

42.

Shrivastava, H., Huddar, V., Bhattacharya, S., Rajan, V.: System and method for predicting health condition of a patient. US Patent 11,087,879, 10 August 2021

43.

Shrivastava, H., Zhang, X., Aluru, S., Song, L.: GRNUlar: gene regulatory network reconstruction using unrolled algorithm from single cell RNA-sequencing data. bioRxiv (2020)

44.

Shrivastava, H., Zhang, X., Song, L., Aluru, S.: GRNUlar: a deep learning framework for recovering single-cell gene regulatory networks. J. Comput. Biol. 29(1), 27–44 (2022)CrossRef

45.

Singh, M., Valtorta, M.: An algorithm for the construction of Bayesian network structures from data. In: Uncertainty in Artificial Intelligence, pp. 259–265. Elsevier (1993)

46.

Städler, N., Bühlmann, P.: Missing values: sparse inverse covariance estimation and an extension to sparse regression. Stat. Comput. 22(1), 219–235 (2012)

47.

United States Department of Health and Human Services (US DHHS), Centers of Disease Control and Prevention (CDC), National Center for Health Statistics (NCHS), Division of Vital Statistics (DVS): Birth Cohort Linked Birth - Infant Death Data Files, 2004–2015, compiled from data provided by the 57 vital statistics jurisdictions through the Vital Statistics Cooperative Program, on CDC WONDER On-line Database. https://www.cdc.gov/nchs/data_access/vitalstatsonline.htm

48.

Vân Anh Huynh-Thu, A.I., Wehenkel, L., Geurts, P.: Inferring regulatory networks from expression data using tree-based methods. PLoS ONE 5(9), e12776 (2010)

49.

Wang, Y., Jang, B., Hero, A.: The Sylvester Graphical Lasso (SyGlasso). In: International Conference on Artificial Intelligence and Statistics, pp. 1943–1953. PMLR (2020)

50.

Williams, D.R.: Beyond Lasso: a survey of nonconvex regularization in Gaussian graphical models (2020)

51.

Yu, J., Smith, V.A., Wang, P.P., Hartemink, A.J., Jarvis, E.D.: Using Bayesian network inference algorithms to recover molecular genetic regulatory networks. In: International Conference on Systems Biology, vol. 2002 (2002)

52.

Yu, Y., Chen, J., Gao, T., Yu, M.: DAG-GNN: DAG structure learning with graph neural networks. In: International Conference on Machine Learning, pp. 7154–7163. PMLR (2019)

53.

Zhang, M., Jiang, S., Cui, Z., Garnett, R., Chen, Y.: D-VAE: a variational autoencoder for directed acyclic graphs. In: Advances in Neural Information Processing Systems, vol. 32 (2019)

54.

Zheng, X., Aragam, B., Ravikumar, P.K., Xing, E.P.: DAGs with NO TEARS: Continuous optimization for structure learning. In: Advances in Neural Information Processing Systems, vol. 31, pp. 9472–9483 (2018)

55.

Zheng, X., Dan, C., Aragam, B., Ravikumar, P., Xing, E.: Learning sparse nonparametric DAGs. In: International Conference on Artificial Intelligence and Statistics, pp. 3414–3425. PMLR (2020)

Titel: Neural Graph Revealers
verfasst von: Harsh Shrivastava
Urszula Chajewska
Verlag: Springer Nature Switzerland
Buch: Machine Learning for Multimodal Healthcare Data
Print ISBN: 978-3-031-47678-5

Electronic ISBN: 978-3-031-47679-2

Copyright-Jahr: 2024
DOI: https://doi.org/10.1007/978-3-031-47679-2_2

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"