Skip to main content
Top

2018 | OriginalPaper | Chapter

Group-Driven Reinforcement Learning for Personalized mHealth Intervention

Authors : Feiyun Zhu, Jun Guo, Zheng Xu, Peng Liao, Liu Yang, Junzhou Huang

Published in: Medical Image Computing and Computer Assisted Intervention – MICCAI 2018

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Due to the popularity of smartphones and wearable devices nowadays, mobile health (mHealth) technologies are promising to bring positive and wide impacts on people’s health. State-of-the-art decision-making methods for mHealth rely on some ideal assumptions. Those methods either assume that the users are completely homogenous or completely heterogeneous. However, in reality, a user might be similar with some, but not all, users. In this paper, we propose a novel group-driven reinforcement learning method for the mHealth. We aim to understand how to share information among similar users to better convert the limited user information into sharper learned RL policies. Specifically, we employ the K-means clustering method to group users based on their trajectory information similarity and learn a shared RL policy for each group. Extensive experiment results have shown that our method can achieve clear gains over the state-of-the-art RL methods for mHealth.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Dempsey, W., Liao, P., Klasnja, P., Nahum-Shani, I., Murphy, S.A.: Randomised trials for the fitbit generation. Significance 12(6), 20–23 (2016)CrossRef Dempsey, W., Liao, P., Klasnja, P., Nahum-Shani, I., Murphy, S.A.: Randomised trials for the fitbit generation. Significance 12(6), 20–23 (2016)CrossRef
2.
go back to reference Geist, M., Pietquin, O.: Algorithmic survey of parametric value function approximation. IEEE TNNLS 24(6), 845–867 (2013) Geist, M., Pietquin, O.: Algorithmic survey of parametric value function approximation. IEEE TNNLS 24(6), 845–867 (2013)
3.
go back to reference Grondman, I., Busoniu, L., Lopes, G.A.D., Babuska, R.: A survey of actor-critic reinforcement learning: standard and natural policy gradients. IEEE Trans. Syst. Man Cybern. 42(6), 1291–1307 (2012)CrossRef Grondman, I., Busoniu, L., Lopes, G.A.D., Babuska, R.: A survey of actor-critic reinforcement learning: standard and natural policy gradients. IEEE Trans. Syst. Man Cybern. 42(6), 1291–1307 (2012)CrossRef
4.
go back to reference Gustafson, D.: A smartphone application to support recovery from alcoholism: a randomized clinical trial. JAMA Psychiatry 71(5), 566–572 (2014)CrossRef Gustafson, D.: A smartphone application to support recovery from alcoholism: a randomized clinical trial. JAMA Psychiatry 71(5), 566–572 (2014)CrossRef
5.
go back to reference Kolter, J.Z., Ng, A.Y.: Regularization and feature selection in least-squares temporal difference learning. In: International Conference on Machine Learning, pp. 521–528 (2009) Kolter, J.Z., Ng, A.Y.: Regularization and feature selection in least-squares temporal difference learning. In: International Conference on Machine Learning, pp. 521–528 (2009)
6.
go back to reference Lagoudakis, M.G., Parr, R.: Least-squares policy iteration. J. Mach. Learn. Res. 4, 1107–1149 (2003)MathSciNetMATH Lagoudakis, M.G., Parr, R.: Least-squares policy iteration. J. Mach. Learn. Res. 4, 1107–1149 (2003)MathSciNetMATH
7.
go back to reference Lei, H., Tewari, A., Murphy, S.: An actor-critic contextual bandit algorithm for personalized interventions using mobile devices. In: NIPS 2014 Workshop: Personalization: Methods and Applications, pp. 1–9 (2014) Lei, H., Tewari, A., Murphy, S.: An actor-critic contextual bandit algorithm for personalized interventions using mobile devices. In: NIPS 2014 Workshop: Personalization: Methods and Applications, pp. 1–9 (2014)
8.
go back to reference Li, T., Levina, E., Zhu, J.: Prediction models for network-linked data. CoRR abs/1602.01192, February 2016 Li, T., Levina, E., Zhu, J.: Prediction models for network-linked data. CoRR abs/1602.01192, February 2016
9.
go back to reference Liao, P., Tewari, A., Murphy, S.: Constructing just-in-time adaptive interventions. Ph.D. Section Proposal, pp. 1–49 (2015) Liao, P., Tewari, A., Murphy, S.: Constructing just-in-time adaptive interventions. Ph.D. Section Proposal, pp. 1–49 (2015)
10.
go back to reference Murphy, S.A., Deng, Y., Laber, E.B., Maei, H.R., Sutton, R.S., Witkiewitz, K.: A batch, off-policy, actor-critic algorithm for optimizing the average reward. CoRR abs/1607.05047 (2016) Murphy, S.A., Deng, Y., Laber, E.B., Maei, H.R., Sutton, R.S., Witkiewitz, K.: A batch, off-policy, actor-critic algorithm for optimizing the average reward. CoRR abs/1607.05047 (2016)
11.
go back to reference Patrick, K., Raab, F., Adams, M., Dillon, L., Zabinski, M., Rock, C., Griswold, W., Norman, G.: A text message-based intervention for weight loss: randomized controlled trial. J. Med. Internet Res. 11(1), e1 (2009)CrossRef Patrick, K., Raab, F., Adams, M., Dillon, L., Zabinski, M., Rock, C., Griswold, W., Norman, G.: A text message-based intervention for weight loss: randomized controlled trial. J. Med. Internet Res. 11(1), e1 (2009)CrossRef
12.
go back to reference Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge (2012)MATH Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge (2012)MATH
14.
go back to reference Xu, Z., Wang, S., Zhu, F., Huang, J.: Seq2seq fingerprint: An unsupervised deep molecular embedding for drug discovery. In: ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (2017) Xu, Z., Wang, S., Zhu, F., Huang, J.: Seq2seq fingerprint: An unsupervised deep molecular embedding for drug discovery. In: ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (2017)
15.
go back to reference Zhu, F., Guo, J., Li, R., Huang, J.: Robust actor-critic contextual bandit for mobile health (mhealth) interventions. arXiv preprint arXiv:1802.09714 (2018) Zhu, F., Guo, J., Li, R., Huang, J.: Robust actor-critic contextual bandit for mobile health (mhealth) interventions. arXiv preprint arXiv:​1802.​09714 (2018)
16.
go back to reference Zhu, F., Liao, P.: Effective warm start for the online actor-critic reinforcement learning based mhealth intervention. In: The Multi-disciplinary Conference on Reinforcement Learning and Decision Making, pp. 6–10 (2017) Zhu, F., Liao, P.: Effective warm start for the online actor-critic reinforcement learning based mhealth intervention. In: The Multi-disciplinary Conference on Reinforcement Learning and Decision Making, pp. 6–10 (2017)
17.
go back to reference Zhu, F., Liao, P., Zhu, X., Yao, Y., Huang, J.: Cohesion-driven online actor-critic reinforcement learning for mhealth intervention. arXiv:1703.10039 (2017) Zhu, F., Liao, P., Zhu, X., Yao, Y., Huang, J.: Cohesion-driven online actor-critic reinforcement learning for mhealth intervention. arXiv:​1703.​10039 (2017)
Metadata
Title
Group-Driven Reinforcement Learning for Personalized mHealth Intervention
Authors
Feiyun Zhu
Jun Guo
Zheng Xu
Peng Liao
Liu Yang
Junzhou Huang
Copyright Year
2018
DOI
https://doi.org/10.1007/978-3-030-00928-1_67

Premium Partner