nach oben

Progress in Artificial Intelligence

Erschienen in:

05.01.2021 | Regular Paper

A synchronous deep reinforcement learning model for automated multi-stock trading

verfasst von: Rasha AbdelKawy, Walid M. Abdelmoez, Amin Shoukry

Erschienen in: Progress in Artificial Intelligence | Ausgabe 1/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Automated trading is one of the research areas that has benefited from the recent success of deep reinforcement learning (DRL) in solving complex decision-making problems. Despite the large number of researches done, casting the stock trading problem in a DRL framework still remains an open research area due to many reasons, including dynamic extraction of financial data features instead of handcrafted features, applying a scalable DRL technique that can benefit from the huge historical trading data available within a reasonable time frame and adopting an efficient trading strategy. In this paper, a novel multi-stock trading model is presented, based on free-model synchronous multi-agent deep reinforcement learning, which is able to interact with the trading market and to capture the financial market dynamics. The model can be executed on a standard personal computer with multiple core CPU or a GPU in a convenient time frame. The superiority of the proposed model is verified on datasets of different characteristics from the American stock market with huge historical trading data.

Vorheriger Artikel To recurse or not to recurse: a low-dose CT study

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

https://goldenjumper.wordpress.com/tag/ta-lib/.

Hasbrouck., J.: 22 Modeling market microstructure time series, In: Handbook of Statistics, Vol. 14, pp. 647-692, ELSEVIER(1996). https://doi.org/10.1016/S0169-7161(96)14024-4

Pate, J., Shah, S., Thakkar, P.: Predicting stock and stock price index movement using trend deterministic data preparation and machine learning techniques. Exp. Syst. Appl. 42(1), 259–268 (2015). https://doi.org/10.1016/j.eswa.2014.07.040. ElsevierCrossRef

Cavalcantea, R.C., Brasileirob, R.C., Souza, V.L., Nobrega, J.P., Oliveirab, A.L.I.: Computational intelligence and financial markets: A survey and future directions. Exp. Syst. Appl. 55, 194–211 (2016). https://doi.org/10.1016/j.eswa.2016.02.006CrossRef

Gupta, P., Mehlawat, M.K., Mittal, G.: Asset portfolio optimization using support vector machines and real-coded genetic algorithm. J. Global Optim. 53(2), 297–315 (2012)MathSciNetCrossRef

Yang, B., Gong, Z.-J., Yang, W.: Stock market index prediction using deep neural network ensemble, In: 36th Chinese Control Conference (CCC), pp. 26-28, Dalian, China (2017). https://doi.org/10.23919/ChiCC.2017.8027964

Zhang, J., Shicheng, C., Yan, X., Qianmu, L., Tao, L.: A novel data-driven stock price trend prediction system. Exp. Syst. Appl. 97, 60–69 (2018). https://doi.org/10.1016/j.eswa.2017.12.026CrossRef

Chonga, E., Han, C., Parka, F.C.: Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies. Exp. Syst. Appl. 83, 187–205 (2017)CrossRef

Lee, J., Kang, J.: Effectively training neural networks for stock index prediction: Predicting the S&P 500 index without using its index data. PLoS ONE V15(4), e0230635 (2020). https://doi.org/10.1371/journal.pone.0230635CrossRef

Sezer, O., Ozbayoglu, M.: Algorithmic financial trading with deep convolutional neural networks: Time series to image conversion approach. Appl. Soft Comput. 70 (2018) https://doi.org/10.1016/j.asoc.2018.04.024

10.

Jiang, W.: Applications of deep learning in stock market prediction: recent progress, arXiv:2003.01859 (2020), Preprint submitted to Elsevier Journal

11.

Murat, A.M., Omer, M.U., Sezer, B.S.: Deep learning for financial applications : A survey. Appl. Soft Comput. 93, 106384 (2020). https://doi.org/10.1016/j.asoc.2020.106384CrossRef

12.

Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T., Hui, F., Sifre, L., Driessche, G., Graepel, T., Hassabis, D.: Mastering the game of go without human knowledge. Int. J. Sci. Nat. 550, 354–359 (2017)

13.

https://colah.github.io/posts/2015-08-Understanding-LSTMs/ Accessed (August 2020)

14.

Watkins, C.J., Dayan, P.: Q-learning, Machine Learning, vol. 8, pp. 279–292. Springer, Berlin (1992)MATH

15.

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)CrossRef

16.

Schmid, Huber J.: Deep learning in neural networks: An overview. Neural Netw. V 61, 85–117 (2015)CrossRef

17.

Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. V18(7), 1527–1554 (2006)MathSciNetCrossRef

18.

Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks. Adv. Neural Inf. Process. Syst. V(19), 153–160 (2006)

19.

Hasselt, H.V.: Double Q-learning. Adv. Neural Inf. Process. Syst. 23, 2613–2621 (2010)

20.

Wang, Z., Freitas, N., de., Lanctot, M.: Dueling network architectures for deep reinforcement learning, In the International Conference on Machine Learning (ICML), (2015). arXiv preprint arXiv:1511.06581

21.

Hessel, M., Modayil, J., van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., Silver, D.: Rainbow: Combining Improvements in Deep Reinforcement Learning, Thirty-Second AAAI Conference on Artificial Intelligence (2017). arXiv preprint arXiv:1710.02298

22.

Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation, Advances in Neural Information Processing Systems, Vol. 12, pp. 1057–1063. (NIPS 1999) MIT Press, Cambridge, MA (2000)

23.

Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., Wierstra, D.: Continuous control with deep reinforcement learning, In: International Conference Learning Representations (2016). arXiv preprint arXiv:1509.02971

24.

Schulman, J., Levine, S., Abbeel, P., Jordan, M.I., Moritz, P.: Trust Region Policy Optimization, In: 32nd International Conference on Machine Learning, Vol. 37, pp. 1889–1897, PMLR. http://proceedings.mlr.press/v37/schulman15.html(2015)

25.

Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms, Computing Research Repository (CoRR), 1707.06347 (2017). arXiv preprint arXiv:1707.06347

26.

Wang, Z., Bapst, V., Heess, N., Mnih, V., Munos, R., Kavukcuoglu, K., Freitas, N.: Sample efficient actor-critic with experience replay,ICLR (2016). arXiv preprint arXiv:1611.01224

27.

OpenAI, https://openai.com/, Accessed 1.7 April 2020

28.

OpenAI Baselines: ACKTR & A2C, https://openai.com/blog/baselines-acktr-a2c/, Accessed 17 April 2020

29.

Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmil-ler, M.: Playing atari with deep reinforcement learning, In NIPS Deep Learning Work-shop (2013)

30.

Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T.P., Harley, T., Silver, D., Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning, In: 33rd International Conference on Machine Learning, Vol. 48, pp. 1928-1937, PMLR (2016)

31.

Moody, J., Saffell, M.: Learning to trade via direct reinforcement. IEEE Trans. Neural Netw. 12(4), 875–889 (2001). https://doi.org/10.1109/72.935097CrossRef

32.

Deng, Y., Bao, F., Youyong, K., Zhiquan, R., Qionghai, D.: Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 653–664 (2017)CrossRef

33.

Almahdi, S., Yang, S.Y.: An adaptive portfolio trading system: A risk-return portfolio optimization using recurrent reinforcement learning with expected maximum drawdown. Exp. Syst. Appl. V(87), 267–279 (2017). https://doi.org/10.1016/j.eswa.2017.06.023CrossRef

34.

Jiang, Z., Xu, D., Liang, J.: A deep reinforcement learning framework for the financial portfolio management problem, arXiv:1706.10059 (2017)

35.

https://github.com/OLPS/OLPS, last accessed October 2020

36.

Li, B., Sahoo, D., S. CH. Hoi.: Olps: A toolbox for online portfolio selection., J. Mach. Learn. Res. (JMLR), (2015)

37.

Li, B., Hoi, S.C.H.: Online portfolio selection: A survey. ACM Comput. Surv. (CSUR) V46(3), 35 (2014)MATH

38.

https://github.com/ZhengyaoJiang/PGPortfolio , last accessed October 2020

39.

Jiang, Z., Liang, J.: Cryptocurrency portfolio management with deep reinforcement learning., Intelligent Systems Conference., SAI Conferences,2017. Preprint: arXiv:1612.01277

40.

Liang, Z., Chen, H., Zhu, J., Jiang, K., Li, Y.: Adversarial Deep Reinforcement Learning in Portfolio Management, arXiv:1808.09940 (2018)

41.

Hegde, S., Kumar, V., Singh, A.: Risk aware portfolio construction using deep deterministic policy gradients, IEEE Symposium Series on Computational Intelligence (2018)

42.

Wang, J., Zhang, Y., Tang, K., Wu, J., Xiong, Z.: AlphaStock: A Buying Winners-and-Selling-Losers Investment Strategy using Interpretable Deep Reinforcement Attention Networks, 25th ACM SIGKDD, pp.1900-1908 (2019)

43.

Li, Y., Zheng, W., Zheng, Z.: Deep Robust Reinforcement Learning for Practical Algorithmic Trading, IEEE Access, pp.108014–108022 (2019)

44.

Soleymani, F., Elodie, P.: Financial portfolio optimization with online deep reinforcement learning and restricted stacked autoencoder - DeepBreath”. Exp. Syst. Appl. 156, 113456 (2020)CrossRef

45.

Leem, J., Kim, H.Y.: Action specialized expert ensemble trading system with extended discrete action space using deep reinforcement learning. PLoS ONE 15(7), e0236178 (2020). https://doi.org/10.1371/journal.pone.0236178CrossRef

46.

Mosavi, A., Ghamisi, P., Faghan, Y., Duan, P.: Shamshirband. Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics. (2020). https://doi.org/10.20944/preprints202003.0309.v1

47.

Charpentier, A., Elie, R., Remlinger, C.: Reinforcement Learning in Economics and Finance (2020) arXiv:2003.10014

Titel: A synchronous deep reinforcement learning model for automated multi-stock trading
verfasst von: Rasha AbdelKawy
Walid M. Abdelmoez
Amin Shoukry
Publikationsdatum: 05.01.2021
Verlag: Springer Berlin Heidelberg
Erschienen in: Progress in Artificial Intelligence / Ausgabe 1/2021
Print ISSN: 2192-6352
Elektronische ISSN: 2192-6360
DOI: https://doi.org/10.1007/s13748-020-00225-z

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Weitere Artikel der Ausgabe 1/2021

Inference and reasoning in a Bayesian knowledge-intensive CBR system

Artificial intelligence-based antivirus in order to detect malware preventively

To recurse or not to recurse: a low-dose CT study

Sentiment analysis of comments of wooden furniture based on naive Bayesian model

Multi-objective re-tuning of nonlinear model for degrading greenhouse