Skip to main content
Top

2024 | OriginalPaper | Chapter

Deep Reinforcement Learning for Automated of Asian Stocks Trading

Authors : Todsapon Panya, Manad Khamkong

Published in: Applications of Optimal Transport to Economics and Related Topics

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In a complex and changeable stock market, algorithmic stock trading has firmly established itself as a fundamental aspect of the present-day financial market, where most transactions are now fully automated. Additionally, Deep Reinforcement Learning (DRL) agents, renowned for their exceptional performance in intricate games such as chess and Go, are increasingly impacting the stock market. In this paper, we examine the potential of deep reinforcement learning to optimize the portfolio returns of 15 Asian stocks. We model stock trading as a Markov decision process problem because of its stochastic and interactive nature. Furthermore, we train a deep reinforcement learning agent using three actor-critic-based algorithms: proximal policy optimization (PPO), advantage actor-critic (A2C), and deep deterministic policy gradient (DDPG). We tested the algorithm on Asian stocks to see how well it performed pre-COVID, during COVID, and post-COVID. The trading agent’s performance using various reinforcement learning algorithms is assessed and compared to the traditional min-variance portfolio allocation strategy. The proposed three individual algorithms are above the minimum variance in risk-adjusted return as evaluated by the Sharpe ratio.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Ataiefard, F.: Adversarial robustness testing of deep reinforcement learning based automated trading software (2022) Ataiefard, F.: Adversarial robustness testing of deep reinforcement learning based automated trading software (2022)
2.
go back to reference Bekiros, S.D.: Fuzzy adaptive decision-making for boundedly rational traders in speculative stock markets. Eur. J. Oper. Res. 202(1), 285–293 (2010)CrossRef Bekiros, S.D.: Fuzzy adaptive decision-making for boundedly rational traders in speculative stock markets. Eur. J. Oper. Res. 202(1), 285–293 (2010)CrossRef
3.
go back to reference Bekiros, S.D.: Heterogeneous trading strategies with adaptive fuzzy actor-critic reinforcement learning: a behavioral approach. J. Econ. Dyn. Control 34(6), 1153–1170 (2010)CrossRef Bekiros, S.D.: Heterogeneous trading strategies with adaptive fuzzy actor-critic reinforcement learning: a behavioral approach. J. Econ. Dyn. Control 34(6), 1153–1170 (2010)CrossRef
4.
5.
go back to reference Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., Efros, A.A.: Large-scale study of curiosity-driven learning (2018). arXiv:1808.04355 Burda, Y., Edwards, H., Pathak, D., Storkey, A., Darrell, T., Efros, A.A.: Large-scale study of curiosity-driven learning (2018). arXiv:​1808.​04355
6.
go back to reference Chen, L., Gao, Q.: Application of deep reinforcement learning on automated stock trading. In: 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS), pp. 29–33. IEEE (2019) Chen, L., Gao, Q.: Application of deep reinforcement learning on automated stock trading. In: 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS), pp. 29–33. IEEE (2019)
7.
go back to reference Chen, Q., Liu, X.Y.: Quantifying ESG alpha using scholar big data: an automated machine learning approach. In: Proceedings of the First ACM International conference on AI in Finance, pp. 1–8 (2020) Chen, Q., Liu, X.Y.: Quantifying ESG alpha using scholar big data: an automated machine learning approach. In: Proceedings of the First ACM International conference on AI in Finance, pp. 1–8 (2020)
8.
go back to reference Chong, T.T.L., Ng, W.K., Liew, V.K.S.: Revisiting the performance of MACD and RSI oscillators. J. Risk Finan. Manag. 7(1), 1–12 (2014)CrossRef Chong, T.T.L., Ng, W.K., Liew, V.K.S.: Revisiting the performance of MACD and RSI oscillators. J. Risk Finan. Manag. 7(1), 1–12 (2014)CrossRef
9.
go back to reference Danish, M.: Optimization of stock trading strategy with reinforcement learning (Doctoral dissertation) (2023) Danish, M.: Optimization of stock trading strategy with reinforcement learning (Doctoral dissertation) (2023)
10.
go back to reference Dang, Q.V.: Reinforcement learning in stock trading. In: International Conference on Computer Science, Applied Mathematics and Applications, pp. 311–322. Springer International Publishing, Cham (2019) Dang, Q.V.: Reinforcement learning in stock trading. In: International Conference on Computer Science, Applied Mathematics and Applications, pp. 311–322. Springer International Publishing, Cham (2019)
11.
go back to reference Deng, Y., Bao, F., Kong, Y., Ren, Z., Dai, Q.: Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 653–664 (2016)CrossRef Deng, Y., Bao, F., Kong, Y., Ren, Z., Dai, Q.: Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 653–664 (2016)CrossRef
12.
go back to reference Faturohman, T., Nugraha, T.: Islamic stock portfolio optimization using deep reinforcement learning. J. Islam. Monetary Econ. Fin. 8(2), 181–200 (2022) Faturohman, T., Nugraha, T.: Islamic stock portfolio optimization using deep reinforcement learning. J. Islam. Monetary Econ. Fin. 8(2), 181–200 (2022)
13.
go back to reference Fischer, T.G.: Reinforcement learning in financial markets-a survey (No. 12/2018). FAU Discussion Papers in Economics (2018) Fischer, T.G.: Reinforcement learning in financial markets-a survey (No. 12/2018). FAU Discussion Papers in Economics (2018)
14.
go back to reference Ge, J., Qin, Y., Li, Y., Huang, Y., Hu, H.: Single stock trading with deep reinforcement learning: a comparative study. In: 2022 14th International Conference on Machine Learning and Computing (ICMLC), pp. 34–43 (2022) Ge, J., Qin, Y., Li, Y., Huang, Y., Hu, H.: Single stock trading with deep reinforcement learning: a comparative study. In: 2022 14th International Conference on Machine Learning and Computing (ICMLC), pp. 34–43 (2022)
15.
go back to reference Gurrib, I.: Performance of the Average Directional Index as a market timing tool for the most actively traded USD based currency pairs. Banks Bank Syst. 13(3), 58–70 (2018)CrossRef Gurrib, I.: Performance of the Average Directional Index as a market timing tool for the most actively traded USD based currency pairs. Banks Bank Syst. 13(3), 58–70 (2018)CrossRef
16.
go back to reference Ilmanen, A.: Expected Returns: An Investor’s Guide to Harvesting Market Rewards, vol. 535. Wiley (2011) Ilmanen, A.: Expected Returns: An Investor’s Guide to Harvesting Market Rewards, vol. 535. Wiley (2011)
17.
go back to reference Jeong, G., Kim, H.Y.: Improving financial trading decisions using deep Q-learning: predicting the number of shares, action strategies, and transfer learning. Expert Syst. Appl. 117, 125–138 (2019)CrossRef Jeong, G., Kim, H.Y.: Improving financial trading decisions using deep Q-learning: predicting the number of shares, action strategies, and transfer learning. Expert Syst. Appl. 117, 125–138 (2019)CrossRef
18.
go back to reference Jiang, Z., Liang, J.: Cryptocurrency portfolio management with deep reinforcement learning. In: 2017 Intelligent Systems Conference (IntelliSys), pp. 905–913. IEEE (2017) Jiang, Z., Liang, J.: Cryptocurrency portfolio management with deep reinforcement learning. In: 2017 Intelligent Systems Conference (IntelliSys), pp. 905–913. IEEE (2017)
19.
go back to reference Kim, Y., Ahn, W., Oh, K.J., Enke, D.: An intelligent hybrid trading system for discovering trading rules for the futures market using rough sets and genetic algorithms. Appl. Soft Comput. 55, 127–140 (2017)CrossRef Kim, Y., Ahn, W., Oh, K.J., Enke, D.: An intelligent hybrid trading system for discovering trading rules for the futures market using rough sets and genetic algorithms. Appl. Soft Comput. 55, 127–140 (2017)CrossRef
20.
go back to reference Li, J., Rao, R., Shi, J.: Learning to trade with deep actor critic methods. In: 2018 11th International Symposium on Computational Intelligence and Design (ISCID), vol. 2, pp. 66–71. IEEE (2018) Li, J., Rao, R., Shi, J.: Learning to trade with deep actor critic methods. In: 2018 11th International Symposium on Computational Intelligence and Design (ISCID), vol. 2, pp. 66–71. IEEE (2018)
21.
go back to reference Li, X., Li, Y., Yang, H., Yang, L., Liu, X.Y.: DP-LSTM: Differential privacy-inspired LSTM for stock prediction using financial news (2019). arXiv:1912.10806 Li, X., Li, Y., Yang, H., Yang, L., Liu, X.Y.: DP-LSTM: Differential privacy-inspired LSTM for stock prediction using financial news (2019). arXiv:​1912.​10806
22.
go back to reference Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N.M.O., Erez, T., Tassa, Y., ... Wierstra, D.P.: U.S. Patent No. 10,776,692. U.S. Patent and Trademark Office, Washington, DC (2020) Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N.M.O., Erez, T., Tassa, Y., ... Wierstra, D.P.: U.S. Patent No. 10,776,692. U.S. Patent and Trademark Office, Washington, DC (2020)
23.
go back to reference Liu, X.Y., Xia, Z., Rui, J., Gao, J., Yang, H., Zhu, M., Guo, J.: FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning. Adv. Neural Inf. Proc. Syst. 35, 1835–1849 (2022) Liu, X.Y., Xia, Z., Rui, J., Gao, J., Yang, H., Zhu, M., Guo, J.: FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning. Adv. Neural Inf. Proc. Syst. 35, 1835–1849 (2022)
24.
go back to reference Liu, X.Y., Xiong, Z., Zhong, S., Yang, H., Walid, A.: Practical deep reinforcement learning approach for stock trading (2018). arXiv:1811.07522 Liu, X.Y., Xiong, Z., Zhong, S., Yang, H., Walid, A.: Practical deep reinforcement learning approach for stock trading (2018). arXiv:​1811.​07522
25.
go back to reference Maitah, M., Prochazka, P., Cermak, M., Šrédl, K.: Commodity channel index: evaluation of trading rule of agricultural commodities. Int. J. Econ. Finan. Issues 6(1), 176–178 (2016) Maitah, M., Prochazka, P., Cermak, M., Šrédl, K.: Commodity channel index: evaluation of trading rule of agricultural commodities. Int. J. Econ. Finan. Issues 6(1), 176–178 (2016)
26.
go back to reference Malini, H.: Behaviour of stock returns during Covid-19 pandemic: evidence from six selected stock market in the world. Jurnal Ekonomi Indones. 9(3), 247–263 (2020)CrossRef Malini, H.: Behaviour of stock returns during Covid-19 pandemic: evidence from six selected stock market in the world. Jurnal Ekonomi Indones. 9(3), 247–263 (2020)CrossRef
27.
go back to reference Markowitz, H.M.: Portfolio Selection. Yale University Press (1968) Markowitz, H.M.: Portfolio Selection. Yale University Press (1968)
28.
go back to reference Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., ... Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937. PMLR (2016) Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., ... Kavukcuoglu, K.: Asynchronous methods for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1928–1937. PMLR (2016)
29.
go back to reference Moody, J., Saffell, M.: Learning to trade via direct reinforcement. IEEE Trans. Neural Netw. 12(4), 875–889 (2001)CrossRef Moody, J., Saffell, M.: Learning to trade via direct reinforcement. IEEE Trans. Neural Netw. 12(4), 875–889 (2001)CrossRef
31.
go back to reference Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable-baselines3: reliable reinforcement learning implementations. J. Mach. Learn. Res. 22(1), 12348–12355 (2021) Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable-baselines3: reliable reinforcement learning implementations. J. Mach. Learn. Res. 22(1), 12348–12355 (2021)
32.
go back to reference Schulman, J., Levine, S., Abbeel, P., Jordan, M., Moritz, P.: Trust region policy optimization. In: International Conference on Machine Learning, pp. 1889–1897. PMLR (2015) Schulman, J., Levine, S., Abbeel, P., Jordan, M., Moritz, P.: Trust region policy optimization. In: International Conference on Machine Learning, pp. 1889–1897. PMLR (2015)
33.
go back to reference Wang, L., Zhang, W., He, X., Zha, H.: Supervised reinforcement learning with recurrent neural network for dynamic treatment recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2447–2456 (2018) Wang, L., Zhang, W., He, X., Zha, H.: Supervised reinforcement learning with recurrent neural network for dynamic treatment recommendation. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2447–2456 (2018)
34.
go back to reference Wu, Y., Tian, Y.: Training agent for first-person shooter game with actor-critic curriculum learning. In: International Conference on Learning Representations (2016) Wu, Y., Tian, Y.: Training agent for first-person shooter game with actor-critic curriculum learning. In: International Conference on Learning Representations (2016)
35.
go back to reference Yamaka, W., Maneejuk, P.: Analyzing the causality and dependence between gold shocks and Asian emerging stock markets: a smooth transition copula approach. Mathematics 8(1), 120 (2020)CrossRef Yamaka, W., Maneejuk, P.: Analyzing the causality and dependence between gold shocks and Asian emerging stock markets: a smooth transition copula approach. Mathematics 8(1), 120 (2020)CrossRef
36.
go back to reference Yang, H., Liu, X.Y., Wu, Q.: A practical machine learning approach for dynamic stock recommendation. In: 2018 17th IEEE International Conference on Trust, Security and Privacy in Computing and Communications/12th IEEE International Conference on Big Data Science and Engineering (TrustCom/BigDataSE), pp. 1693–1697. IEEE (2018) Yang, H., Liu, X.Y., Wu, Q.: A practical machine learning approach for dynamic stock recommendation. In: 2018 17th IEEE International Conference on Trust, Security and Privacy in Computing and Communications/12th IEEE International Conference on Big Data Science and Engineering (TrustCom/BigDataSE), pp. 1693–1697. IEEE (2018)
37.
go back to reference Zhang, Y., Yang, X.: Online portfolio selection strategy based on combining experts’ advice. Comput. Econ. 50, 141–159 (2017)CrossRef Zhang, Y., Yang, X.: Online portfolio selection strategy based on combining experts’ advice. Comput. Econ. 50, 141–159 (2017)CrossRef
38.
go back to reference Zhang, Z., Zohren, S., Stephen, R.: Deep reinforcement learning for trading. J. Finan. Data Sci. (2020) Zhang, Z., Zohren, S., Stephen, R.: Deep reinforcement learning for trading. J. Finan. Data Sci. (2020)
Metadata
Title
Deep Reinforcement Learning for Automated of Asian Stocks Trading
Authors
Todsapon Panya
Manad Khamkong
Copyright Year
2024
DOI
https://doi.org/10.1007/978-3-031-67770-0_37