Skip to main content
Top

2018 | OriginalPaper | Chapter

Topic Detection and Document Similarity on Financial News

Authors : Saeede Sadat Asadi Kakhki, Can Kavaklioglu, Ayse Bener

Published in: Advances in Artificial Intelligence

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Traders often rely on financial news to come up with predictions for stock price changes. Dealing with vast amount of news data makes it essential to use an automated methodology to identify the relevant news items for a given criteria. In this study we use Latent Dirichlet Allocation (LDA) to model the correlation of news items with stock price time series data. LDA model is trained with news items from a time window in the past and then the trained model is used to measure the similarity between the current news items and the news items used for training. Calculated similarity measure can be used as a predictor for switching points in the future. We tested our methodology using a collection of about 1,700,000 financial news items published between 2015-01-01 and 2015-12-31, and compared the results with various standard classification techniques. Our results indicate that use of LDA instead of standard classification techniques makes it possible to achieve the same level of performance by using a much smaller feature space.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
With document similarity threshold = 0.99, 0.98, number of topics = 100, 100, doc-topic prior = 0.01, 0.001, topic-words prior = 0.01, 0.005, max-iter = 200, 200, learning-offset = 64, - and learning-decay = 0.5, - for online and offline learning respectively.
 
Literature
2.
go back to reference Mittermayer, M.A.: Forecasting intraday stock price trends with text mining techniques. In: Proceedings of the 37th Annual Hawaii International Conference on System Sciences, 10-pp. IEEE (2004) Mittermayer, M.A.: Forecasting intraday stock price trends with text mining techniques. In: Proceedings of the 37th Annual Hawaii International Conference on System Sciences, 10-pp. IEEE (2004)
3.
go back to reference Boudoukh, J., Feldman, R., Kogan, S., Richardson, M.: Which news moves stock prices? A textual analysis. National Bureau of Economic Research (2013) Boudoukh, J., Feldman, R., Kogan, S., Richardson, M.: Which news moves stock prices? A textual analysis. National Bureau of Economic Research (2013)
4.
go back to reference Xie, B., Passonneau, R.J., Wu, L., Creamer, G.G.: Semantic frames to predict stock price movement (2013) Xie, B., Passonneau, R.J., Wu, L., Creamer, G.G.: Semantic frames to predict stock price movement (2013)
5.
go back to reference Li, X., Xie, H., Chen, L., Wang, J., Deng, X.: News impact on stock price return via sentiment analysis. Knowl.-Based Syst. 69, 14–23 (2014)CrossRef Li, X., Xie, H., Chen, L., Wang, J., Deng, X.: News impact on stock price return via sentiment analysis. Knowl.-Based Syst. 69, 14–23 (2014)CrossRef
7.
go back to reference Nguyen, T.H., Shirai, K.: Topic modeling based sentiment analysis on social media for stock market prediction. In: ACL (1), pp. 1354–1364 (2015) Nguyen, T.H., Shirai, K.: Topic modeling based sentiment analysis on social media for stock market prediction. In: ACL (1), pp. 1354–1364 (2015)
8.
go back to reference Papadimitriou, C.H., Tamaki, H., Raghavan, P.: Latent semantic indexing: a probabilistic analysis. J. Comput. Syst. Sci. 61, 217–235 (2000)MathSciNetCrossRefMATH Papadimitriou, C.H., Tamaki, H., Raghavan, P.: Latent semantic indexing: a probabilistic analysis. J. Comput. Syst. Sci. 61, 217–235 (2000)MathSciNetCrossRefMATH
9.
go back to reference Hofmann, T.: Probabilistic latent semantic indexing. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 289–296 (1999) Hofmann, T.: Probabilistic latent semantic indexing. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 289–296 (1999)
10.
go back to reference Hoffman, M., Bach, F.R., Blei, D.M.: Online learning for latent Dirichlet allocation. In: Advances in Neural Information Processing Systems, pp. 856–864 (2010) Hoffman, M., Bach, F.R., Blei, D.M.: Online learning for latent Dirichlet allocation. In: Advances in Neural Information Processing Systems, pp. 856–864 (2010)
11.
go back to reference Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)MATH Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)MATH
12.
go back to reference Canini, K., Shi, L., Griffiths, T.: Online inference of topics with latent Dirichlet allocation. In: Artificial Intelligence and Statistics, pp. 65–72 (2009) Canini, K., Shi, L., Griffiths, T.: Online inference of topics with latent Dirichlet allocation. In: Artificial Intelligence and Statistics, pp. 65–72 (2009)
13.
go back to reference Habert, B., Adda, G., Adda-Decker, M., de Marëuil, P.B., Ferrari, S., Ferret, O.: Towards tokenization evaluation. In: Proceedings of LREC, pp. 427–431 (1998) Habert, B., Adda, G., Adda-Decker, M., de Marëuil, P.B., Ferrari, S., Ferret, O.: Towards tokenization evaluation. In: Proceedings of LREC, pp. 427–431 (1998)
14.
go back to reference Sarkar, D.: Text analytics with Python (2016) Sarkar, D.: Text analytics with Python (2016)
15.
go back to reference Alpaydin, E.: Introduction to Machine Learning. MIT Press, Cambridge (2014)MATH Alpaydin, E.: Introduction to Machine Learning. MIT Press, Cambridge (2014)MATH
Metadata
Title
Topic Detection and Document Similarity on Financial News
Authors
Saeede Sadat Asadi Kakhki
Can Kavaklioglu
Ayse Bener
Copyright Year
2018
DOI
https://doi.org/10.1007/978-3-319-89656-4_34

Premium Partner