Skip to main content
Top

2017 | OriginalPaper | Chapter

Sentiment Analysis on Twitter to Improve Time Series Contextual Anomaly Detection for Detecting Stock Market Manipulation

Authors : Koosha Golmohammadi, Osmar R. Zaiane

Published in: Big Data Analytics and Knowledge Discovery

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In this paper, We propose a formalized method to improve the performance of Contextual Anomaly Detection (CAD) for detecting stock market manipulation using Big Data techniques. The method aims to improve the CAD algorithm by capturing the expected behaviour of stocks through sentiment analysis of tweets about stocks. The extracted insights are aggregated per day for each stock and transformed to a time series. The time series is used to eliminate false positives from anomalies that are detected by CAD. We present a case study and explore developing sentiment analysis models to improve anomaly detection in the stock market. The experimental results confirm the proposed method is effective in improving CAD through removing irrelevant anomalies by correctly identifying 28% of false positives.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
2
Bill C-46 (Criminal Code, RSC 1985, c C-46, s 382, 1985).
 
3
Section 9(a)(2) of the Securities Exchange Act (SECURITIES EXCHANGE ACT OF 1934, 2012).
 
4
The firehose access on Streaming API provides access to all tweets. This is very expensive and available upon case-by-case requests from Twitter.
 
7
\({TP}/({TP+FP})\).
 
8
\(TP/(TP+FN)\).
 
Literature
1.
go back to reference Antenucci, D., Cafarella, M., Levenstein, M., Ré, C., Shapiro, M.D.: Using social media to measure labor market flows. Technical report, National Bureau of Economic Research (2014) Antenucci, D., Cafarella, M., Levenstein, M., Ré, C., Shapiro, M.D.: Using social media to measure labor market flows. Technical report, National Bureau of Economic Research (2014)
2.
go back to reference Asur, S., Huberman, B.A.: Predicting the future with social media. In: 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), vol. 1, pp. 492–499. IEEE (2010) Asur, S., Huberman, B.A.: Predicting the future with social media. In: 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), vol. 1, pp. 492–499. IEEE (2010)
3.
go back to reference Bartov, E., Faurel, L., Mohanram, P.S.: Can twitter help predict firm-level earnings and stock returns? Available at SSRN 2782236 (2016) Bartov, E., Faurel, L., Mohanram, P.S.: Can twitter help predict firm-level earnings and stock returns? Available at SSRN 2782236 (2016)
4.
go back to reference Bing, L.: Sentiment Analysis: A Fascinating Problem, pp. 7–143. Morgan and Claypool Publishers (2012) Bing, L.: Sentiment Analysis: A Fascinating Problem, pp. 7–143. Morgan and Claypool Publishers (2012)
5.
go back to reference Bollen, J., Mao, H., Pepe, A.: Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena. ICWSM 11, 450–453 (2011) Bollen, J., Mao, H., Pepe, A.: Modeling public mood and emotion: Twitter sentiment and socio-economic phenomena. ICWSM 11, 450–453 (2011)
6.
go back to reference Bollen, J., Mao, H., Zeng, X.: Twitter mood predicts the stock market. J. Comput. Sci. 2(1), 1–8 (2011)CrossRef Bollen, J., Mao, H., Zeng, X.: Twitter mood predicts the stock market. J. Comput. Sci. 2(1), 1–8 (2011)CrossRef
9.
go back to reference Dillon, M.: Introduction to Modern Information Retrieval: G. Salton and M. Mcgill (1983) Dillon, M.: Introduction to Modern Information Retrieval: G. Salton and M. Mcgill (1983)
10.
go back to reference Feldman, R., Rosenfeld, B., Bar-Haim, R., Fresko, M.: The stock sonar—sentiment analysis of stocks based on a hybrid approach. In: Twenty-Third IAAI Conference, pp. 1642–1647 (2011) Feldman, R., Rosenfeld, B., Bar-Haim, R., Fresko, M.: The stock sonar—sentiment analysis of stocks based on a hybrid approach. In: Twenty-Third IAAI Conference, pp. 1642–1647 (2011)
12.
go back to reference Golmohammadi, K., Zaiane, O.R.: Time series contextual anomaly detection for detecting market manipulation in stock market. In: The 2015 Data Science and Advanced Analytics (DSAA 2015), pp. 1–10. IEEE (2015) Golmohammadi, K., Zaiane, O.R.: Time series contextual anomaly detection for detecting market manipulation in stock market. In: The 2015 Data Science and Advanced Analytics (DSAA 2015), pp. 1–10. IEEE (2015)
13.
go back to reference Graham, M., Hale, S.A., Gaffney, D.: Where in the world are you? Geolocation and language identification in Twitter. Prof. Geogr. 66(4), 568–578 (2014)CrossRef Graham, M., Hale, S.A., Gaffney, D.: Where in the world are you? Geolocation and language identification in Twitter. Prof. Geogr. 66(4), 568–578 (2014)CrossRef
14.
go back to reference King, G.: Ensuring the data-rich future of the social sciences. Science 331(6018), 719–721 (2011)CrossRef King, G.: Ensuring the data-rich future of the social sciences. Science 331(6018), 719–721 (2011)CrossRef
15.
go back to reference Lin, J., Keogh, E., Fu, A., Herle, H.: Approximations to magic: finding unusual medical time series, pp. 329–334. IEEE (2005) Lin, J., Keogh, E., Fu, A., Herle, H.: Approximations to magic: finding unusual medical time series, pp. 329–334. IEEE (2005)
16.
go back to reference Liu, B.: Sentiment analysis and opinion mining. Synth. Lect. Hum. Lang. Tech. 5(1), 1–167 (2012)MathSciNet Liu, B.: Sentiment analysis and opinion mining. Synth. Lect. Hum. Lang. Tech. 5(1), 1–167 (2012)MathSciNet
17.
go back to reference Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142–150. Association for Computational Linguistics, Portland (2011). http://www.aclweb.org/anthology/pp. 11-1015 Maas, A.L., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142–150. Association for Computational Linguistics, Portland (2011). http://​www.​aclweb.​org/​anthology/​pp.​ 11-1015
18.
go back to reference Manning, C.D., Raghavan, P., Schutze, H.: Introduction to Information Retrieval, pp. 405–416. Cambridge University Press (2008). Chap. 20 Manning, C.D., Raghavan, P., Schutze, H.: Introduction to Information Retrieval, pp. 405–416. Cambridge University Press (2008). Chap. 20
19.
go back to reference Mao, Y., Wei, W., Wang, B., Liu, B.: Correlating S&P 500 stocks with Twitter data. In: Proceedings of the First ACM International Workshop on Hot Topics on Interdisciplinary Social Networks Research, pp. 69–72. ACM (2012) Mao, Y., Wei, W., Wang, B., Liu, B.: Correlating S&P 500 stocks with Twitter data. In: Proceedings of the First ACM International Workshop on Hot Topics on Interdisciplinary Social Networks Research, pp. 69–72. ACM (2012)
20.
go back to reference Morinaga, S., Yamanishi, K., Tateishi, K., Fukushima, T.: Mining product reputations on the web. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 341–349. ACM (2002) Morinaga, S., Yamanishi, K., Tateishi, K., Fukushima, T.: Mining product reputations on the web. In: Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 341–349. ACM (2002)
21.
go back to reference Ruiz, E.J., Hristidis, V., Castillo, C., Gionis, A., Jaimes, A.: Correlating financial time series with micro-blogging activity. In: Proceedings of the Fifth ACM International Conference on Web Search and Data Mining, pp. 513–522. ACM (2012) Ruiz, E.J., Hristidis, V., Castillo, C., Gionis, A., Jaimes, A.: Correlating financial time series with micro-blogging activity. In: Proceedings of the Fifth ACM International Conference on Web Search and Data Mining, pp. 513–522. ACM (2012)
22.
go back to reference Schumaker, R.P., Chen, H.: Textual analysis of stock market prediction using breaking financial news: the Azfin text system. ACM Trans. Inf. Syst. (TOIS) 27(2), 12 (2009)CrossRef Schumaker, R.P., Chen, H.: Textual analysis of stock market prediction using breaking financial news: the Azfin text system. ACM Trans. Inf. Syst. (TOIS) 27(2), 12 (2009)CrossRef
23.
go back to reference Song, Y., Cao, L., Wu, X., Wei, G., Ye, W., Ding, W.: Coupled behavior analysis for capturing coupling relationships in group-based market manipulations. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 976–984. ACM (2012) Song, Y., Cao, L., Wu, X., Wei, G., Ye, W., Ding, W.: Coupled behavior analysis for capturing coupling relationships in group-based market manipulations. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 976–984. ACM (2012)
24.
go back to reference Sriastava, A., et al.: Discovering system health anomalies using data mining techniques, pp. 1–7 (2005) Sriastava, A., et al.: Discovering system health anomalies using data mining techniques, pp. 1–7 (2005)
25.
go back to reference Wei, L., Keogh, E., Xi, X.: Saxually explicit images: finding unusual shapes. In: 2006 Proceedings of the Sixth International Conference on Data Mining, ICDM 2006, pp. 711–720. IEEE (2006) Wei, L., Keogh, E., Xi, X.: Saxually explicit images: finding unusual shapes. In: 2006 Proceedings of the Sixth International Conference on Data Mining, ICDM 2006, pp. 711–720. IEEE (2006)
Metadata
Title
Sentiment Analysis on Twitter to Improve Time Series Contextual Anomaly Detection for Detecting Stock Market Manipulation
Authors
Koosha Golmohammadi
Osmar R. Zaiane
Copyright Year
2017
DOI
https://doi.org/10.1007/978-3-319-64283-3_24

Premium Partner