Skip to main content

2020 | OriginalPaper | Buchkapitel

Concept Drift for Big Data

verfasst von : Raihan Seraj, Mohiuddin Ahmed

Erschienen in: Combating Security Challenges in the Age of Big Data

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The term “concept drift” refers to a change in statistical distribution of the data. In machine learning and predictive analysis, a fundamental assumption exits which reasons that the data is a random variable which is being generated independently from an underlying stationary distribution. In this chapter we present discussions on concept drifts that are inherent in the context big data. We discuss different forms of concept drifts that are evident in streaming data and outline different techniques for handling them. Handling concept drift is important for big data where the data flow occurs continuously causing existing learned models to lose their predictive accuracy. This chapter will serve as a reference to academicians and industry practitioners who are interested in the niche area of handling concept drift for big data applications.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Zang W, Zhang P, Zhou C, Guo L (2015) Comparative study between incremental and ensemble learning on data streams: case study. J Big Data 1:5CrossRef Zang W, Zhang P, Zhou C, Guo L (2015) Comparative study between incremental and ensemble learning on data streams: case study. J Big Data 1:5CrossRef
2.
Zurück zum Zitat Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. Johns Hopkins University, Baltimore Cauwenberghs G, Poggio T (2001) Incremental and decremental support vector machine learning. Johns Hopkins University, Baltimore
3.
Zurück zum Zitat Ross DA et al (2008) Incremental learning for robust visual tracking. Int J Computer Vision 77(1–3):125–141CrossRef Ross DA et al (2008) Incremental learning for robust visual tracking. Int J Computer Vision 77(1–3):125–141CrossRef
4.
Zurück zum Zitat Losing V, Wersing BHH (2018) Incremental on-line learning: a review and comparison of state of the art algorithms. Neurocomputing 275:1261–1274CrossRef Losing V, Wersing BHH (2018) Incremental on-line learning: a review and comparison of state of the art algorithms. Neurocomputing 275:1261–1274CrossRef
5.
Zurück zum Zitat Oza NC (2001) Online ensemble learning. University of California, Berkeley Oza NC (2001) Online ensemble learning. University of California, Berkeley
6.
Zurück zum Zitat Liao J-W, Dai B-R (2014) An ensemble learning approach for concept drift. In: Information science and applications (ICISA), 2014 international conference on. IEEE Liao J-W, Dai B-R (2014) An ensemble learning approach for concept drift. In: Information science and applications (ICISA), 2014 international conference on. IEEE
7.
Zurück zum Zitat Gomes HM (2017) A survey on ensemble learning for data stream classification. ACM Computing Surveys (CSUR) 50(2):23CrossRef Gomes HM (2017) A survey on ensemble learning for data stream classification. ACM Computing Surveys (CSUR) 50(2):23CrossRef
8.
Zurück zum Zitat Yoo PD, Ho YS, Zhou BB, Zomaya AY (2008) SiteSeek: post-translational modification analysis using adaptive locality-effective kernel methods and new profiles. BMC Bioinformatics 9:272CrossRef Yoo PD, Ho YS, Zhou BB, Zomaya AY (2008) SiteSeek: post-translational modification analysis using adaptive locality-effective kernel methods and new profiles. BMC Bioinformatics 9:272CrossRef
9.
Zurück zum Zitat Lee W, Stolfo S, Mok K (2000) Adaptive intrusion detection: a data mining approach. Artif Intell Rev 14(6):533–567CrossRef Lee W, Stolfo S, Mok K (2000) Adaptive intrusion detection: a data mining approach. Artif Intell Rev 14(6):533–567CrossRef
10.
Zurück zum Zitat Hilas CS (2009) Designing an expert system for fraud detection in private telecommunications networks. Expert Syst Appl 36(9):11559–11569CrossRef Hilas CS (2009) Designing an expert system for fraud detection in private telecommunications networks. Expert Syst Appl 36(9):11559–11569CrossRef
11.
Zurück zum Zitat Mazhelis O, Puuronen S (2007) Comparing classifier combining techniques for mobile-masquerader detection. In: The second international conference on availability, reliability and security Mazhelis O, Puuronen S (2007) Comparing classifier combining techniques for mobile-masquerader detection. In: The second international conference on availability, reliability and security
12.
Zurück zum Zitat Aminikhanghahi S, Cook DJ (2017) A survey of methods for time series change point detection. Knowl Inf Syst 51(2):339–367CrossRef Aminikhanghahi S, Cook DJ (2017) A survey of methods for time series change point detection. Knowl Inf Syst 51(2):339–367CrossRef
13.
Zurück zum Zitat Kawahara Y (2009) Change-point detection in time-series data by direct density-ratio estimation. In: Proceedings of the 2009 SIAM international conference on data mining. Society for Industrial and Applied Mathematics Kawahara Y (2009) Change-point detection in time-series data by direct density-ratio estimation. In: Proceedings of the 2009 SIAM international conference on data mining. Society for Industrial and Applied Mathematics
14.
Zurück zum Zitat Ghourchian N, Allegue-Martinez M, Precup D (2017) Real-time indoor localization in smart homes using semi-supervised learning. In: AAAI Ghourchian N, Allegue-Martinez M, Precup D (2017) Real-time indoor localization in smart homes using semi-supervised learning. In: AAAI
15.
Zurück zum Zitat Cohn D, Atlas L, Ladner R (1994) Improving generalization with active learning. Mach Learn 15(2):201–221 Cohn D, Atlas L, Ladner R (1994) Improving generalization with active learning. Mach Learn 15(2):201–221
16.
Zurück zum Zitat Zliobaite I, Bifet A, Holmes G, Pfahringer B (2011) MOA concept drift active learning strategies for streaming data. In: Proceedings of the second workshop on applications of pattern analysis Zliobaite I, Bifet A, Holmes G, Pfahringer B (2011) MOA concept drift active learning strategies for streaming data. In: Proceedings of the second workshop on applications of pattern analysis
17.
Zurück zum Zitat Saurav S (2018) Online anomaly detection with concept drift adaptation using recurrent neural networks. In: Proceedings of the ACM India joint international conference on data science and management of data, ACM Saurav S (2018) Online anomaly detection with concept drift adaptation using recurrent neural networks. In: Proceedings of the ACM India joint international conference on data science and management of data, ACM
18.
Zurück zum Zitat Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRef Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780CrossRef
19.
Zurück zum Zitat Cho K (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation, In: arXiv preprint arXiv:1406.1078 Cho K (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation, In: arXiv preprint arXiv:1406.1078
20.
Zurück zum Zitat Gerstner W, Kistler WM (2002) Spiking neuron models: Single neurons, populations, plasticity. Cambridge University Press, CambridgeCrossRef Gerstner W, Kistler WM (2002) Spiking neuron models: Single neurons, populations, plasticity. Cambridge University Press, CambridgeCrossRef
21.
Zurück zum Zitat Lobo JL et al (2018) Evolving spiking neural networks for online learning over drifting data streams. Neural Netw 108:1–19CrossRef Lobo JL et al (2018) Evolving spiking neural networks for online learning over drifting data streams. Neural Netw 108:1–19CrossRef
22.
Zurück zum Zitat Budiman A, Fanany MI, Basaruddin C (2016) Adaptive convolutional ELM for concept drift handling in online stream data. In: arXiv preprint arXiv:1610.02348 Budiman A, Fanany MI, Basaruddin C (2016) Adaptive convolutional ELM for concept drift handling in online stream data. In: arXiv preprint arXiv:1610.02348
23.
Zurück zum Zitat Sethi TS, Kantardzic M (2018) Handling adversarial concept drift in streaming data. Expert Syst Appl 97:18–40CrossRef Sethi TS, Kantardzic M (2018) Handling adversarial concept drift in streaming data. Expert Syst Appl 97:18–40CrossRef
24.
Zurück zum Zitat Niyaz Q, Sun W, Javaid AY, Alam M (2016) A deep learning approach for network intrusion detection system. In: Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies (formerly BIONETICS) ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering) Niyaz Q, Sun W, Javaid AY, Alam M (2016) A deep learning approach for network intrusion detection system. In: Proceedings of the 9th EAI International Conference on Bio-inspired Information and Communications Technologies (formerly BIONETICS) ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering)
25.
Zurück zum Zitat Abramson M (2015) Oward adversarial online learning and the science of deceptive machines. In: AAAI fall symposium series Abramson M (2015) Oward adversarial online learning and the science of deceptive machines. In: AAAI fall symposium series
26.
Zurück zum Zitat Chinavle D et al (2009), Ensembles in adversarial classification for spam. In: Proceedings of the 18th ACM conference on Information and knowledge management, ACM Chinavle D et al (2009), Ensembles in adversarial classification for spam. In: Proceedings of the 18th ACM conference on Information and knowledge management, ACM
27.
Zurück zum Zitat Grosse K et al (2017) On the (statistical) detection of adversarial examples. In: arXiv preprint arXiv Grosse K et al (2017) On the (statistical) detection of adversarial examples. In: arXiv preprint arXiv
28.
Zurück zum Zitat Kantchelian A et al (2013) Approaches to adversarial drift. In: Proceedings of the 2013 ACM workshop on artificial intelligence and security, ACM Kantchelian A et al (2013) Approaches to adversarial drift. In: Proceedings of the 2013 ACM workshop on artificial intelligence and security, ACM
Metadaten
Titel
Concept Drift for Big Data
verfasst von
Raihan Seraj
Mohiuddin Ahmed
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-35642-2_2