Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
ATZ-Fachkongress

Chassis.tech plus 2024


4 Kongresse in einer Veranstaltung

Treffen Sie in München die Fachleute von Chassis, Lenkung, Bremsen sowie Reifen/Räder.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
Introduction Kapitel lesen Erstes Kapitel lesen

2020 | OriginalPaper | Buchkapitel

8. Imbalanced Data Preprocessing for Big Data

verfasst von : Julián Luengo, Diego García-Gil, Sergio Ramírez-Gallego, Salvador García, Francisco Herrera

Erschienen in: Big Data Preprocessing

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The negative impact on learning associated with imbalanced proportion of classes has exploded lately with the exponential growth of “cheap” data. Many real-world problems present scarce number of instances in one class whereas in others their cardinality is several factors greater. The current techniques that treat large-scale imbalanced data are focused on obtaining fast, scalable, and parallel sampling techniques following the standard MapReduce procedure. These generate local balanced solutions in each map, which are eventually combined into a final set. Nevertheless, as we will see later, this divide-and-conquer strategy entails several problems, such as small disjuncts, data lack, etc. In this chapter we also review the latest proposals on imbalanced Big Data preprocessing and present a MapReduce framework for imbalanced preprocessing which includes several state-of-the-art sampling techniques.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Vorheriges Kapitel Big Data Discretization
Nächstes Kapitel Big Data Software
Literatur
1.
Basgall, M. J., Hasperué, W., Naiouf, M., Fernández, A., & Herrera, F. (2018). SMOTE-BD: An exact and scalable oversampling method for imbalanced classification in big data. Journal of Computer Science and Technology, 18(03), e23.CrossRef
2.
Batista, G. E. A. P. A., Prati, R. C., & Monard, M. C. (2004). A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explorations Newsletter, 6(1), 20–29.CrossRef
3.
Bhagat, R. C., & Patil, S. S. (2015). Enhanced smote algorithm for classification of imbalanced big-data using Random Forest. In Souvenir of the 2015 IEEE International Advance Computing Conference, IACC 2015 (pp. 403–408)
4.
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357.CrossRef
5.
del Río, S., Bentez, J. M., & Herrera, F. (2015). Analysis of data preprocessing increasing the oversampling ratio for extremely imbalanced Big Data classification. In 2015 IEEE Trustcom/BigDataSE/ISPA (Vol. 2, pp. 180–185).
6.
del Río, S., López, V., Benítez, J. M., & Herrera, F. (2014). On the use of MapReduce for imbalanced Big Data using random forest. Information Sciences, 285, 112–137.CrossRef
7.
Elkan, C. (2001). The foundations of cost-sensitive learning. In In Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (pp. 973–978).
8.
Fernández, A., del Río, S., Chawla, N. V., & Herrera, F. (2017). An insight into imbalanced big data classification: Outcomes and challenges. Complex & Intelligent Systems, 3(2), 105–120.CrossRef
9.
Fernández, A., López, V., Galar, M., Del Jesus, M. J., & Herrera, F. (2013). Analysing the classification of imbalanced data-sets with multiple classes: Binarization techniques and ad-hoc approaches. Knowledge-Based Systems, 42, 97–110.CrossRef
10.
Guo, Y., Graber, A., McBurney, R. N., & Balasubramanian, R. (2010). Sample size and statistical power considerations in high-dimensionality data settings: A comparative study of classification algorithms. BMC Bioinformatics, 11, 447.CrossRef
11.
Gutierrez, P. D., Lastra, M., Bacardit, J., Benitez, J. M., & Herrera, F. (2016). GPU-SME-kNN: Scalable and memory efficient kNN and lazy learning using GPUs. Information Sciences, 373, 165–182.CrossRef
12.
Gutierrez, P. D., Lastra, M., Benitez, J. M., & Herrera, F. (2017). SMOTE-GPU: Big data preprocessing on commodity hardware for imbalanced classification. Progress in Artificial Intelligence, 6(4), 347–354.CrossRef
13.
He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284.CrossRef
14.
Hu, F., & Li, H. (2013). A novel boundary oversampling algorithm based on neighborhood rough set model: NRSBoundary-SMOTE. Mathematical Problems in Engineering, 2013, 1–10.MathSciNetMATH
15.
Hu, F., Li, H., Lou, H., & Dai, J. (2014). A parallel oversampling algorithm based on NRSBoundary-SMOTE. Journal of Information and Computational Science, 11(13), 4655–4665.CrossRef
16.
Kamal, S., Ripon, S. H., Dey, N., Ashour, A. S., & Santhi, V. (2016). A MapReduce approach to diminish imbalance parameters for big deoxyribonucleic acid dataset. Computer Methods and Programs in Biomedicine, 131, 191–206.CrossRef
17.
Krawczyk, B. (2016). GPU-accelerated extreme learning machines for imbalanced data streams with concept drift. In M. Connolly (Ed.), The International Conference on Computational Science, Procedia Computer Science (Vol. 80, pp. 1692–1701)
18.
López, V., Fernández, A., del Jesus, M. J., & Herrera, F. (2013). A hierarchical genetic fuzzy system based on genetic programming for addressing classification with highly imbalanced and borderline data-sets. Knowledge-Based Systems, 38, 85–104. Special Issue on Advances in Fuzzy Knowledge Systems: Theory and Application.
19.
López, V., Fernández, A., García, S., Palade, V., & Herrera, F. (2013). An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics. Information Sciences, 250, 113–141.CrossRef
20.
Maíllo, J., Ramírez, S., Triguero, I., & Herrera, F. (2017). kNN-IS: An Iterative Spark-based design of the k-nearest neighbors classifier for big data. Knowledge-Based Systems, 117, 3–15.CrossRef
21.
Triguero, I., Derrac, J., García, S., & Herrera, F. (2012). Integrating a differential evolution feature weighting scheme into prototype generation. Neurocomputing, 97, 332–343.CrossRef
22.
Triguero, I., Galar, M., Merino, D., Maillo, J., Bustince, H., & Herrera, F. (2016). Evolutionary undersampling for extremely imbalanced big data classification under apache spark. In IEEE Congress on Evolutionary Computation (CEC 2016), Vancouver (pp. 640–647).
23.
Triguero, I., Galar, M., Vluymans, S., Cornelis, C., Bustince, H., Herrera, F., & Saeys, Y. (2015). Evolutionary undersampling for imbalanced Big Data classification. In 2015 IEEE Congress on Evolutionary Computation (CEC) (pp. 715–722).
24.
Triguero, I., Río, S., López, V., Bacardit, J., Benítez, J. M., & Herrera, F. (2015). ROSEFW-RF: The winner algorithm for the ECBDL’14 Big Data competition: An extremely imbalanced Big Data bioinformatics problem. Knowledge-Based Systems, 87, 69–79.CrossRef
25.
Zhai, J., Zhang, S., & Wang, C. (2015). The classification of imbalanced large data sets based on MapReduce and ensemble of elm classifiers. International Journal of Machine Learning and Cybernetics, 1–9.
Metadaten
Titel
Imbalanced Data Preprocessing for Big Data
verfasst von
Julián Luengo
Diego García-Gil
Sergio Ramírez-Gallego
Salvador García
Francisco Herrera
Copyright-Jahr
2020
Buch
Big Data Preprocessing

Print ISBN: 978-3-030-39104-1

Electronic ISBN: 978-3-030-39105-8

Copyright-Jahr: 2020

DOI
https://doi.org/10.1007/978-3-030-39105-8_8

Premium Partner

    Bildnachweise