Skip to main content

2020 | OriginalPaper | Buchkapitel

Big Data Processing Based on Machine Learning for Multi-user Environments

verfasst von : Kamel H. Rahouma, Farag M. Afify

Erschienen in: Micro-Electronics and Telecommunication Engineering

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Many sources of data yield non-structured data like the Internet of things (IoT), geospatial data, E-commerce, social media, and scientific research that is not appropriate in to traditional, structured warehouses. Nowadays, sophisticated analytical techniques allow companies to obtain perspicacity from data with earlier unachievable levels of accuracy and speed. Real-time analytics for big data is the capability to achieve the most suitable decisions and get significant actions at the best time. First, we present a survey of processing the big data (BD) in real time (RT) and focus on its challenges. Then, we propose an algorithm to handle BD by integration with machine learning operations in multi-user environment optimization operations, reduce maintenance costs and better speed of fault detector and provide common operations necessary to process unstructured information. There are important conditions that have been taken into a concern to guarantee the quality of services (QoS) and transmission velocity and ensure the system’s physical time synchronization and the correctness of the data processing.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Kshetri N (2014) Big data s impact on privacy, security and consumer welfare. Telecommun Policy 38(11):1134–1145CrossRef Kshetri N (2014) Big data s impact on privacy, security and consumer welfare. Telecommun Policy 38(11):1134–1145CrossRef
2.
Zurück zum Zitat Kitchin R (2014) The real-time city? Big data and smart urbanism. Geo J 1–14 Kitchin R (2014) The real-time city? Big data and smart urbanism. Geo J 1–14
3.
Zurück zum Zitat Chang RM, Kauffman RJ, Kwon YO (2014) Understanding the paradigm shift to computational social science in the presence of big data. Decis Support Syst 63:67–80CrossRef Chang RM, Kauffman RJ, Kwon YO (2014) Understanding the paradigm shift to computational social science in the presence of big data. Decis Support Syst 63:67–80CrossRef
4.
Zurück zum Zitat Chen J, Chen Y, Du X et al (2013) Big data challenge: a data management perspective. Front Comput Sci 7(2):157–164MathSciNetCrossRef Chen J, Chen Y, Du X et al (2013) Big data challenge: a data management perspective. Front Comput Sci 7(2):157–164MathSciNetCrossRef
5.
Zurück zum Zitat Zhao JM, Wang WS, Liu X et al (2014) Big data benchmark big DS. Advancing Big Data Benchmarks. Springer International Publishing, pp 49–57 Zhao JM, Wang WS, Liu X et al (2014) Big data benchmark big DS. Advancing Big Data Benchmarks. Springer International Publishing, pp 49–57
6.
Zurück zum Zitat Wang CC, Chen CL, Hou ZY et al (2015) A 60 V tolerance transceiver with ESD protection for FlexRay-based communication systems. IEEE Trans Circ Syst I: Regul Pap 62(3):752–760MathSciNet Wang CC, Chen CL, Hou ZY et al (2015) A 60 V tolerance transceiver with ESD protection for FlexRay-based communication systems. IEEE Trans Circ Syst I: Regul Pap 62(3):752–760MathSciNet
7.
Zurück zum Zitat Hwang K, Chen M (2017) Big data analytics for cloud/IoT and cognitive learning. Wiley, UK Hwang K, Chen M (2017) Big data analytics for cloud/IoT and cognitive learning. Wiley, UK
8.
Zurück zum Zitat Hwang K, Chen M, Wu J (2016) Mobile big data management and innovative applications (editorial). IEEE Trans Serv Comput 9(5):784–785 Hwang K, Chen M, Wu J (2016) Mobile big data management and innovative applications (editorial). IEEE Trans Serv Comput 9(5):784–785
9.
Zurück zum Zitat Bende S, Shedge R (2016) Dealing with small files problem in hadoop distributed file system. Procedia Comput Sci 79:1001–1012CrossRef Bende S, Shedge R (2016) Dealing with small files problem in hadoop distributed file system. Procedia Comput Sci 79:1001–1012CrossRef
11.
Zurück zum Zitat Cheng D, Zhou X, Lama P, Wu J, Jiang C (2017) Cross-platform resource scheduling for spark and MapReduce on YARN. IEEE Trans Comput 66:1341 Cheng D, Zhou X, Lama P, Wu J, Jiang C (2017) Cross-platform resource scheduling for spark and MapReduce on YARN. IEEE Trans Comput 66:1341
12.
Zurück zum Zitat Wang B, Jiang J, Wu Y, Yang G, Li K (2016) Accelerating MapReduce on commodity clusters: an SSD-empowered approach. In: IEEE transactions on big data, IEEE, 2016 Wang B, Jiang J, Wu Y, Yang G, Li K (2016) Accelerating MapReduce on commodity clusters: an SSD-empowered approach. In: IEEE transactions on big data, IEEE, 2016
13.
Zurück zum Zitat Y. Liu, M. Qiu, C. Liu, et al., Big data challenges in ocean observation: a survey, Personal Ubiquitous Comput. 2017 Y. Liu, M. Qiu, C. Liu, et al., Big data challenges in ocean observation: a survey, Personal Ubiquitous Comput. 2017
14.
Zurück zum Zitat Yildiz O, Ibrahim S, Antoniu G (2017) Enabling fast failure recovery in shared Hadoopclusters: towards failure-aware scheduling. Future Gener Comput Syst 74:208–219CrossRef Yildiz O, Ibrahim S, Antoniu G (2017) Enabling fast failure recovery in shared Hadoopclusters: towards failure-aware scheduling. Future Gener Comput Syst 74:208–219CrossRef
15.
Zurück zum Zitat Mavridis I, Karatza H (2017) Performance evaluation of cloud-based log file analysis with Apache Hadoop and Apache Spark. J Syst Softw 125:133–151CrossRef Mavridis I, Karatza H (2017) Performance evaluation of cloud-based log file analysis with Apache Hadoop and Apache Spark. J Syst Softw 125:133–151CrossRef
16.
Zurück zum Zitat Xu H, Lau WC (2017) Optimization for speculative execution in big data processing clusters. IEEE Trans Parallel Distrib Syst 28(2):530–545 Xu H, Lau WC (2017) Optimization for speculative execution in big data processing clusters. IEEE Trans Parallel Distrib Syst 28(2):530–545
17.
Zurück zum Zitat Gani A, Siddiqa A, Shamshirband S, Hanum F (2016) A survey on indexing techniques for big data: taxonomy and performance evaluation. Knowl Inf Syst 46(2):241–284CrossRef Gani A, Siddiqa A, Shamshirband S, Hanum F (2016) A survey on indexing techniques for big data: taxonomy and performance evaluation. Knowl Inf Syst 46(2):241–284CrossRef
18.
Zurück zum Zitat Gil D, Song I-Y (2016) Modeling and management of big data: challenges and opportunities. Future Gener Comput Syst 63:96–99CrossRef Gil D, Song I-Y (2016) Modeling and management of big data: challenges and opportunities. Future Gener Comput Syst 63:96–99CrossRef
19.
Zurück zum Zitat Xu H, Lau WC (2017) Optimization for speculative execution in big data processing clusters. IEEE Trans Parallel Distrib Syst 28(2):530–545 Xu H, Lau WC (2017) Optimization for speculative execution in big data processing clusters. IEEE Trans Parallel Distrib Syst 28(2):530–545
20.
Zurück zum Zitat Sivarajah Uthayasankar (2017) Muhammad Mustafa Kamal, Zahir Irani, Vishanth Weerakkody, Critical analysis of Big Data challenges and analytical methods. J Bus Res 70:263–286CrossRef Sivarajah Uthayasankar (2017) Muhammad Mustafa Kamal, Zahir Irani, Vishanth Weerakkody, Critical analysis of Big Data challenges and analytical methods. J Bus Res 70:263–286CrossRef
21.
Zurück zum Zitat Zhou L, Pan S, Wang J, Vasilakos AV (2017) Machine learning on big data: opportunities and challenges. Neurocomputing 237(10 May):350–361 Zhou L, Pan S, Wang J, Vasilakos AV (2017) Machine learning on big data: opportunities and challenges. Neurocomputing 237(10 May):350–361
24.
Zurück zum Zitat Yokota R, Wu W (eds) (2018) Supercomputing frontiers. In: 4th Asian conference, SCFA 2018 Singapore, 26–29 March 2018 Yokota R, Wu W (eds) (2018) Supercomputing frontiers. In: 4th Asian conference, SCFA 2018 Singapore, 26–29 March 2018
25.
Zurück zum Zitat Feldman D, Schmidt M, Sohler C (2013) Turning big data into tiny data: constant-size coresets for k-means, PCA and projective clustering. In: SODA, 2013 Feldman D, Schmidt M, Sohler C (2013) Turning big data into tiny data: constant-size coresets for k-means, PCA and projective clustering. In: SODA, 2013
26.
Zurück zum Zitat Jiang F, Leung CK (2015) A data analytic algorithm for managing, querying, and processing uncertain big data in cloud environments. Algorithms 8:1175–1194CrossRef Jiang F, Leung CK (2015) A data analytic algorithm for managing, querying, and processing uncertain big data in cloud environments. Algorithms 8:1175–1194CrossRef
27.
Zurück zum Zitat Cuzzocrea A, Cosulschi M, De Virgilio R (2016) An effective and efficient MapReduce algorithm for computing BFS-based traversals of large-scale RDF graphs. Algorithms Cuzzocrea A, Cosulschi M, De Virgilio R (2016) An effective and efficient MapReduce algorithm for computing BFS-based traversals of large-scale RDF graphs. Algorithms
28.
29.
Zurück zum Zitat Zhou L, Pan S, Wang J, Vasilakos AV (2017) Machine learning on big data: opportunities and challenges. Neurocomputing 237(10):350–361CrossRef Zhou L, Pan S, Wang J, Vasilakos AV (2017) Machine learning on big data: opportunities and challenges. Neurocomputing 237(10):350–361CrossRef
Metadaten
Titel
Big Data Processing Based on Machine Learning for Multi-user Environments
verfasst von
Kamel H. Rahouma
Farag M. Afify
Copyright-Jahr
2020
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-15-2329-8_68

Neuer Inhalt