Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2017 | OriginalPaper | Chapter

Big Data Storage and Management: Challenges and Opportunities

share
SHARE

Abstract

The paper is focused on today’s very popular theme – Big Data. We describe and discuss its characteristics by eleven V’s (Volume, Velocity, Variety, Veracity, etc.) and Big Data quality. These characteristics represent both data and process challenges. Then we continue with problems of Big Data storage and management. Principles of NoSQL databases are explained including their categorization. We also shortly describe Hadoop and MapReduce technologies as well as their inefficiency for some interactive queries and applications within the domain of large-scale graph processing and streaming data. NoSQL databases and Hadoop M/R are designed to take advantage of cloud computing architectures and allow massive computations to be run inexpensively and efficiently. The term of Big Data 1.0 was introduced for these technologies. We continue with some new approaches called currently Big Data 2.0 processing systems. Particularly their four categories are introduced and discussed: General purpose Big Data Processing Systems, Big SQL Processing Systems, Big Graph Processing Systems, and Big Stream Processing Systems. Then, an attention is devoted to Big Analytics – the main application area for Big Data storage and processing. We argue that enterprises with complex, heterogeneous environments no longer want to adopt a BI access point just for one data source (Hadoop). More heterogeneous software platforms are needed. Even Hadoop has become a multi-purpose engine for ad hoc analysis. Finally, we mention some problems with Big Data. We also remind that Big Data creates a new type of digital divide. Having access and knowledge of Big Data technologies gives companies and people a competitive edge in today’s data driven world.

To get access to this content you need the following product:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 69.000 Bücher
  • über 500 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Testen Sie jetzt 15 Tage kostenlos.

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 50.000 Bücher
  • über 380 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




Testen Sie jetzt 15 Tage kostenlos.

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 58.000 Bücher
  • über 300 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Testen Sie jetzt 15 Tage kostenlos.

Footnotes
Literature
1.
go back to reference Bajaber, F., Elshawi, R., Batarfi, O., Altalhi, A., Barnawi, A., Sakr, S.: Big data 2.0 processing systems: taxonomy and open challenges. J. Grid Comput. 14, 379–405 (2016) CrossRef Bajaber, F., Elshawi, R., Batarfi, O., Altalhi, A., Barnawi, A., Sakr, S.: Big data 2.0 processing systems: taxonomy and open challenges. J. Grid Comput. 14, 379–405 (2016) CrossRef
3.
go back to reference Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008) CrossRef Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008) CrossRef
4.
go back to reference Gamble, M., Goble, C.: Quality, trust and utility of scientific data on the web: toward a joint model. In: Proceedings of WebSci 2011 Conference, Koblenz, Germany, Article No. 15. ACM (2011) Gamble, M., Goble, C.: Quality, trust and utility of scientific data on the web: toward a joint model. In: Proceedings of WebSci 2011 Conference, Koblenz, Germany, Article No. 15. ACM (2011)
7.
go back to reference Malewicz, G., Austern, M.H., Bik, A.J.C., Dehnert, J.C., Horn, I., Leiser, N., Czajkowski, G.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, SIGMOD 2010, pp. 135–146 (2010) Malewicz, G., Austern, M.H., Bik, A.J.C., Dehnert, J.C., Horn, I., Leiser, N., Czajkowski, G.: Pregel: a system for large-scale graph processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, SIGMOD 2010, pp. 135–146 (2010)
8.
go back to reference Nasser, T., Tariq, R.S.: Big data challenges. J. Comput. Eng. Inf. Technol. 4(3), 1–6 (2015) MathSciNet Nasser, T., Tariq, R.S.: Big data challenges. J. Comput. Eng. Inf. Technol. 4(3), 1–6 (2015) MathSciNet
9.
go back to reference Pokorny, J.: Database technologies in the world of big data. In: Proceedings of the 16th International Conference on Computer Systems and Technologies, CompSysTech 2015. ACM International Conference Proceeding Series, vol. 1008, pp. 1–12. ACM, New York (2015) Pokorny, J.: Database technologies in the world of big data. In: Proceedings of the 16th International Conference on Computer Systems and Technologies, CompSysTech 2015. ACM International Conference Proceeding Series, vol. 1008, pp. 1–12. ACM, New York (2015)
11.
go back to reference Pokorný, J., Stantic, B.: Challenges and opportunities in big data processing (Chapter 1). In: Ma, Z. (ed.) Managing Big Data in Cloud Computing Environments. IGI Global, Advances in Data Mining and Database Management (2016) Pokorný, J., Stantic, B.: Challenges and opportunities in big data processing (Chapter 1). In: Ma, Z. (ed.) Managing Big Data in Cloud Computing Environments. IGI Global, Advances in Data Mining and Database Management (2016)
14.
go back to reference Stonebraker, M.: Technical perspective - one size fits all: an idea whose time has come and gone. Commun. ACM 51(12), 76 (2008) CrossRef Stonebraker, M.: Technical perspective - one size fits all: an idea whose time has come and gone. Commun. ACM 51(12), 76 (2008) CrossRef
15.
go back to reference Tivari, S.: Professional NoSQL. Wiley/Wrox, Hoboken (2011) Tivari, S.: Professional NoSQL. Wiley/Wrox, Hoboken (2011)
16.
go back to reference Wu, C., Buyya, R., Ramamohanarao, K.: Big data analytics = machine learning + cloud computing. In: Buyya, R., Calheiros, R., Dastjerdi, A. (eds.) Big Data: Principles and Paradigms. Morgan Kaufmann, Burlington (2016) CrossRef Wu, C., Buyya, R., Ramamohanarao, K.: Big data analytics = machine learning + cloud computing. In: Buyya, R., Calheiros, R., Dastjerdi, A. (eds.) Big Data: Principles and Paradigms. Morgan Kaufmann, Burlington (2016) CrossRef
Metadata
Title
Big Data Storage and Management: Challenges and Opportunities
Author
Jaroslav Pokorný
Copyright Year
2017
DOI
https://doi.org/10.1007/978-3-319-89935-0_3

Premium Partner