nach oben

Erschienen in:

2018 | OriginalPaper | Buchkapitel

Evaluating Several Design Patterns and Trends in Big Data Warehousing Systems

verfasst von : Carlos Costa, Maribel Yasmina Santos

Erschienen in: Advanced Information Systems Engineering

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The Big Data characteristics, namely volume, variety and velocity, currently highlight the severe limitations of traditional Data Warehouses (DWs). Their strict relational model, costly scalability, and, sometimes, inefficient performance open the way for emerging techniques and technologies. Recently, the concept of Big Data Warehousing is gaining attraction, aiming to study and propose new ways of dealing with the Big Data challenges in Data Warehousing contexts. The Big Data Warehouse (BDW) can be seen as a flexible, scalable and highly performant system that uses Big Data techniques and technologies to support mixed and complex analytical workloads (e.g., streaming analysis, ad hoc querying, data visualization, data mining, simulations) in several emerging contexts like Smart Cities and Industries 4.0. However, due to the almost embryonic state of this topic, the ambiguity of the constructs and the lack of common approaches still prevails. In this paper, we discuss and evaluate some design patterns and trends in Big Data Warehousing systems, including data modelling techniques (e.g., star schemas, flat tables, nested structures) and some streaming considerations for BDWs (e.g., Hive vs. NoSQL databases), aiming to foster and align future research, and to help practitioners in this area.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel VizDSL: A Visual DSL for Interactive Information Visualization

Nächstes Kapitel KAYAK: A Framework for Just-in-Time Data Preparation in a Data Lake

Chandarana, P., Vijayalakshmi, M.: Big Data analytics frameworks. In: 2014 International Conference on Circuits, Systems, Communication and Information Technology Applications (CSCITA), pp. 430–434 (2014)

Ward, J.S., Barker, A.: Undefined by data: a survey of big data definitions. arXiv:13095821 CsDB (2013)

Chen, M., Mao, S., Liu, Y.: Big data: a survey. Mob. Netw. Appl. 19, 171–209 (2014). https://doi.org/10.1007/s11036-013-0489-0CrossRef

Madden, S.: From databases to big data. IEEE Internet Comput. 16, 4–6 (2012). https://doi.org/10.1109/MIC.2012.50CrossRef

Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big Data: The Next Frontier for Innovation, Competition, and Productivity. McKinsey Global Institute, San Francisco (2011)

NBD-PWG: NIST Big Data Interoperability Framework, vol. 6, Reference Architecture. National Institute of Standards and Technology (2015)

Goss, R.G., Veeramuthu, K.: Heading towards big data building a better data warehouse for more data, more speed, and more users. In: 2013 24th Annual SEMI on Advanced Semiconductor Manufacturing Conference (ASMC), pp. 220–225. IEEE (2013)

Krishnan, K.: Data Warehousing in the Age of Big Data. Morgan Kaufmann Publishers Inc., San Francisco (2013)

Mohanty, S., Jagadeesh, M., Srivatsa, H.: Big Data Imperatives: Enterprise Big Data Warehouse, BI Implementations and Analytics. Apress, New York City (2013)CrossRef

10.

Kobielus, J.: Hadoop: nucleus of the next-generation big data warehouse (2012). http://www.ibmbigdatahub.com/blog/hadoop-nucleus-next-generation-big-data-warehouse

11.

Kimball, R., Ross, M.: The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling. Wiley, New York (2013)

12.

Golab, L., Johnson, T.: Data stream warehousing. In: 2014 IEEE 30th International Conference on Data Engineering (ICDE), pp. 1290–1293 (2014)

13.

Russom, P.: Evolving Data Warehouse Architectures in the Age of Big Data. The Data Warehouse Institute (2014)

14.

Russom, P.: Data Warehouse Modernization in the Age of Big Data Analytics. The Data Warehouse Institute (2016)

15.

Sun, L., Hu, M., Ren, K., Ren, M.: Present situation and prospect of data warehouse architecture under the Background of Big Data. In: 2013 International Conference on Information Science and Cloud Computing Companion (ISCC-C), pp. 529–535. IEEE (2013)

16.

Philip Chen, C.L., Zhang, C.-Y.: Data-intensive applications, challenges, techniques and technologies: a survey on Big Data. Inf. Sci. 275, 314–347 (2014). https://doi.org/10.1016/j.ins.2014.01.015CrossRef

17.

Hashem, I.A.T., Yaqoob, I., Anuar, N.B., Mokhtar, S., Gani, A., Khan, S.U.: The rise of “big data” on cloud computing: review and open research issues. Inf. Syst. 47, 98–115 (2015). https://doi.org/10.1016/j.is.2014.07.006CrossRef

18.

O’Neil, P.E., O’Neil, E.J., Chen, X.: The star schema benchmark (SSB) (2009)

19.

TPC: TPC-H – Homepage. http://www.tpc.org/tpch/

20.

Costa, C.: SSB+ GitHub Repository. https://github.com/epilif1017a/bigdatabenchmarks

21.

Peffers, K., Tuunanen, T., Rothenberger, M., Chatterjee, S.: A design science research methodology for information systems research. J. Manage. Inf. Syst. 24, 45–77 (2007). https://doi.org/10.2753/MIS0742-1222240302CrossRef

22.

Chevalier, M., El Malki, M., Kopliku, A., Teste, O., Tournier, R.: Implementing multidimensional data warehouses into NoSQL. In: International Conference on Enterprise Information Systems (ICEIS 2015), pp. 172–183 (2015)

23.

Gröger, C., Schwarz, H., Mitschang, B.: The deep data warehouse: link-based integration and enrichment of warehouse data and unstructured content. In: IEEE 18th International Enterprise Distributed Object Computing Conference (EDOC), pp. 210–217 (2014)

24.

Cattell, R.: Scalable SQL and NoSQL data stores. ACM SIGMOD Rec. 39, 12–27 (2011). https://doi.org/10.1145/1978915.1978919CrossRef

25.

Thusoo, A., Sarma, J.S., Jain, N., Shao, Z., Chakka, P., Zhang, N., Antony, S., Liu, H., Murthy, R.: Hive-a petabyte scale data warehouse using Hadoop. In: IEEE 26th International Conference on Data Engineering (ICDE), pp. 996–1005. IEEE (2010)

26.

Floratou, A., Minhas, U.F., Özcan, F.: SQL-on-Hadoop: full circle back to shared-nothing database architectures. Proc. VLDB Endow. 7, 1295–1306 (2014). https://doi.org/10.14778/2732977.2733002CrossRef

27.

Huai, Y., Chauhan, A., Gates, A., Hagleitner, G., Hanson, E.N., O’Malley, O., Pandey, J., Yuan, Y., Lee, R., Zhang, X.: Major technical advancements in apache hive. In: Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, pp. 1235–1246. ACM, New York (2014)

28.

Song, J., Guo, C., Wang, Z., Zhang, Y., Yu, G., Pierson, J.-M.: HaoLap: A Hadoop based OLAP system for big data. J. Syst. Softw. 102, 167–181 (2015). https://doi.org/10.1016/j.jss.2014.09.024CrossRef

29.

Wang, H., Qin, X., Zhou, X., Li, F., Qin, Z., Zhu, Q., Wang, S.: Efficient query processing framework for big data warehouse: an almost join-free approach. Front. Comput. Sci. 9, 224–236 (2015)MathSciNetCrossRef

30.

Li, X., Mao, Y.: Real-time data ETL framework for big real-time data analysis. In: 2015 IEEE International Conference on Information and Automation, pp. 1289–1294 (2015)

31.

Thusoo, A., Shao, Z., Anthony, S., Borthakur, D., Jain, N., Sen Sarma, J., Murthy, R., Liu, H.: Data warehousing and analytics infrastructure at Facebook. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 1013–1020. ACM, New York (2010)

32.

Wang, S., Pandis, I., Wu, C., He, S., Johnson, D., Emam, I., Guitton, F., Guo, Y.: High dimensional biological data retrieval optimization with NoSQL technology. BMC Genom. 15(Suppl. 8), S3 (2014). https://doi.org/10.1186/1471-2164-15-S8-S3CrossRef

33.

Clegg, D.: Evolving data warehouse and BI architectures: the big data challenge (2015)

34.

Santos, M.Y., Costa, C., Galvão, J., Andrade, C., Martinho, B., Lima, F.V., Costa, E.: Evaluating SQL-on-Hadoop for Big Data warehousing on Not-So-Good hardware. In: Proceedings of International Database Engineering & Applications Symposium (IDEAS 2017), Bristol, UK (2017)

35.

Costa, E., Costa, C., Santos, M.Y.: Efficient Big Data modelling and organization for Hadoop hive-based data warehouses. In: Presented at the EMCIS 2017, Coimbra, Portugal (2017)CrossRef

36.

Costa, C., Santos, M.Y.: The SusCity Big Data warehousing approach for smart cities. In: Proceedings of International Database Engineering & Applications Symposium, p. 10 (2017)

37.

Santos, M.Y., Oliveira e Sá, J., Andrade, C., Vale Lima, F., Costa, E., Costa, C., Martinho, B., Galvão, J.: A Big Data system supporting Bosch Braga Industry 4.0 strategy. Int. J. Inf. Manag. 37(6), 750–760 (2017). https://doi.org/10.1016/j.ijinfomgt.2017.07.012CrossRef

38.

Mackey, G., Sehrish, S., Wang, J.: Improving metadata management for small files in HDFS. In: 2009 IEEE International Conference on Cluster Computing and Workshops, pp. 1–4 (2009)

39.

White, T.: Hadoop: The Definitive Guide. O’Reilly Media, Sebastopol (2015)

Titel: Evaluating Several Design Patterns and Trends in Big Data Warehousing Systems
verfasst von: Carlos Costa
Maribel Yasmina Santos
Verlag: Springer International Publishing
Buch: Advanced Information Systems Engineering
Print ISBN: 978-3-319-91562-3

Electronic ISBN: 978-3-319-91563-0

Copyright-Jahr: 2018
DOI: https://doi.org/10.1007/978-3-319-91563-0_28

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner