nach oben

Erschienen in:

2017 | OriginalPaper | Buchkapitel

Managing Lifecycle of Big Data Applications

verfasst von : Ivan Ermilov, Axel-Cyrille Ngonga Ngomo, Aad Versteden, Hajira Jabeen, Gezim Sejdiu, Giorgos Argyriou, Luigi Selmi, Jürgen Jakobitsch, Jens Lehmann

Erschienen in: Knowledge Engineering and Semantic Web

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The growing digitization and networking process within our society has a large influence on all aspects of everyday life. Large amounts of data are being produced continuously, and when these are analyzed and interlinked they have the potential to create new knowledge and intelligent solutions for economy and society. To process this data, we developed the Big Data Integrator (BDI) Platform with various Big Data components available out-of-the-box. The integration of the components inside the BDI Platform requires components homogenization, which leads to the standardization of the development process. To support these activities we created the BDI Stack Lifecycle (SL), which consists of development, packaging, composition, enhancement, deployment and monitoring steps. In this paper, we show how we support the BDI SL with the enhancement applications developed in the BDE project. As an evaluation, we demonstrate the applicability of the BDI SL on three pilots in the domains of transport, social sciences and security.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Ephedra: Efficiently Combining RDF Data and Services Using SPARQL Federation

Nächstes Kapitel Ontology-Based Representation of Learner Profiles for Accessible OpenCourseWare Systems

http://hadoop.apache.org/.

https://spark.apache.org/.

https://flink.apache.org/.

https://hive.apache.org/.

https://www.big-data-europe.eu/bdi-components/.

At the time of writing, more than 30 components are available in BDE Components Library.

https://hortonworks.com/.

https://www.cloudera.com/.

https://mapr.com.

http://bigtop.apache.org/.

https://dcos.io/.

https://jujucharms.com/store.

https://github.com/big-data-europe/.

https://github.com/big-data-europe/app-stack-builder.

https://kafka.apache.org/.

https://github.com/big-data-europe/app-pipeline-builder.

https://github.com/big-data-europe/mu-pipeline-service.

https://github.com/big-data-europe/mu-bde-logging.

https://github.com/big-data-europe/app-integrator-ui.

https://github.com/big-data-europe/app-swarm-ui.

https://www.big-data-europe.eu/pilot-transport/.

https://www.elastic.co/products/elasticsearch.

https://www.elastic.co/products/kibana.

https://github.com/big-data-europe/pilot-sc4-fcd-applications.

https://www.big-data-europe.eu/pilot-social-sciences/.

http://linkedeconomy.org/en.

https://github.com/LinkedEcon/LinkedEconomyOntology-ELOD.

http://www.accountingverse.com/managerial-accounting/fs-analysis/financial-ratios.html.

http://flume.apache.org.

https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html.

https://virtuoso.openlinksw.com/.

https://en.wikipedia.org/wiki/Financial_ratio.

https://www.w3.org/TR/sparql11-query/.

https://www.poolparty.biz/.

https://www.poolparty.biz/poolparty-semantic-graph-search-server/.

https://github.com/big-data-europe/pilot-sc6-cycle2.

https://docs.docker.com/engine/reference/builder/#healthcheck.

https://github.com/big-data-europe/mu-bde-logging.

https://www.big-data-europe.eu/security/.

http://cassandra.apache.org/.

http://www.gadm.org/.

https://scihub.copernicus.eu/.

The typical loading time for a set of images: 400 s.

The typical Spark job execution time for Change Detector: 1000 s.

https://github.com/big-data-europe/pilot-sc7-change-detector.

Auer, S., et al.: The BigDataEurope platform – supporting the variety dimension of big data. In: Cabot, J., Virgilio, R., Torlone, R. (eds.) ICWE 2017. LNCS, vol. 10360, pp. 41–59. Springer, Cham (2017). doi:10.1007/978-3-319-60131-1_3. http://jens-lehmann.org/files/2017/icwe_bde.pdf CrossRef

Ermilov, I.: Scalable spark/hdfs workbench using docker (2016), https://www.big-data-europe.eu/scalable-sparkhdfs-workbench-using-docker/. Retrieved 21 May 2017

Ermilov, I.: Developing spark applications with docker and BDE (2017), https://www.big-data-europe.eu/developing-spark-applications-with-docker-and-bde/. Retrieved 21 May 2017

Ermilov, I.: User interface integration in BDI platform (integrator UI application) (2017), https://www.big-data-europe.eu/user-interface-integration-in-bdi-platform-integrator-ui-application/. Retrieved 21 May 2017

Grady, N.W.: KDD meets big data. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 1603–1608. IEEE (2016)

Harney, J., Lim, S.H., Sukumar, S., Stansberry, D., Xenopoulos, P.: On-demand data analytics in HPC environments at leadership computing facilities: Challenges and experiences. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 2087–2096. IEEE (2016)

Heit, J., Liu, J., Shah, M.: An architecture for the deployment of statistical models for the big data era. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 1377–1384. IEEE (2016)

Jabeen, H.: Bde vs. other hadoop distributions (2016), https://www.big-data-europe.eu/bde-vs-other-hadoop-distributions/. Retrieved 21 May 2017

Konstantopoulos, S., Charalambidis, A., Mouchakis, G., Troumpoukis, A., Jakobitch, J., Karkaletsis, V.: Semantic web technologies and big data infrastructures: SPARQL federated querying of heterogeneous big data stores. In: ISWC Demos and Posters Track (2016)

10.

Kyzirakos, K., Karpathiotakis, M., Koubarakis, M.: Strabon: a semantic geospatial DBMS. In: Cudré-Mauroux, P., Heflin, J., Sirin, E., Tudorache, T., Euzenat, J., Hauswirth, M., Parreira, J.X., Hendler, J., Schreiber, G., Bernstein, A., Blomqvist, E. (eds.) ISWC 2012. LNCS, vol. 7649, pp. 295–311. Springer, Heidelberg (2012). doi:10.1007/978-3-642-35176-1_19 CrossRef

11.

Kyzirakos, K., Vlachopoulos, I., Savva, D., Manegold, S., Koubarakis, M.: Geotriples: a tool for publishing geospatial data as RDF graphs using R2RML mappings. In: Proceedings of the 2014 International Conference on Posters & Demonstrations Track, vol. 1272, pp. 393–396. CEUR-WS. org (2014)

12.

Nikolaou, C., Dogani, K., Bereta, K., Garbis, G., Karpathiotakis, M., Kyzirakos, K., Koubarakis, M.: Sextant: Visualizing time-evolving linked geospatial data. Web Semant. Sci. Serv. Agents World Wide Web 35, 35–52 (2015)CrossRef

13.

Rahman, F., Slepian, M., Mitra, A.: A novel big-data processing framework for healthcare applications: big-data-healthcare-in-a-box. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 3548–3555. IEEE (2016)

14.

Rodriguez, P., Haghighatkhah, A., Lwakatare, L.E., Teppola, S., Suomalainen, T., Eskeli, J., Karvonen, T., Kuvaja, P., Verner, J.M., Oivo, M.: Continuous deployment of software intensive products and services: a systematic mapping study. J. Syst. Softw. 123, 263–291 (2017)CrossRef

15.

Sebrechts, M., Borny, S., Vanhove, T., Van Seghbroeck, G., Wauters, T., Volckaert, B., De Turck, F.: Model-driven deployment and management of workflows on analytics frameworks. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 2819–2826. IEEE (2016)

16.

Sezer, O.B., Dogdu, E., Ozbayoglu, M., Onal, A.: An extended iot framework with semantics, big data, and analytics. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 1849–1856. IEEE (2016)

17.

Shearer, C.: The CRISP-DM model: the new blueprint for data mining. J. Data Warehouse. 5(4), 13–22 (2000)

18.

Tsakalozos, K., Johns, C., Monroe, K., VanderGiessen, P., Mcleod, A., Rosales, A.: Open big data infrastructures to everyone. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 2127–2129. IEEE (2016)

19.

Versteden, A., Pauwels, E.: State-of-the-dart web applications using microservices and linked data. In: Maleshkova, M., Verborgh, R., Keppmann, F.L. (eds.) 4th Workshop on Services and Applications over Linked APIs and Data (SALAD), vol. 1629, pp. 25–36. CEUR Workshop Proceedings, Aachen (2016). http://ceur-ws.org/Vol-1629/paper4.pdf

Titel: Managing Lifecycle of Big Data Applications
verfasst von: Ivan Ermilov
Axel-Cyrille Ngonga Ngomo
Aad Versteden
Hajira Jabeen
Gezim Sejdiu
Giorgos Argyriou
Luigi Selmi
Jürgen Jakobitsch
Jens Lehmann
Verlag: Springer International Publishing
Buch: Knowledge Engineering and Semantic Web
Print ISBN: 978-3-319-69547-1

Electronic ISBN: 978-3-319-69548-8

Copyright-Jahr: 2017
DOI: https://doi.org/10.1007/978-3-319-69548-8_18

Neuer Inhalt

Bildnachweise

VDI-Icon, Profil Icon, inhalt2, Springer Professional Modul/© Springer Fachmedien Wiesbaden GmbH, Internationaler Motorenkongress/© [M] ATZlive | Chisnikov / Fotolia.com, Search Icon, Banner Hanser, Benny Hahn/© ZEP GmbH, Customer Experience/© © oatawa / Getty Images / iStock, Erdgasmotor 1.5 TGI evo von Volkswagen/© Volkswagen AG, Zeitschrift Wissensmanagement Cover, PatentFit-Logo/© Springer Fachmedien Wiesbaden GmbH, 2023_Antrieb/© supervisuell, ATZ-Webinar: Prototypenfreie Entwicklung durch Offline- und Driver-in-the-Loop-HiL-Tests /© (c) VI-grade, chassis.tech plus 2023/© [M] ATZlive / TÜV SÜD PRODUCT SERVICE GMBH

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Neuer Inhalt

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.