Skip to main content

2018 | OriginalPaper | Buchkapitel

4. Variety Management for Big Data

verfasst von : Wolfgang Mayer, Georg Grossmann, Matt Selway, Jan Stanek, Markus Stumptner

Erschienen in: Semantic Applications

Verlag: Springer Berlin Heidelberg

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Of the core challenges originally associated with Big Data, namely Volume, Velocity, and Variety, the Variety aspect is the one that is least addressed by the standard analytics architectures. In this chapter, we analyze types and sources of variety and describe data- and metadata management principles for organizing data lakes. We discuss how semantic metadata can help describe and manage variety in structure, provenance, visibility and permitted use. Moreover, ontologies and metadata catalogs can aid discovery, navigation, exploration, and interpretation of heterogeneous data lakes, and can simplify interpretation, lift data quality, and simplify integration of multiple data sets. We present an application of these principles in a data architecture for the Law Enforcement domain in Australia.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Laney D (2001) 3D data management: controlling data volume, velocity and variety. META Group Inc, Stamford, Connecticut Laney D (2001) 3D data management: controlling data volume, velocity and variety. META Group Inc, Stamford, Connecticut
2.
Zurück zum Zitat NewVantage Partners LLC (2016) Big Data executive survey 2016. NewVantage Partners, Boston, MA NewVantage Partners LLC (2016) Big Data executive survey 2016. NewVantage Partners, Boston, MA
3.
Zurück zum Zitat Dayley A, Logan D (2015) Organizations will need to tackle three challenges to curb unstructured data glut and neglect. Gartner report G00275931. Updated Jan 2017 Dayley A, Logan D (2015) Organizations will need to tackle three challenges to curb unstructured data glut and neglect. Gartner report G00275931. Updated Jan 2017
4.
Zurück zum Zitat Marz N, Warren J (2013) Big Data: principles and best practices of scalable realtime data systems. Manning Publications, Manning, New York Marz N, Warren J (2013) Big Data: principles and best practices of scalable realtime data systems. Manning Publications, Manning, New York
5.
Zurück zum Zitat Russom P (2017) Data lakes: purposes, practices, patterns, and platforms. Technical report, TDWI Russom P (2017) Data lakes: purposes, practices, patterns, and platforms. Technical report, TDWI
6.
Zurück zum Zitat D2D CRC (2016) Big Data reference architecture, vol 1–4. Data to Decisions Cooperative Research Centre, Adelaide D2D CRC (2016) Big Data reference architecture, vol 1–4. Data to Decisions Cooperative Research Centre, Adelaide
7.
Zurück zum Zitat Stumptner M, Mayer W, Grossmann G, Liu J, Li W, Casanovas P, De Koker L, Mendelson D, Watts D, Bainbridge B (2016) An architecture for establishing legal semantic workflows in the context of Integrated Law Enforcement. In: Proceedings of the third workshop on legal knowledge and the semantic web (LK&SW-2016). Co-located with EKAW-2016, ArXiv Stumptner M, Mayer W, Grossmann G, Liu J, Li W, Casanovas P, De Koker L, Mendelson D, Watts D, Bainbridge B (2016) An architecture for establishing legal semantic workflows in the context of Integrated Law Enforcement. In: Proceedings of the third workshop on legal knowledge and the semantic web (LK&SW-2016). Co-located with EKAW-2016, ArXiv
8.
Zurück zum Zitat Mayer W, Stumptner M, Casanovas P, de Koker L (2017) Towards a linked information architecture for integrated law enforcement. In: Proceedings of the workshop on linked democracy: artificial intelligence for democratic innovation (LINKDEM 2017), vol 1897. Co-located with the 26th international joint conference on artificial intelligence (IJCAI 2017), CEUR Mayer W, Stumptner M, Casanovas P, de Koker L (2017) Towards a linked information architecture for integrated law enforcement. In: Proceedings of the workshop on linked democracy: artificial intelligence for democratic innovation (LINKDEM 2017), vol 1897. Co-located with the 26th international joint conference on artificial intelligence (IJCAI 2017), CEUR
9.
10.
Zurück zum Zitat Bellahsene Z, Bonifati A, Rahm E (2011) Schema matching and mapping. Springer, Berlin, Heidelberg Bellahsene Z, Bonifati A, Rahm E (2011) Schema matching and mapping. Springer, Berlin, Heidelberg
11.
Zurück zum Zitat Del Corro L, Gemulla R (2013) ClausIE: clause-based open information extraction. In: Proceedings of WWW. ACM New York, NY, USA Del Corro L, Gemulla R (2013) ClausIE: clause-based open information extraction. In: Proceedings of WWW. ACM New York, NY, USA
12.
Zurück zum Zitat Beheshti S-M-R, Tabebordbar A, Benatallah B, Nouri R (2017) On automating basic data curation tasks. In: Proceedings of WWW. ACM, Geneva, Switzerland. pp 165–169 Beheshti S-M-R, Tabebordbar A, Benatallah B, Nouri R (2017) On automating basic data curation tasks. In: Proceedings of WWW. ACM, Geneva, Switzerland. pp 165–169
13.
Zurück zum Zitat Sun Y-JJ, Barukh MC, Benatallah B, Beheshti S-M-R (2015) Scalable SaaS-based process customization with CaseWalls. In: Proceedings of ICSOC. LNCS, vol 9435. Springer, Berlin, Heidelberg. pp 218–233CrossRef Sun Y-JJ, Barukh MC, Benatallah B, Beheshti S-M-R (2015) Scalable SaaS-based process customization with CaseWalls. In: Proceedings of ICSOC. LNCS, vol 9435. Springer, Berlin, Heidelberg. pp 218–233CrossRef
14.
Zurück zum Zitat Drogemuller A, Cunningham A, Walsh J, Ross W, Thomas B (2017) VRige: exploring social network interactions in immersive virtual environments. In: Proceedings of the international symposium on big data visual analytics (BDVA). IEEE NJ, USA Drogemuller A, Cunningham A, Walsh J, Ross W, Thomas B (2017) VRige: exploring social network interactions in immersive virtual environments. In: Proceedings of the international symposium on big data visual analytics (BDVA). IEEE NJ, USA
15.
Zurück zum Zitat Bastiras J, Thomas BH, Walsh JA, Baumeister J (2017) Combining virtual reality and narrative visualisation to persuade. In: Proceedings of the international symposium on big data visual analytics (BDVA). IEEE NJ, USA Bastiras J, Thomas BH, Walsh JA, Baumeister J (2017) Combining virtual reality and narrative visualisation to persuade. In: Proceedings of the international symposium on big data visual analytics (BDVA). IEEE NJ, USA
16.
Zurück zum Zitat Kurtev I, Jouault F, Allilaire F, Bezivin J (2008) ATL: a model transformation tool. Sci Comput Program 72(1):31–39MathSciNetMATH Kurtev I, Jouault F, Allilaire F, Bezivin J (2008) ATL: a model transformation tool. Sci Comput Program 72(1):31–39MathSciNetMATH
17.
Zurück zum Zitat Polack F, Kolovos DS, Paige RF (2008) The Epsilon transformation language. In: Proceedings of ICMT. LNCS, vol 5063. Springer, Berlin, Heidelberg Polack F, Kolovos DS, Paige RF (2008) The Epsilon transformation language. In: Proceedings of ICMT. LNCS, vol 5063. Springer, Berlin, Heidelberg
18.
Zurück zum Zitat Shvaiko P, Euzenat J (2013) Ontology matching. Springer, Berlin, Heidelberg Shvaiko P, Euzenat J (2013) Ontology matching. Springer, Berlin, Heidelberg
19.
Zurück zum Zitat Szekely P, Knoblock CA, Yang F, Zhu X, Fink EE, Allen R, Goodlander G (2013) Connecting the Smithsonian American Art Museum to the linked data cloud. In: Proceedings of ESWC Szekely P, Knoblock CA, Yang F, Zhu X, Fink EE, Allen R, Goodlander G (2013) Connecting the Smithsonian American Art Museum to the linked data cloud. In: Proceedings of ESWC
20.
Zurück zum Zitat Russom P (2016) Best practices for data lake management. Technical report, TDWI Russom P (2016) Best practices for data lake management. Technical report, TDWI
Metadaten
Titel
Variety Management for Big Data
verfasst von
Wolfgang Mayer
Georg Grossmann
Matt Selway
Jan Stanek
Markus Stumptner
Copyright-Jahr
2018
Verlag
Springer Berlin Heidelberg
DOI
https://doi.org/10.1007/978-3-662-55433-3_4

Premium Partner