Skip to main content
Erschienen in:
Buchtitelbild

2014 | OriginalPaper | Buchkapitel

Big Data: Opportunities, Challenges and Solutions

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The problems related to the phenomenon of Big Data are currently among the top 10 hottest topics of information and communication technology. Big Data phenomenon refers to the data explosion observed today. At present, the term is widely used in different communities of many application domains, including researchers and practitioners. Big Data analysis can provide for many new opportunities in many respects motivating and stimulating industrial and commercial take-up of novel emerging technologies. The in-depth analysis of Big Data processing and analytics publications shows that the most of them write about “new opportunities” and “new challenges”. However, very few papers present the solutions for predictive analytics that go beyond the limits of OLAP-like processing models and technologies. The goal of this paper is to outline in more detail not only the nature of opportunities and particular challenges but also some original solutions to attack them.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
Hereinafter Big Data refers to the corresponding problem domain whereas big data refers to big data samples.
 
2
This application is used only to explain the essence of the big data analysis tasks arisen in social networks.
 
3
Unfortunately, there are many industrial practitioners who still trust that unlimited computing resources are capable to cope with any big data-related problem.
 
4
See [6] for impressive graphical illustrations of the error accumulation and the spurious correlation effects.
 
5
At this step, the testing procedure has to be applied only for data subset assigned label \( \bar{\omega }_{k} \).
 
6
Practically, causal analysis based on Bayesian network model can be used for data dimensionalities of no more than 20.
 
Literatur
1.
Zurück zum Zitat Aliferis, C.F., Statnikov, A., Tsamardinos, I., Xenofon, S.M., Koutsoukos, D.: Local causal and markov blanket induction for causal discovery and feature selection for classification Part I: Algorithms and empirical evaluation. J. Mach. Learn. Res. 11, 171–234 (2010)MATHMathSciNet Aliferis, C.F., Statnikov, A., Tsamardinos, I., Xenofon, S.M., Koutsoukos, D.: Local causal and markov blanket induction for causal discovery and feature selection for classification Part I: Algorithms and empirical evaluation. J. Mach. Learn. Res. 11, 171–234 (2010)MATHMathSciNet
4.
Zurück zum Zitat Bizer, C., Heath, T., Berners-Lee, T.: Linked data – the story so far. Int. J Semant. Web Inf. Syst. 5(3), 1–22 (2009)CrossRef Bizer, C., Heath, T., Berners-Lee, T.: Linked data – the story so far. Int. J Semant. Web Inf. Syst. 5(3), 1–22 (2009)CrossRef
7.
Zurück zum Zitat Fan, J., Guo, S., Hao, N.: Variance estimation using refitted cross-validation in ultrahigh dimensional regression. J. Roy. Stat. Soc. Ser. B (Stat. Methodol.) 74(1), 37–65 (2012)CrossRefMathSciNet Fan, J., Guo, S., Hao, N.: Variance estimation using refitted cross-validation in ultrahigh dimensional regression. J. Roy. Stat. Soc. Ser. B (Stat. Methodol.) 74(1), 37–65 (2012)CrossRefMathSciNet
8.
Zurück zum Zitat Fan, J., Fan, Y.: High dimensional classification using features annealed independence rules. Ann. Stat. 36(6), 2605–2637 (2008)CrossRefMATH Fan, J., Fan, Y.: High dimensional classification using features annealed independence rules. Ann. Stat. 36(6), 2605–2637 (2008)CrossRefMATH
9.
Zurück zum Zitat Gorodetsky, V., Samoylov, V., Tushkanova, O.: Agent-based customer profile learning in 3G recommending systems. In: Proceedings of 9-th International Workshop on Agent and Data Mining Interaction (ADMI -2014) Associated with International Conference on Autonomous Agents and Multi-agent Systems (AAMAS -2014), Paris (2014) Gorodetsky, V., Samoylov, V., Tushkanova, O.: Agent-based customer profile learning in 3G recommending systems. In: Proceedings of 9-th International Workshop on Agent and Data Mining Interaction (ADMI -2014) Associated with International Conference on Autonomous Agents and Multi-agent Systems (AAMAS -2014), Paris (2014)
10.
Zurück zum Zitat Gorodetsky, V., Samoylov, V., Serebryakov, S.: Context–Driven Data and Information Fusion. In: Proceedings of International Conference on Information Fusion (Fusion 2012), pp. 1830–1837, Singapore (2012) Gorodetsky, V., Samoylov, V., Serebryakov, S.: Context–Driven Data and Information Fusion. In: Proceedings of International Conference on Information Fusion (Fusion 2012), pp. 1830–1837, Singapore (2012)
11.
Zurück zum Zitat Gorodetsky, V., Samoylov, V., Serebryakov, S.: Ontology–based context–dependent personalization technology. In: Proceedings of WI/IAT/ACM International Conference, Associated Workshop “Web Personalization and Recommender Systems”, Toronto (2010) Gorodetsky, V., Samoylov, V., Serebryakov, S.: Ontology–based context–dependent personalization technology. In: Proceedings of WI/IAT/ACM International Conference, Associated Workshop “Web Personalization and Recommender Systems”, Toronto (2010)
12.
Zurück zum Zitat Gorodetsky, V., Serebryakov, S.: Methods and algorithms of collective recognition. Autom. Remote Control. 69(11), 1821–1851 (2008)CrossRefMathSciNet Gorodetsky, V., Serebryakov, S.: Methods and algorithms of collective recognition. Autom. Remote Control. 69(11), 1821–1851 (2008)CrossRefMathSciNet
13.
Zurück zum Zitat Hall, P., Pittelkow, Y., Ghosh, M.: Theoretical measures of relative performance of classifiers for high dimensional data with small sample sizes. J. Roy. Stat. Soc. Ser. B (Stat. Methodol.) 70(1), 159–173 (2008)CrossRefMATHMathSciNet Hall, P., Pittelkow, Y., Ghosh, M.: Theoretical measures of relative performance of classifiers for high dimensional data with small sample sizes. J. Roy. Stat. Soc. Ser. B (Stat. Methodol.) 70(1), 159–173 (2008)CrossRefMATHMathSciNet
18.
Zurück zum Zitat Kuncheva, L., Whitaker, C.: Measures of diversity in classifier ensembles. Mach. Learn. 51, 181–207 (2003)CrossRefMATH Kuncheva, L., Whitaker, C.: Measures of diversity in classifier ensembles. Mach. Learn. 51, 181–207 (2003)CrossRefMATH
19.
Zurück zum Zitat Li, J., Le, T.D., Liu, L., Liu, J., Jin, Z., Sun, B.: Mining causal association rules. In: Proceedings of International ICDM-2013 Workshop on Causal Discovery, Dallas, USA (2013) Li, J., Le, T.D., Liu, L., Liu, J., Jin, Z., Sun, B.: Mining causal association rules. In: Proceedings of International ICDM-2013 Workshop on Causal Discovery, Dallas, USA (2013)
21.
Zurück zum Zitat Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco (1991) Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco (1991)
22.
Zurück zum Zitat Pearl, J. and Verma, T.S.: A Theory of Inferred Causation. In: Proc. Second International Conference on the Principles of Knowledge Representation and Reasoning, pp. 441–452 (1991) Pearl, J. and Verma, T.S.: A Theory of Inferred Causation. In: Proc. Second International Conference on the Principles of Knowledge Representation and Reasoning, pp. 441–452 (1991)
23.
Zurück zum Zitat Silverstein, C., Brin, S., Motwani, R.: Scalable techniques for mining causal structures. In: Proceedings of 24th VLDB Conference, New York, USA, pp 594–605 (1998) Silverstein, C., Brin, S., Motwani, R.: Scalable techniques for mining causal structures. In: Proceedings of 24th VLDB Conference, New York, USA, pp 594–605 (1998)
24.
Zurück zum Zitat Skormin, V.A., Gorodetski, V.I., Popyack, L.J.: Data mining technology for failure prognostic of avionics. IEEE Trans. Aerosp. Electron. Syst. 38(2), 388–403 (2002)CrossRef Skormin, V.A., Gorodetski, V.I., Popyack, L.J.: Data mining technology for failure prognostic of avionics. IEEE Trans. Aerosp. Electron. Syst. 38(2), 388–403 (2002)CrossRef
Metadaten
Titel
Big Data: Opportunities, Challenges and Solutions
verfasst von
Vladimir Gorodetsky
Copyright-Jahr
2014
DOI
https://doi.org/10.1007/978-3-319-13206-8_1