Top

Published in:

2016 | OriginalPaper | Chapter

Harmony Search to Self-Configuration of Fault-Tolerant Grids for Big Data

Authors : Jerzy Balicki, Waldemar Korłub, Maciej Tyszka

Published in: Advanced and Intelligent Computations in Diagnosis and Control

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In this paper, harmony search algorithms have been proposed to self-configuration of fault-tolerant grids for big data processing. Some tasks related to big data processing have been considered. Moreover, two criteria have been applied to evaluate quality of grids. The first criterion is a probability that all tasks meet their deadlines and the second one is grid reliability. Furthermore, some intelligent agents based on harmony search have been developed to support a middleware layer of grids.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Automatic Graph-Based Local Edge Detection

Afshari, S., Aminshahidy, B., Pishvaie, M.R.: Application of an improved harmony search algorithm in well placement optimization using streamline simulation. J. Petrol. Sci. Eng. 78, 664–678 (2011)CrossRef

Ahmed, A.M., Bakar, A.A. Hamdan, A.R.: Harmony search algorithm for optimal word size in symbolic time series representation. In: Proceedings of Conferences on Data Mining and Optimization, Malaysia, pp. 57–62 (2011)

Ajith, A.P., Murthy, C.S.R.: Algorithms for reliability-oriented module allocation in distributed computing systems. J. Syst. Softw. 40, 125–138 (1998)CrossRef

Al-Betar, M.A., Khader, A.T., Zaman, M.: University course timetabling using a hybrid harmony search metaheuristic algorithm. IEEE Trans. Syst. Man Cybern: Part C: Appl. Rev 42, 66–681 (2012)CrossRef

Apache Hadoop, http://hadoop.apache.org/, Accessed (2015)

Balicki, J.: Negative selection with ranking procedure in tabu-based multi-criterion evolutionary algorithm for task assignment. In: Alexandrov, V.N., et al. (eds.) Proceedings the 6th International Conference on Computational Science. Lecture Notes in Computer Science, 3993, pp. 863–870. Reading, England (2006)

Balicki, J.: Multi-criterion optimisation of distributed system performance by evolutionary task assignments. J. Res. Pract. Inf. Technol. 33, 173–185 (2001)

Balicki, J.: An adaptive quantum-based multiobjective evolutionary algorithm for efficient task assignment in distributed systems. In: Mastorakis, N. et al. (eds.) Recent Advances in Computer Engineering. Proceedings of the 13th WSEAS International Conferences on Computers, pp. 417–422, Rhodes, Greece (2009)

Balicki, J.: Genetic Programming with negative selection for volunteer computing system optimization, In: Paja, W.A., Wilamowski, B.M (eds.) Proceedings on the 6th International Conferences on Human System Interactions, pp. 271–278, Gdansk, Poland (2013)

10.

Balicki, J., Korłub, W., Szymańsk, J., Zakidalski, M.: Big data paradigm developed in volunteer grid system with genetic programming scheduler. In: Rutkowski, L. et al, (eds.) Artificial Intelligence and Soft Computing. Lecture Notes in Computer Science, 8467. Proceedings of the 13th International Conferences on Artificial Intelligence and Soft Computing ICAISC, Part II, pp. 771–782, Zakopane, Poland (2014)

11.

Cao, L., Gorodetsky, V., Mitkas, P.A.: Agent mining: the synergy of agents and data mining. IEEE Intell. Syst. 24, 64–72 (2009)CrossRef

12.

Comcute, http://comcute.eti.pg.gda.pl/. Accessed (2015)

13.

Geem, Z.W., Kim, J.H., Loganathan, G.V.: A new heuristic optimization algorithm: harmony search. Simulation 76, 60–68 (2001)CrossRef

14.

Gunarathne, T. et al.: Cloud computing paradigms for pleasingly parallel biomedical applications. In: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, pp. 460–469, Chicago, Illinois (2010)

15.

Guojun, L., Ming, Z., Fei, Y.: Large-scale social network analysis based on MapReduce. In: Proceedings International Conference on Computational Aspects of Social Networks, pp. 487–490 (2010)

16.

Huang, Z., Wang, C., Liu, L., Peng, Y.: Improve availability of fault-tolerant computing: optimal multi-task allocation in MapReduce. In: Proceedings on 7th International Conference on Computer Science & Education, pp. 249–254 (2012)

17.

Jennings, N.R., Wooldridge, M.: Applications of intelligent agents. In: Jennings, N.R., Wooldridge, M. (eds.) Intelligent Agents, pp. 3–28. Springer, New York (1998)

18.

Kafil, M., Ahmad, I.: Optimal task assignment in heterogeneous distributed computing systems. IEEE Concurrency 6, 42–51 (1998)CrossRef

19.

Kartik, S., Murthy, C.S.R.: Task allocation algorithms for maximizing reliability of distributed computing systems. IEEE Trans. Comput. 46, 719–724 (1997)MATHCrossRef

20.

Leyton-Brown, K., Shoham, Y.: Multiagent Systems: Algorithmic, Game-theoretic, and Logical Foundations. Cambridge University Press, UK (2008)

21.

Li, H.X., Chosler, R.: Application of multilayered multi-agent data mining architecture to bank domain. In: Proceedings the International Conferences on Wireless Communications, Networking and Mobile Computing, pp. 6721–6724 (2007)

22.

Manjarres, D., et al.: A Survey on Applications of the Harmony Search Algorithm. Eng. Appl. Artif. Intell. 26, 1818–1831 (2013)CrossRef

23.

Mardani, S., Akbari, M.K., Sharifian, S.: Fraud detection in process aware information systems using MapReduce. In: Proceedings on Information and Knowledge Technology, pp. 88–91 (2014)

24.

Marz, N., Warren, J.: Big Data – Principles and Best Practices of Scalable Realtime Data Systems. Manning Pub. Co., USA (2014)

25.

O’Leary, D.E.: Artificial intelligence and big data. IEEE Intell. Syst. 28, 96–99 (2013)CrossRef

26.

Ostrowski, D.A.: MapReduce design patterns for social networking analysis. In: Proceedings International Conference on Semantic Computing, pp. 316–319 (2014)

27.

Paluszak, J.: Optimizing the Use of Resources in Distributed Systems with Grid Architecture, (Ph.D. Dissertation). Gdańsk University of Technology, Gdańsk, Poland (2015)

28.

Qiu, X. et al.: Using MapReduce technologies in bioinformatics and medical informatics. In: Proceedings International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 22–30, Portland (2009)

29.

Sarvari, H., Zamanifar, K.: A Self-adaptive harmony search algorithm for engineering and reliability problems. In: Second International Conferences on Computer Intelligence, Modelling and Simulation, pp. 59–64 (2010)

30.

Schneidewind, N.: Allocation and Analysis of Reliability: Multiple Levels: System, Subsystem, and Module, Innovations in System and Software Engineering, 2, pp. 121–136. Springer, London (2006)

31.

Shatz, S.M., Wang, J.P.: Models & algorithms for reliability-oriented task-allocation in redundant distributed-computer systems. IEEE Trans. Reliab. 38, 16–27 (1989)CrossRef

32.

Shvachko, K. et al.: The hadoop distributed file system. In: The 26 Symposium on Mass Storage System and Technology, pp. 1–10 (2010)

33.

Snijders, C., Matzat, U., Reips, U.D.: ‘Big Data’: big gaps of knowledge in the field of internet. Int. J. Internet Sci. 7, 1–5 (2010)

34.

Shwe, T., Win, A.: A fault tolerant approach in cluster computing system. In: The 5th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology, 1, pp. 149–152 (2010)

35.

Twardowski, B., Ryzko, D.: Multi-agent architecture for real-time big data processing. In: Proceedings International Conferences on Web Intelligence and Intelligent Agent Technologies 3, pp. 333–337 (2014)

36.

Varvarigou, T., Trotter. J.: Module replication for fault-tolerant real-time distributed systems. IEEE Trans. Reliab. 47, 8–18 (1998)

37.

Vavilapalli, V.K.: Apache hadoop yarn: yet another resource negotiator. In: Proceedings of the 4th Annual Symposium on Cloud Computing, pp. 5:1–5:16, New York, USA (2013)

38.

Verbrugge, T., Dunin-Kęplicz, B.: Teamwork in Multi-agent Systems. A Formal Approach. Wiley (2010)

39.

Wang, L., Li. L.P.: A coevolutionary differential evolution with harmony search for reliability-redundancy optimization. Expert Syst. Appl. 39, 5271–5278 (2012)

40.

Węglarz, J., Błażewicz, J., Kovalyov, M.: Preemptable malleable task scheduling problem. IEEE Trans. Comput. 55, 486–490 (2006)CrossRef

41.

Wooldridge, M: Introduction to Multiagent Systems. Wiley, (2002)

42.

Zhou, D. et al.: Multi-agent distributed data mining model based on algorithm analysis and task prediction. In: Proceedings 2nd International Conferences on Information Engineering and Computer Science pp. 1–4 (2010)

43.

Zou, D., et al.: A Novel Global Harmony Search Algorithm for Reliability Problems. Comput. Ind. Eng. 58, 307–316 (2010)CrossRef

Title: Harmony Search to Self-Configuration of Fault-Tolerant Grids for Big Data
Authors: Jerzy Balicki
Waldemar Korłub
Maciej Tyszka
Publisher: Springer International Publishing
Book: Advanced and Intelligent Computations in Diagnosis and Control
Print ISBN: 978-3-319-23179-2

Electronic ISBN: 978-3-319-23180-8

Copyright Year: 2016
DOI: https://doi.org/10.1007/978-3-319-23180-8_30

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner