Skip to main content
Top

2019 | OriginalPaper | Chapter

HPC Technologies from Scientific Computing to Big Data Applications

Authors : L. M. Patnaik, Srinidhi Hiriyannaiah

Published in: Advances in Mathematical Methods and High Performance Computing

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Recent advances in information technology and its widespread growth in the areas of business, engineering, medical and scientific studies have resulted in information explosion. High performance computing (HPC) systems are essential for solving scientific problems involving massive data using high computation power and high throughput networks. Due to the widespread growth of data in various fields, knowledge discovery and decision making is a challenging task and has resulted in the emerging trend of Big Data analytics. Big data is related to complex, diverse and massive data sets comprising of structured, semi-structured and unstructured data. Such data cannot be processed and analysed with the traditional database technologies. HPC systems can be extended to Big data applications for large-scale processing and analysis thus shifting the paradigm from traditional scientific computing domain to data intensive domain or Big data. The aim of this paper is to present an overview of the evolution and principles starting from scientific computing to present Big Data analytics.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990). Basic local alignment search tool. Journal of molecular biology, 215(3), 403–410.CrossRef Altschul, S. F., Gish, W., Miller, W., Myers, E. W., & Lipman, D. J. (1990). Basic local alignment search tool. Journal of molecular biology, 215(3), 403–410.CrossRef
2.
go back to reference Ekanayake, J., Pallickara, S., & Fox, G. (2008). Mapreduce for data intensive scientific analyses. In eScience, 2008. eScience’08. IEEE Fourth International Conference on (pp. 277–284). IEEE. Ekanayake, J., Pallickara, S., & Fox, G. (2008). Mapreduce for data intensive scientific analyses. In eScience, 2008. eScience’08. IEEE Fourth International Conference on (pp. 277–284). IEEE.
4.
go back to reference Gropp, W., Lusk, E., Doss, N., & Skjellum, A. (1996). A high-performance, portable implementation of the MPI message passing interface standard. Parallel computing, 22(6), 789–828.CrossRef Gropp, W., Lusk, E., Doss, N., & Skjellum, A. (1996). A high-performance, portable implementation of the MPI message passing interface standard. Parallel computing, 22(6), 789–828.CrossRef
5.
go back to reference Litzkow, M. J., Livny, M., & Mutka, M. W. (1988, June). Condor-a hunter of idle workstations. In Distributed Computing Systems, 1988., 8th International Conference on (pp. 104–111). IEEE. Litzkow, M. J., Livny, M., & Mutka, M. W. (1988, June). Condor-a hunter of idle workstations. In Distributed Computing Systems, 1988., 8th International Conference on (pp. 104–111). IEEE.
6.
go back to reference Thain, D., Tannenbaum, T., & Livny, M. (2005). Distributed computing in practice: the Condor experience. Concurrency and computation: practice and experience, 17(24), 323–356.CrossRef Thain, D., Tannenbaum, T., & Livny, M. (2005). Distributed computing in practice: the Condor experience. Concurrency and computation: practice and experience, 17(24), 323–356.CrossRef
7.
go back to reference Foster, I., & Kesselman, C. (Eds.). (2003). The Grid 2: Blueprint for a new computing infrastructure. Elsevier. Foster, I., & Kesselman, C. (Eds.). (2003). The Grid 2: Blueprint for a new computing infrastructure. Elsevier.
8.
go back to reference Chetty, M., & Buyya, R. (2002). Weaving computational Grids: How analogous are they with electrical Grids?. Computing in Science & Engineering, 4(4), 61–71.CrossRef Chetty, M., & Buyya, R. (2002). Weaving computational Grids: How analogous are they with electrical Grids?. Computing in Science & Engineering, 4(4), 61–71.CrossRef
9.
go back to reference Chin, J., Harvey, M. J., Jha, S., & Coveney, P. V. (2005). Scientific grid computing: The first generation. Computing in science & engineering, 7(5), 24–32.CrossRef Chin, J., Harvey, M. J., Jha, S., & Coveney, P. V. (2005). Scientific grid computing: The first generation. Computing in science & engineering, 7(5), 24–32.CrossRef
10.
go back to reference Vecchiola, C., Chu, X., & Buyya, R. (2009). Aneka: a software platform for .NET-based cloud computing. High Speed and Large Scale Scientific Computing, 18, 267–295. Vecchiola, C., Chu, X., & Buyya, R. (2009). Aneka: a software platform for .NET-based cloud computing. High Speed and Large Scale Scientific Computing, 18, 267–295.
11.
go back to reference Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., Konwinski, A., …& Zaharia, M. (2009). Above the clouds: A Berkeley view of cloud computing (Vol. 17). Technical Report UCB/EECS-2009-28, EECS Department, University of California, Berkeley. Armbrust, M., Fox, A., Griffith, R., Joseph, A. D., Katz, R. H., Konwinski, A., …& Zaharia, M. (2009). Above the clouds: A Berkeley view of cloud computing (Vol. 17). Technical Report UCB/EECS-2009-28, EECS Department, University of California, Berkeley.
12.
go back to reference Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., & Brandic, I. (2009). Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation computer systems, 25(6), 599–616.CrossRef Buyya, R., Yeo, C. S., Venugopal, S., Broberg, J., & Brandic, I. (2009). Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation computer systems, 25(6), 599–616.CrossRef
13.
go back to reference Dean, J., & Ghemawat, S. (2008). MapReduce: simplified data processing on large clusters. Communications of the ACM, 51(1), 107–113.CrossRef Dean, J., & Ghemawat, S. (2008). MapReduce: simplified data processing on large clusters. Communications of the ACM, 51(1), 107–113.CrossRef
14.
go back to reference White, T. (2012). Hadoop: The definitive guide.“ O’Reilly Media, Inc”. White, T. (2012). Hadoop: The definitive guide.“ O’Reilly Media, Inc”.
16.
go back to reference John Walker, S. (2014). Big data: A revolution that will transform how we live, work, and think. John Walker, S. (2014). Big data: A revolution that will transform how we live, work, and think.
17.
go back to reference Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009). Detecting influenza epidemics using search engine query data. Nature, 457(7232), 1012–1014.CrossRef Ginsberg, J., Mohebbi, M. H., Patel, R. S., Brammer, L., Smolinski, M. S., & Brilliant, L. (2009). Detecting influenza epidemics using search engine query data. Nature, 457(7232), 1012–1014.CrossRef
19.
go back to reference USING, M. (1994). Portable parallel programming with the Message-Passing Interface. USING, M. (1994). Portable parallel programming with the Message-Passing Interface.
20.
go back to reference Kirk, D. B., & Wen-Mei, W. H. (2016). Programming massively parallel processors: a hands-on approach. Morgan kaufmann. Kirk, D. B., & Wen-Mei, W. H. (2016). Programming massively parallel processors: a hands-on approach. Morgan kaufmann.
21.
go back to reference Tinetti, F. G. (2010). Using OpenMP: Portable Shared Memory Parallel Programming. Journal of Computer Science & Technology, 10. Tinetti, F. G. (2010). Using OpenMP: Portable Shared Memory Parallel Programming. Journal of Computer Science & Technology, 10.
22.
go back to reference Cronin, M. J., & Seid, G. (1983). U.S. Patent No. 4,419,926. Washington, DC: U.S. Patent and Trademark Office. Cronin, M. J., & Seid, G. (1983). U.S. Patent No. 4,419,926. Washington, DC: U.S. Patent and Trademark Office.
23.
go back to reference Patnaik, L. M., & Hiriyannaiah, S. (2017). Business Analytics Using Recommendation Systems. In International Conference on Computational Intelligence, Communications, and Business Analytics (pp. 35–44). Springer, Singapore.CrossRef Patnaik, L. M., & Hiriyannaiah, S. (2017). Business Analytics Using Recommendation Systems. In International Conference on Computational Intelligence, Communications, and Business Analytics (pp. 35–44). Springer, Singapore.CrossRef
Metadata
Title
HPC Technologies from Scientific Computing to Big Data Applications
Authors
L. M. Patnaik
Srinidhi Hiriyannaiah
Copyright Year
2019
DOI
https://doi.org/10.1007/978-3-030-02487-1_19

Premium Partner