Skip to main content
Top

2017 | OriginalPaper | Chapter

Beyond Map-Reduce: LATNODE – A New Programming Paradigm for Big Data Systems

Authors : Chai Yit Sheng, Phang Keat Keong

Published in: Information Science and Applications 2017

Publisher: Springer Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The Compute Aggregate model used to model Map Reduce does not allow for dynamic node reordering once a job has started, assumes homogenous nodes and a balanced tree layout. We introduce heterogeneous nodes into the tree structure, thereby causing unbalanced trees. Finally, we present a new programming abstraction to allow for dynamic tree balancing.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Communications of the ACM 51, 107–113 (2008)CrossRef Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Communications of the ACM 51, 107–113 (2008)CrossRef
2.
go back to reference Konstantin, S., Hairong, K., Sanjay, R., Robert, C.: The Hadoop distributed file system. In: Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) (2010) Konstantin, S., Hairong, K., Sanjay, R., Robert, C.: The Hadoop distributed file system. In: Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST) (2010)
3.
go back to reference Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., et al.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, San Jose, CA (2012) Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., et al.: Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In: Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, San Jose, CA (2012)
4.
go back to reference Culhane, W., Kogan, K., Jayalath, C., Eugster, P.: Optimal communication structures for big data aggregation. In: 2015 IEEE Conference on Computer Communications (INFOCOM), pp. 1643–1651 (2015) Culhane, W., Kogan, K., Jayalath, C., Eugster, P.: Optimal communication structures for big data aggregation. In: 2015 IEEE Conference on Computer Communications (INFOCOM), pp. 1643–1651 (2015)
5.
go back to reference Cheng, Y.C., Robertazzi, T.G.: Distributed computation for a tree network with communication delays. IEEE Transactions on Aerospace and Electronic Systems 26, 511–516 (1990)CrossRef Cheng, Y.C., Robertazzi, T.G.: Distributed computation for a tree network with communication delays. IEEE Transactions on Aerospace and Electronic Systems 26, 511–516 (1990)CrossRef
6.
go back to reference Hyoung Joong, K., Gyu-in, J., Jang Gyu, L.: Optimal load distribution for tree network processors. IEEE Transactions on Aerospace and Electronic Systems 32, 607–612 (1996)CrossRef Hyoung Joong, K., Gyu-in, J., Jang Gyu, L.: Optimal load distribution for tree network processors. IEEE Transactions on Aerospace and Electronic Systems 32, 607–612 (1996)CrossRef
7.
go back to reference Morozov, D., Weber, G.: Distributed merge trees. SIGPLAN Not. 48, 93–102 (2013)CrossRef Morozov, D., Weber, G.: Distributed merge trees. SIGPLAN Not. 48, 93–102 (2013)CrossRef
Metadata
Title
Beyond Map-Reduce: LATNODE – A New Programming Paradigm for Big Data Systems
Authors
Chai Yit Sheng
Phang Keat Keong
Copyright Year
2017
Publisher
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-4154-9_9