Skip to main content

2017 | OriginalPaper | Buchkapitel

8. Acceleration of MapReduce Framework on a Multicore Processor

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

MapReduce framework is widely used in massive data processing, such as financial prediction, online marketing, and so on. Multicore processor is a great platform to implement MapReduce because of its inherent parallelism and flexibility. This book chapter extracts features of MapReduce applications, and proposes a software–hardware co-design framework based on a multi-core processor to improve the performance of MapReduce applications. Experimental results show that the MapReduce framework with hardware accelerators speeds up by 40 times at maximum compared to the pure software solution, and the proposed Topo-MapReduce speeds up further by 29% at maximum compared to the original MapReduce.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat M. Saecker, V. Markl, Big data analytics on modern hardware architectures: a technology survey, in Business Intelligence (Springer, Berlin/Heidelberg, 2013), pp. 125–149 M. Saecker, V. Markl, Big data analytics on modern hardware architectures: a technology survey, in Business Intelligence (Springer, Berlin/Heidelberg, 2013), pp. 125–149
2.
Zurück zum Zitat J. Dean, S. Ghemawat, MapReduce: simplified data processing on large clusters, in Proceedings of Operating Systems Design and Implementation, San Francisco, CA (2004), pp. 137–150 J. Dean, S. Ghemawat, MapReduce: simplified data processing on large clusters, in Proceedings of Operating Systems Design and Implementation, San Francisco, CA (2004), pp. 137–150
3.
Zurück zum Zitat L. Zhou, Z. Yu, J. Lin, S. Zhu, W. Shi, H. Zhou, K. Song, X. Zeng, Acceleration of Naive-Bayes algorithm on multicore processor for massive text classification, in International Symposium on Integrated Circuits (ISIC) (2014), pp. 244–247 L. Zhou, Z. Yu, J. Lin, S. Zhu, W. Shi, H. Zhou, K. Song, X. Zeng, Acceleration of Naive-Bayes algorithm on multicore processor for massive text classification, in International Symposium on Integrated Circuits (ISIC) (2014), pp. 244–247
5.
Zurück zum Zitat C. Ranger, R. Raghuraman, A. Penmetsa, et al., Evaluating MapReduce for multi-core and multiprocessor systems, in IEEE 13th International Symposium on High Performance Computer Architecture, 2007. HPCA 2007 (IEEE, Washington, 2007), pp. 13–24 C. Ranger, R. Raghuraman, A. Penmetsa, et al., Evaluating MapReduce for multi-core and multiprocessor systems, in IEEE 13th International Symposium on High Performance Computer Architecture, 2007. HPCA 2007 (IEEE, Washington, 2007), pp. 13–24
6.
Zurück zum Zitat Z. Yu, K. You, R. Xiao, H. Quan, P. Ou, Y. Ying, H. Yang, M. Jing, X. Zeng, An 800MHz 320mW 16-core processor with message-passing and shared-memory inter-core communication mechanisms, in IEEE International Solid-State Circuits Conference (ISSCC) (2012), pp. 64–65 Z. Yu, K. You, R. Xiao, H. Quan, P. Ou, Y. Ying, H. Yang, M. Jing, X. Zeng, An 800MHz 320mW 16-core processor with message-passing and shared-memory inter-core communication mechanisms, in IEEE International Solid-State Circuits Conference (ISSCC) (2012), pp. 64–65
7.
Zurück zum Zitat P. Ou, J. Zhang, H. Quan, Y. Li, M. He, Z. Yu, X. Yu, S. Cui, J. Feng, S. Zhu, J. Lin, M. Jing, X. Zeng, Z. Yu, A 65nm 39GOPS/W 24-core processor with 11Tb/s/W packet controlled circuit-switched double-layer network-on-chip and heterogeneous execution array, in IEEE International Solid-State Circuits Conference (ISSCC) (2013), pp. 56–57 P. Ou, J. Zhang, H. Quan, Y. Li, M. He, Z. Yu, X. Yu, S. Cui, J. Feng, S. Zhu, J. Lin, M. Jing, X. Zeng, Z. Yu, A 65nm 39GOPS/W 24-core processor with 11Tb/s/W packet controlled circuit-switched double-layer network-on-chip and heterogeneous execution array, in IEEE International Solid-State Circuits Conference (ISSCC) (2013), pp. 56–57
8.
Zurück zum Zitat V. Felipe, et al., GPU-NB: a fast CUDA-based implementation of Naive Bayes, in Proceedings - Symposium on Computer Architecture and High Performance Computing (2013), pp. 168–175 V. Felipe, et al., GPU-NB: a fast CUDA-based implementation of Naive Bayes, in Proceedings - Symposium on Computer Architecture and High Performance Computing (2013), pp. 168–175
Metadaten
Titel
Acceleration of MapReduce Framework on a Multicore Processor
verfasst von
Lijun Zhou
Zhiyi Yu
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-54840-1_8

Neuer Inhalt