Skip to main content
Top

2020 | OriginalPaper | Chapter

15. On Performance of Commodity Single Board Computer-Based Clusters: A Big Data Perspective

Authors : Basit Qureshi, Anis Koubaa

Published in: Smart Infrastructure and Applications

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In recent times, the commodity Single Board Computers (SBCs) have now become sufficiently powerful that they can run standard operating systems and mainstream workloads. In this chapter, we investigate the design and implementation of Single Board Computer (SBC)-based Hadoop clusters. We provide a compact design layout and build two clusters each using 20 nodes. We extensively test the performance of these clusters using popular performance benchmarks for task execution time, memory/storage utilization, network throughput, and energy consumption. We investigate the cost of operating SBC-based clusters by correlating energy utilization for the execution time of various benchmarks using workloads of different sizes. Although the low-cost benefit of a cluster built with ARM-based SBCs is desirable, these clusters yield low comparable performance and energy efficiency due to limited onboard capabilities. It is, however, possible to tweak Hadoop configuration parameters to ARM-based SBC specifications to efficiently utilize available resources. Finally, a discussion on the design implications of these clusters as a testbed for inexpensive and green cloud computing research is presented.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
2
Usingsysbench ---test=cpu ---cpu-max-prime=10000 ---num-threads= n run
 
Literature
2.
go back to reference Chaari, R., Ellouze, F., Koubaa, A., Qureshi, B., Pereira, N., Youssef, H., Tovar, E.: Cyber-physical systems clouds: a survey. Comput. Netw. 18, 260–278 (2016)CrossRef Chaari, R., Ellouze, F., Koubaa, A., Qureshi, B., Pereira, N., Youssef, H., Tovar, E.: Cyber-physical systems clouds: a survey. Comput. Netw. 18, 260–278 (2016)CrossRef
3.
go back to reference Gómez, A., Cuiñas, D., Catalá, P., Xin, L., Li, W., Conway, S., Lack, D.: Use of single board computers as smart sensors in the manufacturing industry. Proc. Eng. 132, 153–159 (2015)CrossRef Gómez, A., Cuiñas, D., Catalá, P., Xin, L., Li, W., Conway, S., Lack, D.: Use of single board computers as smart sensors in the manufacturing industry. Proc. Eng. 132, 153–159 (2015)CrossRef
4.
go back to reference Grigoriev, S.N., Martinov, G.M.: An ARM-based multi-channel CNC solution for multi-tasking turning and milling machines. Proc. CIRP. 46, 525–528 (2016)CrossRef Grigoriev, S.N., Martinov, G.M.: An ARM-based multi-channel CNC solution for multi-tasking turning and milling machines. Proc. CIRP. 46, 525–528 (2016)CrossRef
5.
go back to reference Fernandes, S.L., Bala, G.J.: ODROID XU4 based implementation of decision level fusion approach for matching computer-generated sketches. J. Comput. Sci. 16, 217–224 (2016)CrossRef Fernandes, S.L., Bala, G.J.: ODROID XU4 based implementation of decision level fusion approach for matching computer-generated sketches. J. Comput. Sci. 16, 217–224 (2016)CrossRef
6.
go back to reference Grisenthwaite, R.: ARMv8-A Technology Preview. Online. Accessed 31 Oct 2011 Grisenthwaite, R.: ARMv8-A Technology Preview. Online. Accessed 31 Oct 2011
8.
go back to reference Abrahamsson, P., Helmer, S., Phaphoom, N., Nicolodi, L., Preda, N., Miori, L., Angriman, M., Rikkila, J., Wang, X., Hamily, K., Bugoloni, S.: Affordable and energy-efficient cloud computing clusters: the Bolzano raspberry pi cloud cluster experiment. In: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom), vol. 2, pp. 170–175 (2013) Abrahamsson, P., Helmer, S., Phaphoom, N., Nicolodi, L., Preda, N., Miori, L., Angriman, M., Rikkila, J., Wang, X., Hamily, K., Bugoloni, S.: Affordable and energy-efficient cloud computing clusters: the Bolzano raspberry pi cloud cluster experiment. In: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom), vol. 2, pp. 170–175 (2013)
9.
go back to reference Cox, S.J., Cox, J.T., Boardman, R.P., Johnston, S.J., Scott, M., O’Brien, N.S.: Iridis-pi: a low-cost, compact demonstration cluster. Clust. Comput. 17(2), 349–358 (2014)CrossRef Cox, S.J., Cox, J.T., Boardman, R.P., Johnston, S.J., Scott, M., O’Brien, N.S.: Iridis-pi: a low-cost, compact demonstration cluster. Clust. Comput. 17(2), 349–358 (2014)CrossRef
10.
go back to reference Tso, F.P., White, D.R., Jouet, S., Singer, J., Pezaros, D.P.: The Glasgow raspberry pi cloud: a scale model for cloud computing infrastructures. In: 2013 IEEE 33rd International Conference on Distributed Computing Systems Workshops (ICDCSW), pp. 108–112. IEEE, Philadelphia, PA (2013) Tso, F.P., White, D.R., Jouet, S., Singer, J., Pezaros, D.P.: The Glasgow raspberry pi cloud: a scale model for cloud computing infrastructures. In: 2013 IEEE 33rd International Conference on Distributed Computing Systems Workshops (ICDCSW), pp. 108–112. IEEE, Philadelphia, PA (2013)
14.
go back to reference Baun, C.: Mobile clusters of single board computers: an option for providing resources to student projects and researchers. SpringerPlus. 5, 360 (2016). PMC. Web. 16 Aug 2016CrossRef Baun, C.: Mobile clusters of single board computers: an option for providing resources to student projects and researchers. SpringerPlus. 5, 360 (2016). PMC. Web. 16 Aug 2016CrossRef
15.
go back to reference Huang, S., Huang, J., Dai, J., Xie, T., Huang, B.: The HiBench benchmark suite: characterization of the MapReduce-based data analysis. In: International Conference on Data Engineering Workshops (ICDEW), March 2010 Huang, S., Huang, J., Dai, J., Xie, T., Huang, B.: The HiBench benchmark suite: characterization of the MapReduce-based data analysis. In: International Conference on Data Engineering Workshops (ICDEW), March 2010
16.
go back to reference Ivanov, T., Niemann, R., Izberovic, S., Rosselli, M., Tolle, K., Zicari, R.V.: Performance evaluation of enterprise big data platforms with HiBench. In: IEEE Trustcom/BigDataSE/ISPA, Helsinki (2015) Ivanov, T., Niemann, R., Izberovic, S., Rosselli, M., Tolle, K., Zicari, R.V.: Performance evaluation of enterprise big data platforms with HiBench. In: IEEE Trustcom/BigDataSE/ISPA, Helsinki (2015)
17.
go back to reference Ge, R., Feng, X., Song, S., Chang, H.C., Li, D., Cameron, K.W.: PowerPack: energy profiling and analysis of high-performance systems and applications. IEEE Trans. Parallel Distrib. Syst. 21(5), 658–671 (2010)CrossRef Ge, R., Feng, X., Song, S., Chang, H.C., Li, D., Cameron, K.W.: PowerPack: energy profiling and analysis of high-performance systems and applications. IEEE Trans. Parallel Distrib. Syst. 21(5), 658–671 (2010)CrossRef
18.
go back to reference Cloutier, M.F., Paradis, C., Weaver, V.M.: Design and analysis of a 32-bit embedded high-performance cluster optimized for energy and performance. In: Hardware-Software Co-Design for High-Performance Computing (Co-HPC), 2014, New Orleans, LA (2014) Cloutier, M.F., Paradis, C., Weaver, V.M.: Design and analysis of a 32-bit embedded high-performance cluster optimized for energy and performance. In: Hardware-Software Co-Design for High-Performance Computing (Co-HPC), 2014, New Orleans, LA (2014)
19.
go back to reference Psaroudakis, I., Kissinger, T., Porobic, D., Ilsche, T., Liarou, E., Tözün, P., Ailamaki, A., Lehner, W.: Dynamic fine-grained scheduling for energy-efficient main-memory queries. In: The Tenth International Workshop on Data Management on New Hardware (DaMoN’14) (2014) Psaroudakis, I., Kissinger, T., Porobic, D., Ilsche, T., Liarou, E., Tözün, P., Ailamaki, A., Lehner, W.: Dynamic fine-grained scheduling for energy-efficient main-memory queries. In: The Tenth International Workshop on Data Management on New Hardware (DaMoN’14) (2014)
20.
go back to reference Schöne, R., Treibig, J., Dolz, M.F., et al.: Tools and methods for measuring and tuning the energy efficiency of HPC systems. J. Sci. Program. 22(4), 273–283 (2014) Schöne, R., Treibig, J., Dolz, M.F., et al.: Tools and methods for measuring and tuning the energy efficiency of HPC systems. J. Sci. Program. 22(4), 273–283 (2014)
21.
go back to reference Piga, L., Bergamaschi, R.A., Breternitz, M., Rigo, S.: Adaptive global power optimization for Web servers. J. Supercomput. 68, 1088 (2014)CrossRef Piga, L., Bergamaschi, R.A., Breternitz, M., Rigo, S.: Adaptive global power optimization for Web servers. J. Supercomput. 68, 1088 (2014)CrossRef
22.
go back to reference Ilsche, T., Hackenberg, D., Graul, S., Schöne, R., Schuchart, J.: Power measurements for compute nodes: improving sampling rates, granularity and accuracy. In: Sixth International Green Computing Conference and Sustainable Computing Conference (IGSC), Las Vegas, NV (2015) Ilsche, T., Hackenberg, D., Graul, S., Schöne, R., Schuchart, J.: Power measurements for compute nodes: improving sampling rates, granularity and accuracy. In: Sixth International Green Computing Conference and Sustainable Computing Conference (IGSC), Las Vegas, NV (2015)
23.
go back to reference Hackenberg, D., Ilsche, T., Schöne, R., Molka, D., Schmidt, M., Nagel, W.E.: Power measurement techniques on standard compute nodes: a quantitative comparison. In: 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), Austin, TX, pp. 194–204 (2013) Hackenberg, D., Ilsche, T., Schöne, R., Molka, D., Schmidt, M., Nagel, W.E.: Power measurement techniques on standard compute nodes: a quantitative comparison. In: 2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), Austin, TX, pp. 194–204 (2013)
24.
go back to reference Divakaran, D., Le, T., Gurusamy, M.: An online integrated resource allocator for guaranteed performance in data centers. IEEE Trans. Parallel Distrib. Syst. 25(6), 1382–1392 (2014)CrossRef Divakaran, D., Le, T., Gurusamy, M.: An online integrated resource allocator for guaranteed performance in data centers. IEEE Trans. Parallel Distrib. Syst. 25(6), 1382–1392 (2014)CrossRef
25.
go back to reference Dalvandi, A., Gurusamy, M., Chua, K.C.: Time-Aware VMFlow Placement, Routing, and Migration for Power Efficiency in Data Centers. IEEE Trans. Netw. Serv. Manag. 12(3), 349–362 (2015)CrossRef Dalvandi, A., Gurusamy, M., Chua, K.C.: Time-Aware VMFlow Placement, Routing, and Migration for Power Efficiency in Data Centers. IEEE Trans. Netw. Serv. Manag. 12(3), 349–362 (2015)CrossRef
26.
go back to reference Cook, G.: How clean is your cloud? Catalysing an energy revolution. Greenpeace International, Amsterdam (2012) Cook, G.: How clean is your cloud? Catalysing an energy revolution. Greenpeace International, Amsterdam (2012)
28.
go back to reference Xiao, Z., Song, W., Chen, Q.: Dynamic resource allocation using virtual machines for cloud computing environment. IEEE Trans. Parallel Distrib. Syst. 24(6), 1107–1117 (2013)CrossRef Xiao, Z., Song, W., Chen, Q.: Dynamic resource allocation using virtual machines for cloud computing environment. IEEE Trans. Parallel Distrib. Syst. 24(6), 1107–1117 (2013)CrossRef
29.
go back to reference Qureshi, B., Koubaa, A.: Power efficiency of a SBC based Hadoop cluster. In: Proceedings of EAI Smart Societies, Infrastructure, Technologies and Applications (SCITA 2017), Jeddah, Saudi Arabia, pp. 52–60, 27–29 Nov 2017 Qureshi, B., Koubaa, A.: Power efficiency of a SBC based Hadoop cluster. In: Proceedings of EAI Smart Societies, Infrastructure, Technologies and Applications (SCITA 2017), Jeddah, Saudi Arabia, pp. 52–60, 27–29 Nov 2017
30.
go back to reference Conejero, J., Rana, O., Burnap, P., Morgan, J., Caminero, B., Carrión, C.: Analyzing Hadoop power consumption and impact on application QoS. Futur. Gener. Comput. Syst. 55, 213–223 (2016)CrossRef Conejero, J., Rana, O., Burnap, P., Morgan, J., Caminero, B., Carrión, C.: Analyzing Hadoop power consumption and impact on application QoS. Futur. Gener. Comput. Syst. 55, 213–223 (2016)CrossRef
Metadata
Title
On Performance of Commodity Single Board Computer-Based Clusters: A Big Data Perspective
Authors
Basit Qureshi
Anis Koubaa
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-030-13705-2_15