Skip to main content
Top
Published in: The Journal of Supercomputing 4/2016

01-04-2016

A self-organized volunteer Cloud for e-Science

Authors: Walid Saad, Heithem Abbes, Christophe Cérin, Mohamed Jemni

Published in: The Journal of Supercomputing | Issue 4/2016

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Nowadays, the adoption of Cloud Computing platforms and Service Computing technologies are almost natural for the different e-Science communities. Cost benefits for data-intensive applications, ease of access, rich and varied offers for services are examples of positive returns by users. However, beyond this favorable welcome for the technology, some research problems remain and are still challenging. In this paper, we focus on the problems of automatically deploying IaaS for computing and for data management, using the SlapOS Cloud. The core of the system is a distributed protocol for orchestrating data and compute nodes. Using this interaction scheme, users are able to deploy, without any system administrator intervention, a PaaS inside the IaaS basically a Desktop Grid middleware. The aim of this paper is to demonstrate that the Desktop Grid and Cloud paradigms may merge and may be widely used by non-experts in the different areas of e-Science. We propose a fully self-organized volunteer Cloud for researchers where they can carry out e-Science experiments and process large amounts of data in a coherent way.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Abbes H, Cérin C, Jemni M (2009) Bonjourgrid: orchestration of multi-instances of grid middlewares on institutional desktop grids. In: 23rd IEEE international symposium on parallel and distributed processing, IPDPS, pp 1–8 Abbes H, Cérin C, Jemni M (2009) Bonjourgrid: orchestration of multi-instances of grid middlewares on institutional desktop grids. In: 23rd IEEE international symposium on parallel and distributed processing, IPDPS, pp 1–8
2.
go back to reference Abbes H, Cérin C, Jemni M, Saad W (2010) Fault tolerance based on the publish-subscribe paradigm for the bonjourgrid middleware. In: Proceedings of the 11th IEEE/ACM international conference on grid computing GRID, pp 57–64 Abbes H, Cérin C, Jemni M, Saad W (2010) Fault tolerance based on the publish-subscribe paradigm for the bonjourgrid middleware. In: Proceedings of the 11th IEEE/ACM international conference on grid computing GRID, pp 57–64
3.
go back to reference Abidi L, Cérin C, Geldwerth-Feniger D, Lafaille M (2015) Cloud computing for e-sciences at université sorbonne paris cité. In: Cloudway workshop, service oriented and cloud computing—4th European conference, ESOCC 2015, Taormina, Italy, 15–17 September 2015, Proceedings of Springer CCIS series. vol 567 Abidi L, Cérin C, Geldwerth-Feniger D, Lafaille M (2015) Cloud computing for e-sciences at université sorbonne paris cité. In: Cloudway workshop, service oriented and cloud computing—4th European conference, ESOCC 2015, Taormina, Italy, 15–17 September 2015, Proceedings of Springer CCIS series. vol 567
4.
go back to reference Anderson DP (2004) BOINC: a system for public-resource computing and storage. In: 5th international workshop on grid computing GRID, pp 4–10 Anderson DP (2004) BOINC: a system for public-resource computing and storage. In: 5th international workshop on grid computing GRID, pp 4–10
5.
go back to reference Antoniu G, Costan A, Bigot J, Desprez F, Fedak G, Gault S, Pérez C, Simonet A, Tang B, Blanchet C, Terreux R, Bougé L, Briant F, Cappello F, Keahey K, Nicolae B, Suter F (2013) Scalable data management for map-reduce-based data-intensive applications: a view for cloud and hybrid infrastructures. IJCC 2(2/3):150–170CrossRef Antoniu G, Costan A, Bigot J, Desprez F, Fedak G, Gault S, Pérez C, Simonet A, Tang B, Blanchet C, Terreux R, Bougé L, Briant F, Cappello F, Keahey K, Nicolae B, Suter F (2013) Scalable data management for map-reduce-based data-intensive applications: a view for cloud and hybrid infrastructures. IJCC 2(2/3):150–170CrossRef
6.
go back to reference Butt AR, Zhang R, Hu YC (2006) A self-organizing flock of condors. J Parallel Distrib Comput 66(1):145–161CrossRefMATH Butt AR, Zhang R, Hu YC (2006) A self-organizing flock of condors. J Parallel Distrib Comput 66(1):145–161CrossRefMATH
7.
go back to reference Chlumsky V, Klusácek D, Ruda M (2012) The extension of torque scheduler allowing the use of planning and optimization in grids. Comput Sci (AGH) 13(2):5–20CrossRef Chlumsky V, Klusácek D, Ruda M (2012) The extension of torque scheduler allowing the use of planning and optimization in grids. Comput Sci (AGH) 13(2):5–20CrossRef
8.
go back to reference Fedak G, He H, Cappello F (2009) BitDew: a data management and distribution service with multi-protocol file transfer and metadata abstraction. J Netw Comput Appl 32(5):961–975CrossRef Fedak G, He H, Cappello F (2009) BitDew: a data management and distribution service with multi-protocol file transfer and metadata abstraction. J Netw Comput Appl 32(5):961–975CrossRef
9.
go back to reference Foster I (2011) Globus online: accelerating and democratizing science through cloud-based services. IEEE Internet Comput 15(3):70–73CrossRef Foster I (2011) Globus online: accelerating and democratizing science through cloud-based services. IEEE Internet Comput 15(3):70–73CrossRef
10.
go back to reference Fox G, von Laszewski G, Diaz J, Keahey K, Fortes J, Figueiredo R, Smallen S, Smith W, Grimshaw A (2013) Futuregrid: a reconfigurable testbed for cloud, hpc, and grid computing. In: Contemporary high performance computing: from petascale toward exascale, CRC computational science. Chapman & Hall Fox G, von Laszewski G, Diaz J, Keahey K, Fortes J, Figueiredo R, Smallen S, Smith W, Grimshaw A (2013) Futuregrid: a reconfigurable testbed for cloud, hpc, and grid computing. In: Contemporary high performance computing: from petascale toward exascale, CRC computational science. Chapman & Hall
12.
go back to reference Grossman RL, Greenway M, Heath AP, Powell R, Suarez RD, Wells W, White KP, Atkinson MP, Klampanos IA, Alvarez HL, Harvey C, Mambretti J (2012) The design of a community science cloud: the open science data cloud perspective. In: SC companion: high performance computing. Networking storage and analysis, Salt Lake City, UT, USA, pp 1051–1057 Grossman RL, Greenway M, Heath AP, Powell R, Suarez RD, Wells W, White KP, Atkinson MP, Klampanos IA, Alvarez HL, Harvey C, Mambretti J (2012) The design of a community science cloud: the open science data cloud perspective. In: SC companion: high performance computing. Networking storage and analysis, Salt Lake City, UT, USA, pp 1051–1057
13.
go back to reference Juve G, Rynge M, Deelman E, Vöckler J, Berriman GB (2013) Comparing futuregrid, amazon ec2, and open science grid for scientific workflows. Comput Sci Eng 15(4):20–29CrossRef Juve G, Rynge M, Deelman E, Vöckler J, Berriman GB (2013) Comparing futuregrid, amazon ec2, and open science grid for scientific workflows. Comput Sci Eng 15(4):20–29CrossRef
14.
go back to reference Kang U, Chau DH, Faloutsos C (2012) Pegasus: Mining billion-scale graphs in the cloud. In: 2012 IEEE international conference on acoustics, speech and signal processing, ICASSP 2012, Kyoto, Japan, 25–30 March 2012, pp 5341–5344 Kang U, Chau DH, Faloutsos C (2012) Pegasus: Mining billion-scale graphs in the cloud. In: 2012 IEEE international conference on acoustics, speech and signal processing, ICASSP 2012, Kyoto, Japan, 25–30 March 2012, pp 5341–5344
15.
go back to reference Kosar T, Livny M (2004) Stork: making data placement a first class citizen in the grid. In: 24th international conference on distributed computing systems ICDCS, pp 342–349 Kosar T, Livny M (2004) Stork: making data placement a first class citizen in the grid. In: 24th international conference on distributed computing systems ICDCS, pp 342–349
17.
go back to reference Nicolae B (2010) BlobSeer: towards efficient data storage management for large-scale, distributed systems. Theses, Université Rennes1 Nicolae B (2010) BlobSeer: towards efficient data storage management for large-scale, distributed systems. Theses, Université Rennes1
18.
go back to reference Nicolae B, Antoniu G, Bougé L, Moise D, Carpen-Amarie A (2011) Blobseer: next-generation data management for large scale infrastructures. J Parallel Distrib Comput 71(2):169–184CrossRef Nicolae B, Antoniu G, Bougé L, Moise D, Carpen-Amarie A (2011) Blobseer: next-generation data management for large scale infrastructures. J Parallel Distrib Comput 71(2):169–184CrossRef
19.
go back to reference Redis Protocol (2015) Open source (BSD licensed) publish/subscribe and in-memory data structure store. http://redis.io/. Accessed 15 Feb 2015 Redis Protocol (2015) Open source (BSD licensed) publish/subscribe and in-memory data structure store. http://​redis.​io/​. Accessed 15 Feb 2015
20.
go back to reference Saad W, Abbes H, Cérin C, Jemni M (2012) A self-configurable desktop grid system on-demand. In: Proceedings of the 2012 seventh international conference on P2P, parallel, grid, cloud and internet computing 3PGCIC, pp 196–203 Saad W, Abbes H, Cérin C, Jemni M (2012) A self-configurable desktop grid system on-demand. In: Proceedings of the 2012 seventh international conference on P2P, parallel, grid, cloud and internet computing 3PGCIC, pp 196–203
21.
go back to reference Saad W, Abbes H, Cérin C, Jemni M (2013) A data prefetching model for desktop grids and the condor use case. In: Proceedings of the 2013 12th IEEE international conference on trust, security and privacy in computing and communications TRUSTCOM ’13, pp 1065–1072 Saad W, Abbes H, Cérin C, Jemni M (2013) A data prefetching model for desktop grids and the condor use case. In: Proceedings of the 2013 12th IEEE international conference on trust, security and privacy in computing and communications TRUSTCOM ’13, pp 1065–1072
22.
go back to reference Saad W, Abbes H, Cérin C, Jemni M (2014a) Designing and implementing a cloud-hosted saas for data movement and sharing with slapos. Int J Big Data Intell IJBDI 1(1/2):18–35CrossRef Saad W, Abbes H, Cérin C, Jemni M (2014a) Designing and implementing a cloud-hosted saas for data movement and sharing with slapos. Int J Big Data Intell IJBDI 1(1/2):18–35CrossRef
23.
go back to reference Saad W, Abidi L, Abbes H, Cérin C, Jemni M (2014b) Wide area bonjourgrid as a data desktop grid: Modeling and implementation on top of redis. In: 26th IEEE international symposium on computer architecture and high performance computing, SBAC-PAD, pp 286–293 Saad W, Abidi L, Abbes H, Cérin C, Jemni M (2014b) Wide area bonjourgrid as a data desktop grid: Modeling and implementation on top of redis. In: 26th IEEE international symposium on computer architecture and high performance computing, SBAC-PAD, pp 286–293
24.
go back to reference Simonet A, Chard K, Fedak G, Foster IT (2015) Using active data to provide smart data surveillance to e-science users. In: 23rd Euromicro international conference on parallel, distributed, and Network-Based Processing, PDP 2015, Turku, Finland, 4–6 March 2015, pp 269–273 Simonet A, Chard K, Fedak G, Foster IT (2015) Using active data to provide smart data surveillance to e-science users. In: 23rd Euromicro international conference on parallel, distributed, and Network-Based Processing, PDP 2015, Turku, Finland, 4–6 March 2015, pp 269–273
25.
go back to reference Smets-Solanes JP, Cérin C, Courteaud R (2011) Slapos: A multi-purpose distributed cloud operating system based on an erp billing model. In: IEEE international conference on services computing SCC. IEEE, pp 765–766 Smets-Solanes JP, Cérin C, Courteaud R (2011) Slapos: A multi-purpose distributed cloud operating system based on an erp billing model. In: IEEE international conference on services computing SCC. IEEE, pp 765–766
26.
go back to reference Steinberg D, Cheshire S (2005) Zero configuration networking: the definitive guide, 1st edn. O’Reilly Media, Inc., Sebastopol Steinberg D, Cheshire S (2005) Zero configuration networking: the definitive guide, 1st edn. O’Reilly Media, Inc., Sebastopol
27.
go back to reference Tang B, Moca M, Chevalier S, He H, Fedak G (2010) Towards mapreduce for desktop grid computing. In: 3PGCIC 2010, international conference on P2P, parallel, grid, cloud and internet computing, 4–6 Nov 2010. Fukuoka Institute of Technology, Fukuoka, pp 193–200 Tang B, Moca M, Chevalier S, He H, Fedak G (2010) Towards mapreduce for desktop grid computing. In: 3PGCIC 2010, international conference on P2P, parallel, grid, cloud and internet computing, 4–6 Nov 2010. Fukuoka Institute of Technology, Fukuoka, pp 193–200
28.
go back to reference White T (2015) Hadoop–the definitive guide: storage and analysis at internet scale, 4th edn. O’Reilly, Sebastopol (revised and updated) White T (2015) Hadoop–the definitive guide: storage and analysis at internet scale, 4th edn. O’Reilly, Sebastopol (revised and updated)
Metadata
Title
A self-organized volunteer Cloud for e-Science
Authors
Walid Saad
Heithem Abbes
Christophe Cérin
Mohamed Jemni
Publication date
01-04-2016
Publisher
Springer US
Published in
The Journal of Supercomputing / Issue 4/2016
Print ISSN: 0920-8542
Electronic ISSN: 1573-0484
DOI
https://doi.org/10.1007/s11227-015-1564-z

Other articles of this Issue 4/2016

The Journal of Supercomputing 4/2016 Go to the issue

Premium Partner