nach oben

2014 | Buch

Kapitel lesen Erstes Kapitel lesen

Science Gateways for Distributed Computing Infrastructures

Development Framework and Exploitation by Scientific User Communities

herausgegeben von: Péter Kacsuk

Verlag: Springer International Publishing

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten

Über dieses Buch

The book describes the science gateway building technology developed in the SCI-BUS European project and its adoption and customization method, by which user communities, such as biologists, chemists, and astrophysicists, can build customized, domain-specific science gateways. Many aspects of the core technology are explained in detail, including its workflow capability, job submission mechanism to various grids and clouds, and its data transfer mechanisms among several distributed infrastructures.

The book will be useful for scientific researchers and IT professionals engaged in the development of science gateways.

Inhaltsverzeichnis

Frontmatter

WS-PGRADE/gUSE Science Gateway Framework

Frontmatter

Chapter 1. Introduction to Science Gateways and Science Gateway Frameworks

Abstract

This chapter gives a short introduction to the basic architecture and functionalities of science gateways, as well as their development methods. It then briefly describes the EU FP7 SCI-BUS project that is developing a core science gateway framework called as WS-PGRADE/gUSE. A large number of various user communities have developed application-oriented science gateways by adapting and customizing the WS-PGRADE/gUSE gateway framework. The chapter also explains the vision of SCI-BUS on a collaboration-based SG instance development methodology. Finally, it gives a guide on how to read the rest of the book.

Péter Kacsuk

Chapter 2. Introduction to the WS-PGRADE/gUSE Science Gateway Framework

Abstract

WS-PGRADE/gUSE is a gateway framework that offers a set of high-level grid and cloud services by which interoperation between grids, clouds, and scientific user communities can be achieved. gUSE is also a workflow system that enables scientific communities to compose and execute a series of computational or data manipulation steps in a scientific application on Distributed Computing Infrastructures (DCIs). The current chapter summarizes the most important features of WS-PGRADE/gUSE.

Tibor Gottdank

Chapter 3. Workflow Concept of WS-PGRADE/gUSE

Abstract

This chapter introduces the data-driven workflow concept supported by the WS-PGRADE/gUSE system. Workflow management systems were investigated by Workflow Management Coalition, among others in aspects of implemented data flow structures, and several workflow patterns are identified as commonly used and meaningful workflow structures. The workflow concept of gUSE is shown by introducing the supported data patterns and illustrating their creation in the system. Moreover, the possibilities of utilizing parallelization techniques are described, and then the different views of a workflow design and management are described covering the whole lifecycle of a workflow development. Finally, more complex composition of patterns and their creation techniques are shown.

Ákos Balaskó

Chapter 4. DCI Bridge: Executing WS-PGRADE Workflows in Distributed Computing Infrastructures

Abstract

Solving distributed computing infrastructure (DCI) incompatibility issues in a generic way is a challenging and complex task. Gateways and workflow management systems are often tightly bound to some limited number of supported DCIs. To enable gateways access to many different DCIs and to solve DCI compatibility among the very different workflow management systems, we have developed a generic solution, the DCI Bridge. In this chapter we describe its internal architecture, provide usage scenario and show how the DCI Bridge resolves interoperability issues between various middleware-based DCIs. We also provide insight about the capabilities of the realized system. The generic DCI Bridge service seamlessly enables the execution of workflows (and jobs) on major DCI platforms such as ARC, Globus, gLite, UNICORE, SGE, PBS, as well as web services or clouds.

Miklos Kozlovszky, Krisztián Karóczkai, István Márton, Péter Kacsuk, Tibor Gottdank

Chapter 5. Remote Storage Resource Management in WS-PGRADE/gUSE

Abstract

State-of-the-art gateways are connected to several distributed computing infrastructures (DCIs) that are able to run jobs and workflows simultaneously in all those different DCIs. Data of e-Science applications might be stored on different storage resources, making it difficult for various user communities to browse, access, or update it. This chapter presents the Data Avenue concept and related tools that aim at hiding technical details of accessing different storage resources, and providing an easy-to-use file browsing interface to manage, upload and download, and even transfer data between different types of storage resources. Usage scenarios to exploit Data Avenue services as well as security considerations are discussed.

Ákos Hajnal, Zoltán Farkas, Péter Kacsuk, Tamás Pintér

Chapter 6. WS-PGRADE/gUSE Security

Abstract

If any data is stored on publicly available services, then it is obligatory for the service to apply different security policies in order to prevent information leakage to unauthorized users. This also applies to e-Science gateways, where sensitive (for example, medical) data might be stored. Moreover, e-Science gateways usually enable the usage of distributed computing infrastructures that apply some sort of authentication, which should either be provided by the users, or be completely hidden and implemented inside the gateway in the form of robot credentials. In this chapter we discuss some aspects of security applied within WS-PGRADE/gUSE.

Zoltán Farkas

Chapter 7. WS-PGRADE/gUSE and Clouds

Abstract

Cloud computing is becoming more and more popular compared to grid computing due to its virtualization capabilities, flexibility and elasticity. In order to keep up with this trend, WS-PGRADE/gUSE has been extended with the possibility to use cloud infrastructures in workflows, either directly as described in Chap. 4 or via the CloudBroker Platform service. In this chapter we present the integration of WS-PGRADE/gUSE and the CloudBroker Platform, which is particularly useful in the case of accessing commercial clouds and/or using preregistered applications in commercial and academic clouds.

Zoltán Farkas, Ákos Hajnal, Péter Kacsuk

Chapter 8. Developing Science Gateways at Various Levels of Granularity Using WS-PGRADE/gUSE

Abstract

Science gateways can provide access to distributed computing resources and applications at very different levels of granularity. Some gateways do not even hide the details of the underlying infrastructure, while on the other hand some provide completely customized high-level interfaces to end-users. In this chapter the different granularity levels at which science gateways can be developed with WS-PGRADE/gUSE are analysed. The differences between these various granularity levels are also illustrated via the example of a molecular docking gateway and its four different implementations.

Tamás Kiss, Gábor Terstyánszky, Péter Borsody, Péter Kacsuk, Ákos Balaskó

Chapter 9. Sharing Science Gateway Artefacts Through Repositories

Abstract

Researchers want to run scientific experiments focusing on their disciplines; they do not necessarily want to know how and where the experiments are executed. Science gateways hide details by coordinating the execution of experiments using different infrastructures and workflow systems. ER-flow/SHIWA and the SCI-BUS project developed repositories to share artefacts such as applications, portlets, workflows, etc. inside and among research communities. Sharing artefacts in repositories enables gateway developers to reuse them when building a new gateway and/or creating a new application.

Gábor Terstyánszky, Edward Michniak, Tamás Kiss, Ákos Balaskó

Domain-Specific Science Gateways Customized from the WS-PGRADE/gUSE Framework

Frontmatter

Chapter 10. Computational Neuroscience Gateway: A Science Gateway Based on the WS-PGRADE/gUSE

Abstract

Computational neuroscientists face challenges to manage ever-increasing large volume of data and to process them with applications that require great computational power. The Brain Imaging Centre of the Academic Medical Centre of the University of Amsterdam is a community of neuroscientists who are involved in various computational neuroscience research studies. They face various challenges to process and manage a growing amount of neuroimaging data. The goal of the computational neuroscience gateway is to facilitate large-scale data processing on distributed infrastructures and to enhance data management and collaboration for scientific research. The gateway is based on the WS-PGRADE/gUSE generic science gateway framework as platform for distributed computing, and it is connected to a data server based on the eXtensible Neuroimaging Archive Toolkit (XNAT). This chapter presents the design and architecture of the gateway with focus on the utilization of the WS-PGRADE/gUSE framework, and the lessons learned during its implementation and operation.

Shayan Shahand, Mohammad Mahdi Jaghoori, Ammar Benabdelkader, Juan Luis Font-Calvo, Jordi Huguet, Matthan W.A. Caan, Antoine H.C. van Kampen, Sílvia D. Olabarriaga

Chapter 11. Molecular Simulation Grid (MosGrid): A Science Gateway Tailored to the Molecular Simulation Community

Abstract

The Molecular Simulation Grid (MoSGrid) science gateway developed on top of WS-PGRADE/gUSE targets the needs of the molecular simulation community in a virtual research environment. The existing science gateway infrastructure has been extended on several levels and layers to especially support the computing- and data-intensive research domains of quantum chemistry, molecular dynamics, and docking. On the security level, the MoSGrid project has implemented a granular security concept, which applies Security Assertion Markup Language (SAML) assertions for trust delegations. Support of the highly scalable grid middleware UNICORE has been added to the middleware layer as has the support of the object-based file system XtreemFS for distributed data management. Furthermore, developers are supported by the PortletAPI, which eases the creation of domain-specific portlets. The standardized data exchange format of Molecular Simulation Markup Language (MSML) which sets the stage for the MoSGrid metadata management features, is a main building block for the PortletAPI.

Sandra Gesing, Jens Krüger, Richard Grunzke, Luis de la Garza, Sonja Herres-Pawlis, Alexander Hoffmann

Chapter 12. Statistical Seismology Science Gateway

Abstract

Seismic hazard assessment and risk analysis are critical for human life since these fields study and model expected earthquakes to produce seismic hazard maps and to determine the risk of damage from potential earthquakes to buildings, dams, etc. The Statistical Seismology Science Gateway, built on the gUSE framework, holds a comprehensive set of tools and models covering seismic data analysis, hazard assessment and risk analysis. It provides an environment in such a way that skilled users can experiment their own approaches by simply accessing the gateway services, while novice users can easily use existing applications. This chapter focuses on the development of the workflows implementing these tools and models as well as the portlets to access them. It also discusses how easily gUSE framework was customised to build this science gateway and how it is operated.

Çelebi Kocair, Cevat Şener, Ayşen D. Akkaya

Chapter 13. VisIVO Gateway and VisIVO Mobile for the Astrophysics Community

Abstract

The availability of large-scale digital surveys offers tremendous opportunities for advancing scientific knowledge in the astrophysics community. Nevertheless, the analysis of these data often requires very powerful computational resources. This chapter focuses on the development issues to design and implement a science gateway and a mobile application tailored for astrophysics needs by customizing the WS-PGRADE/gUSE technologies. The VisIVO Gateway integrates services for processing and visualizing large-scale multidimensional astrophysical datasets on distributed computing infrastructures. The core tools and services supported, employing the gUSE ASM API are presented. The gateway is exploited by a mobile application called VisIVO Mobile, which allows smartphone devices to perform analysis and visual discovery of large-scale astrophysical datasets. The mobile application configures and submits the VisIVO workflows by means of the gUSE remote API.

Eva Sciacca, Fabio Vitello, Ugo Becciani, Alessandro Costa, Piero Massimino

Chapter 14. HELIOGate, a Portal for the Heliophysics Community

Abstract

Heliophysics is the branch of physics that investigates the interactions between the Sun and the other bodies of the solar system. Heliophysicists rely on data collected from numerous sources scattered across the Solar System. The data collected from these sources is processed to extract metadata and the metadata extracted in this fashion is then used to build indexes of features and events called catalogues. Heliophysicists also develop conceptual and mathematical models of the phenomena and the environment of the Solar System. More specifically, they investigate the physical characteristics of the phenomena and they simulate how they propagate throughout the Solar System with mathematical and physical abstractions called propagation models. HELIOGate aims at addressing the need to combine and orchestrate existing web services in a flexible and easily configurable fashion to tackle different scientific questions. HELIOGate also offers a tool capable of connecting to sizeable computation and storage infrastructures to execute data processing codes that are needed to calibrate raw data and to extract metadata.

Gabriele Pierantoni, Eoin Carley

Chapter 15. Science Gateway for the Serbian Condensed Matter Physics Community

Abstract

The Academic and Educational Grid Initiative of Serbia hosts and maintains the scientific gateway of the Serbian condensed matter physics community. The gateway is built around a code set addressing problems from the physics of ultracold quantum gases, solving the nonlinear Schroedinger equation, the Gross–Pitaevskii equation in real and imaginary time, and a path integral algorithm for estimation of quantum-mechanical transition amplitudes, which are relevant for various applications. Here we present the realization of the gateway and technologies used for its implementation.

Dušan Vudragović, Antun Balaž

Further Applications of WS-PGRADE/gUSE

Frontmatter

Chapter 16. WS-PGRADE/gUSE-Based Science Gateways in Teaching

Abstract

Various WS-PGRADE/gUSE science gateways have been extensively used in educational contexts, supporting courses offered by different European universities and organizations. This chapter presents some examples of how WS-PGRADE/gUSE generic and customized gateways have been used in such courses. These examples include practical cases from a variety of scientific fields and educational styles. For each case, the educational context and the course organization are presented, with emphasis on how the respective portal has been adopted for the practical exercises. A summary of experiences are also reported, including advantages and difficulties faced for using these gateways in teaching.

Sílvia Delgado Olabarriaga, Ammar Benabdelkader, Matthan W.A. Caan, Mohammad Mahdi Jaghoori, Jens Krüger, Luis de la Garza, Christopher Mohr, Benjamin Schubert, Anatoli Danezi, Tamas Kiss

Chapter 17. WS-PGRADE/gUSE in European Projects

Abstract

Besides core project partners, the SCI-BUS project also supported several external user communities in developing and setting up customized science gateways. The focus was on large communities typically represented by other European research projects. However, smaller local efforts with the potential of generalizing the solution to wider communities were also supported. This chapter gives an overview of support activities related to user communities external to the SCI-BUS project. A generic overview of such activities is provided, followed by the detailed description of three gateways developed in collaboration with European projects: the agINFRA Science Gateway for Workflows for agricultural research, the VERCE Science Gateway for seismology, and the DRIHM Science Gateway for weather research and forecasting.

Tamás Kiss, Péter Kacsuk, Róbert Lovas, Ákos Balaskó, Alessandro Spinuso, Malcolm Atkinson, Daniele D’Agostino, Emanuele Danovaro, Michael Schiffers

Chapter 18. Creating Gateway Alliances Using WS-PGRADE/gUSE

Abstract

The STARnet Gateway Federation is a unique example of a federated network of science gateways based on WS-PGRADE/gUSE technologies, and explicitly designed and tuned to the needs of the astronomical and astrophysical (A&A) community in Europe. The use of a federated gateway infrastructure allows scientists to explore new collaboration opportunities and advancing the scientific research activity within A&A. STARnet Gateways share a common authentication system, a distributed computing infrastructure, data archives, portlets, and workflow repositories. Building upon these technologies, a number of challenging applications from different A&A domains have been successfully prototyped and tested.

Ugo Becciani, Eva Sciacca, Alessandro Costa, Piero Massimino, Fabio Vitello, Santi Cassisi, Adriano Pietrinferni, Giuliano Castelli, Cristina Knapic, Riccardo Smareglia, Giuliano Taffoni, Claudio Vuerli, Marian Jakubik, Lubos Neslusan, Mel Krokos, Gong-Bo Zhao

Chapter 19. Commercial Use of WS-PGRADE/gUSE

Abstract

Although originally an academic and research product, the WS-PGRADE/gUSE framework is increasingly applied by commercial institutions too. Within the SCI-BUS project, several commercial gateways have been developed by various companies. WS-PGRADE/gUSE is also intensively used within another European research project, CloudSME (Cloud-based Simulation Platform for Manufacturing and Engineering). This chapter provides an overview and describes in detail some commercial WS-PGRADE/gUSE-based gateway implementations. Two representative case studies from the SCI-BUS project, the Build and Test portal and the eDOX Archiver Gateway, are introduced. An overview of WS-PGRADE/gUSE-based gateways for running simulation applications in the cloud within the CloudSME project is also provided.

Tamás Kiss, Péter Kacsuk, Éva Takács, Áron Szabó, Péter Tihanyi, Simon J. E. Taylor

Backmatter

Titel: Science Gateways for Distributed Computing Infrastructures
herausgegeben von: Péter Kacsuk
Verlag: Springer International Publishing
Electronic ISBN: 978-3-319-11268-8
Print ISBN: 978-3-319-11267-1
DOI: https://doi.org/10.1007/978-3-319-11268-8