Skip to main content
Top

Open Access 2018 | Open Access | Book

Cover of the book

Heterogeneity, High Performance Computing, Self-Organization and the Cloud

insite
SEARCH

About this book

This book is open access under a CC BY NC ND license. It addresses the most recent developments in cloud computing such as HPC in the Cloud, heterogeneous cloud, self-organising and self-management, and discusses the business implications of cloud computing adoption. Establishing the need for a new architecture for cloud computing, it discusses a novel cloud management and delivery architecture based on the principles of self-organisation and self-management. This focus shifts the deployment and optimisation effort from the consumer to the software stack running on the cloud infrastructure. It also outlines validation challenges and introduces a novel generalised extensible simulation framework to illustrate the effectiveness, performance and scalability of self-organising and self-managing delivery models on hyperscale cloud infrastructures. It concludes with a number of potential use cases for self-organising, self-managing clouds and the impact on those businesses.

Table of Contents

Frontmatter

Open Access

Chapter 1. Addressing the Complexity of HPC in the Cloud: Emergence, Self-Organisation, Self-Management, and the Separation of Concerns
Abstract
New use scenarios, workloads, and increased heterogeneity combined with rapid growth in adoption are increasing the management complexity of cloud computing at all levels. High performance computing (HPC) is a particular segment of the IT market that provides significant technical challenges for cloud service providers and exemplifies many of the challenges facing cloud service providers as they conceptualise the next generation of cloud architectures. This chapter introduces cloud computing, HPC, and the challenges of supporting HPC in the cloud. It discusses how heterogeneous computing and the concepts of self-organisation, self-management, and separation of concerns can be used to inform novel cloud architecture designs and support HPC in the cloud at hyperscale. Three illustrative application scenarios for HPC in the cloud—(i) oil and gas exploration, (ii) ray tracing, and (iii) genomics—are discussed.
Theo Lynn

Open Access

Chapter 2. Cloud Architectures and Management Approaches
Abstract
An overview of the traditional three-layer cloud architecture is presented as background for motivating the transition to clouds containing heterogeneous resources. Whereas this transition adds many important features to the cloud, including improved service delivery and reduced energy consumption, it also results in a number of challenges associated with the efficient management of these new and diverse resources. The CloudLightning architecture is proposed as a candidate for addressing this emerging complexity, and a description of its components and their relationships is given.
Dapeng Dong, Huanhuan Xiong, Gabriel G. Castañe, John P. Morrison

Open Access

Chapter 3. Self-Organising, Self-Managing Frameworks and Strategies
Abstract
A novel, general framework that can be used for constructing a self-organising and self-managing system is introduced. This framework is independent of the application domain. It embodies directed evolution, can be parameterised with different strategies, and supports both local and global goals. This framework is then used to apply the principles of self-organisation and self-management to resource management within the CloudLightning architecture.
Huanhuan Xiong, Christos Filelis-Papadopoulos, Gabriel G. Castañe, Dapeng Dong, John P. Morrison

Open Access

Chapter 4. Application Blueprints and Service Description
Abstract
In the context of creating a self-organising and self-managing cloud infrastructure we propose a set of extensions to the existing Service Description Languages (SDLs) and Application Blueprints in order to establish a common ground for the various CloudLightning components. By implementing this SDL and all the missing links one can assure that the CloudLightning system works in such a way that users can easily interact with it. In this chapter we present in detail the design decisions that were made during the development of various components alongside with their formal description.
Ioan Dragan, Teodor-Florin Fortiș, Marian Neagul, Dana Petcu, Teodora Selea, Adrian Spataru

Open Access

Chapter 5. Simulating Heterogeneous Clouds at Scale
Abstract
In this chapter, a review of existing cloud simulation frameworks is given along with an overview of the recently proposed CloudLightning simulation framework. Moreover, the parallel architecture and parallel implementation details of the CloudLightning simulator are presented along with the characteristics of the supported cloud architectures. These architectures include the traditional centralised approach as well as the Self-Organised and Self-Managed CloudLightning approach. The supported memory, network, and application execution models are reviewed. Furthermore, a recently proposed class of power models for heterogeneous CPU-Accelerator-based hardware is discussed. Finally, large-scale simulations for traditional and Self-Organised and Self-Managed cloud environments are presented and compared.
Christos K. Filelis-Papadopoulos, Konstantinos M. Giannoutakis, George A. Gravvanis, Charalampos S. Kouzinopoulos, Antonios T. Makaratzis, Dimitrios Tzovaras

Open Access

Chapter 6. Concluding Remarks
Abstract
Traditionally, access to high performance computing was restricted by architectural complexity, availability of trained personnel, and budgetary issues. At the same time, research suggests that existing measures for greater data centre energy efficiencies will reach theoretical and practical limits in the near future. This concluding chapter briefly discusses the potential of (i) cloud computing to disrupt the high performance computing sector, and (ii) new heterogeneous cloud architectures, based on the concepts of self-organisation, self-management, and the separation of concerns, to disrupt extant cloud resource management approaches.
Theo Lynn, John P. Morrison
Backmatter
Metadata
Title
Heterogeneity, High Performance Computing, Self-Organization and the Cloud
Editors
Theo Lynn
John P. Morrison
David Kenny
Copyright Year
2018
Electronic ISBN
978-3-319-76038-4
Print ISBN
978-3-319-76037-7
DOI
https://doi.org/10.1007/978-3-319-76038-4

Premium Partner