Techniques in DPM allow a dynamic adjustment of the system's power states based on the system's current behavior. Additionally, they assume that the prediction of the workloads requirements is possible passable, which enables the adaptation of the future system behavior leading to perform the appropriate actions according to these requirements.
Thus, the reduction in DPM is temporary in the system, and it would last for an indefinite period of time according to the available resources and current workloads.
These techniques are distinguished by the level they are applied: hardware level or software level, as further explained in the next two sections.
Software Level Solutions
A hardware level solution is very sophisticated. It is difficult to implement any modification or reconfiguration at this level. Therefore, shifting to the software level solutions is highly demanded.
There have been some proposed solutions for managing power consumption, such as Advanced Power Management (APM) performed by Basic Input/Output System (BIOS), firmware based, and platform-specific power management systems. But these solutions are hardware and platform-dependent.
The first attempt to address a software solution was made in 1996, when the first version of the Advanced Configuration and Power Interface (ACPI) was proposed. ACPI was a platform-independent interface. It improved the existing power and configuration standards for hardware devices and allowed operating systems to control power management and efficiently configured the hardware platform they ran on. ACPI has been widely accepted as a standard that brings DPM into the operating system (OS) control of all aspects of power management and device configuration [
20,
21].
ACPI defines a number of power states. These states can be enabled in the system at runtime. Also, it gives software developers the ability to leverage the flexibility in adjusting the system’s power states [
1].
The states which are relevant in the context of DPM are:
i.
C-states: C-states are the CPU power states C0, C1, C2, and C3. They denote the Operating State, Halt, Stop-Clock, and Sleep Mode respectively. Recently, deep power down states (e.g. C4, C5, C6, and C7) were introduced to define different levels of lower activity.
ii.
P-states: P-states are the power-performance states when the processor is operating. P-states can be one of several states, and each state represents a specific combination setting of DVFS values. They are implementation-dependent, but P0 is always the highest performance one. If there is implementation-specific limit of n, then P1 to Pn are successively lower performance states.
As mentioned earlier, data centers consume huge amounts of electrical power. Although DVFS technique can provide an efficient direction in managing power consumption of the CPU, more power reduction is required. The server consumes over 50 % of its actual peak power and up to 70 % in some cases, even when it is completely idle [
22]. Switching PMs off is the only possible way to eliminate their SPC. These circumstances led to propose some solutions which are suit the data centers environment. Those solutions aimed to consolidate the workload to fewer PMs and deactivating the idle ones. The consolidation is a complicated problem. The performance of the applications can be affected from unnecessary consolidation. Therefore Quality of Service (QoS) requirements restricts consolidation. In general, QoS are defined in terms of Service Level Agreement (SLA) between cloud users (or their brokers) and cloud providers.
Many studies and approaches have been dedicated to enhance the power-efficiency in virtualized data centers.
Management techniques that take into account the concept of virtualized systems were first explored by Nathuji and Schwan in 2007 [
23]. The authors examined ways of integrating power management mechanisms into the virtualized data center environments and presented a power management model for such environments, called VirtualPower, which controlled the power consumption of underlying platforms. A new technique, called “soft resource scaling”, was applied in the model of that study. This technique emulated hardware scaling by providing less time for the VM to utilize a resource. It is a very efficient technique when hardware scaling is not supported. VirtualPower provided a set of virtualized power states, called VirtualPower Management (VPM) states, which were taken into account in all management actions. VirtualPower was able to modify model-specific registers (MSRs) in the PM and change the power states. Thus, an abstraction of (VPM) channels could be created. The channels then delivered guest VM power management actions as a set of ‘soft’ states that provided a virtual layer for application-aware management.
In the same year, Nathuji, Isci, and Gorbatov [
24] pioneered in exploiting platform heterogeneity to improve the power management, while taking into account the emergence of the virtualization concept. They defined an intelligent workload allocation system that efficiently mapped workloads to the best fitted resources in heterogeneous platforms. The model consisted of three components: platform/workload descriptors, power/performance predictor, and allocator. The workload descriptor consisted of modules labeled by attributes, and a list of values for these attributes was provided. The platform descriptor consisted of individual modules representing system components that were used to convey information according to the PM's power management and hardware capabilities. The power/performance predictor used these descriptors to estimate the performance and power savings in the data center. Finally, the allocator used these predictions to map workloads to a specific type of platform. The allocator of the study evaluated the power efficiency tradeoffs of assigning a workload to many kinds of platforms, while each kind of platforms was associated with a cost metric. Jobs of the workloads were queued according to the values of the cost metric. Then, a mapping process was performed based on this queue with priority given to the job with the highest cost.
In [
25], the authors investigated several previous solutions to the problem of high power consumption in data centers. Until then, there had not been any corresponding work on coordinating all these solutions. Hence, the authors characterized the existing solutions and classified them into hardware/software and local/global levels.
Then, the authors proposed their solution which was a model of multiple feedback controllers at various levels. The solution implemented in a distributed manner. Being the core level of that model, the Efficiency Controller (EC) was implemented in the system that served as a “container”. This container was used as a reference to the controller depending on the desired part of its capacity. There was also a sensor to compare the value with the actual utilization of the container. By managing the actual resource utilization in the system and the reference value of the container, EC dynamically “resized the container”. EC also monitored previous/former resource utilization and adjusted a processor P-state accordingly in order to match estimated demand. In this case, the consumed power was adapted to the total resource demand of the workloads in real time fashion. The second controller in that model, called Server Manager (SM), implemented the server level power capping. It measured the consumed power in the server and compared it with the power budget. The third controller, called Enclosure Manager (EM), implemented enclosure-level power capping. It monitored the total power consumption of the code enclosure and compared it with an enclosure-level power budget periodically. The forth controller, which implemented the group-level power capping, was called Group Manager (GM). It worked at either the rack level or the data center level. GM function is to compare the actual power consumption of the group with the group power budget. The final controller in the model was the Virtual Machine Controller (VMC). The function of VMC is to collect the resource utilizations of the individual VMs in the system and performed a new VM-to-Servers mapping to minimize the total power based on the utilization information. That model provided a feedback control loop to federate multiple power management solutions at different levels. However, the proposed architecture needs an extension to include coordination with other solutions in the performance domains.
The authors investigated the design and implementation of a power-aware application placement model under heterogeneous virtualized server environment in [
26]. When matching the application containers to the PMs, the placement model considered the power and migration costs.
The authors divided this work into two parts: The first part presented methods to implement the placement of cost-aware application on real servers, while the second one presented an application placement framework, called (pMapper), to solve the problem of high power consumption and reduce this amount as much as possible. The architecture of that framework consisted of three managers (Performance Manager, Power Manager, and Migration Manager) and an Arbitrator. Performance Manager monitored the behavior of the workload and rearranged the VMs while taking into account both the current resource requirements and SLAs. It consulted a knowledge base for an application performance in addition to the cost of its VM migration from one machine to another. Power Manager monitored the current power consumption and adjusted hardware power states. It used the power model in the knowledge base to determine the placement, and thus could estimate the power for every placement scenario and suggest the server throttling. Migration Manager estimated the cost of moving from a given placement to a new one and issues instructions for VM live migration to consolidate the overall VMs of the application. Arbitrator, which was the central intelligence part in pMapper, received the estimations from the three managers, configured a space for the VM placements, computed the best VMs sizes, and implemented an algorithm to choose the best VM placement. Once the Arbitrator decided on the new configuration, the three managers executed the following operations respectively: VM sizing, server throttling and live migration.
The authors in [
27] implemented a dynamic resource provisioning of VMs for web applications in virtualized server environments. In this study, the provisioning problem was defined as sequential optimization under uncertainty. The authors proposed a framework called Limited Look-ahead Control (LLC) to solve this problem. The framework monitored the process of VMs provisioning and calculated the switching costs resulting from this provisioning, then it encoded the corresponding cost value in the optimization problem. In order to reduce power consumption and maintain the SLA that was defined in term of the processing rate for each application, the work proposed an online controller to decide the number of PMs and VMs to be allocated to each job in the workload. In this case, it is possible to turn the PMs on and off upon the controller’s decision which may vary according to workload types.
In [
28], the authors designed a trace-based workload placement controller to optimize the VMs allocation based on historical information, while maintaining specific QoS requirements. Then, they proactively mapped the workload periodically to the PMs. The work traded off between the consumed power of the required resources and the number of migrations that may occur after VMs placement. This approach was based on the fact that the historic traces of any application offer a model for describing the future application behavior. Traces were used to decide how to consolidate the VMs workloads to the PMs. The workload trace-based approach was considered a proactive controller that caused the VMs of the workloads migration among PMs in order to consolidate the system periodically. In this work, the optimization algorithm was enhanced by reducing the total number of migrations that occurred during successive control intervals. In addition to the proactive controller, the study also introduced a reactive migration controller. It detected the overload/under-load conditions in the PM and initiated the VMs migration. It dynamically added and removed PMs to maintain a balance of supply and demand resources by turning PMs on and off. The controller’s act was based on the real-time data collected from the resource usage. However, during the workload placement approach, this work did not consider the effect of cache sharing on power efficiency.
The authors in [
29] proposed novel techniques for placement and power consolidation of VMs in the virtualized data centers. Most hypervisors (e.g. Xen and VMware) provide administrators that are able to adjust the minimum number of the required resources (min) and the maximum number of allowable resources for a VM (max). Values such as (min) and (max) are very useful in ensuring the intelligent distribution of resources between different workloads. Hence, this study leverages min, max and the shares parameters supported in virtualization technologies. The proposed techniques provided a tradeoff mechanism for power and performance when running heterogeneous workloads in data centers. They involved sorting physical machines (PMs) in an increasing order of the power cost per unit of capacity, wherein the objective function included power consumption and the utilization resulted from the execution of a specific VM, which was a priori assumption. The placement strategy then is to place all the VMs at their maximum resource requirements in a first-fit manner. An amount of 10 % of the capacity is leaved as a spare to handle any future growth of the resource usage.
The authors in [
30] proposed a solution that facilitated coordination between the virtualization concept and the loosely couple platforms in data centers. This solution, known as vManage, provided an execution policy for better virtualization management. The design of vManager, which consisted of registry and proxy mechanisms, had several features such as a unified monitoring over platform and virtualization domains, the coordinators are able to interface with existing management controllers, easy portable across different hardware platforms with an independent implementation, and flexibility and extensibility in allowing new management solutions to participate in a “plug-and-play” manner. vManager is based on the concept of estimating “stability.” In other words, it is based on the probability that a proposed VM reallocation will stay efficient for an appropriate future period of time. The predictions of future resource demands are computed using a probability function. This study provided an example of stability management and coordinated VM placement in the data center called the stabilizer, which is a plug-in’ component. The VM placement policy considers the platform requirements as well as the requirements of the VM including CPU, memory, and network constraints. However, the proposed solution needs to be extended for larger scale data centers.
In [
31], the authors tackled the problem of optimizing resource allocation for multitier applications in consolidated server environments. This was achieved by exploring a runtime cost-sensitive adaptation engine that weighed the potential benefits of automatic reconfiguration and their corresponding costs. The study suggested an offline cost model for each application to decide when and how to reconfigure the VMs. The model estimated the cost according to the changes in the utility for the application, which was a function of the response time for that application. This model improved the application response time and ameliorated the period over which the system remained functional in the new VM configuration.
In order to optimize resource allocation in data centers, the authors in [
32] presented a multi-tiered resource scheduling scheme in that environment. This scheme, named RAINBOW, automatically provided on-demand capacities via resources flowing that indicated which resources were released by some VMs to be allocated to other VMs. The resource flowing was modeled using optimization theory and was resolved by the Simplex Method. Based on this model, the authors proposed a scheduling algorithm to optimize resource allocation and to ensure performance of some critical services, leading to reduction in consumed power. The proposed scheduling algorithm had three levels. The first was Application-Level scheduler: which dispatched requests across the VMs by applying VM migration; the second was Local-Level scheduler: which allocated resources of a PM to VMs according to their priorities; and the third was Global-level scheduler: which controlled the flow of resources among the applications.
In order to meet the QoS in this study, the resources were allocated to the applications according to the application priorities. In case of limited resources, the resources of low priority applications would be allocated to critical ones. The authors stated that performance of critical applications was guaranteed using this scenario. However, there is a need to analyze the effect and overhead caused by each tier of the proposed multi-tiered resource scheduling.
In [
33], Stillwell, Schanzenbach, Vivien, and Casanova demonstrated the utilization of resource allocation management systems in virtualization technology for sharing parallel computing resources among competing jobs. The model focused on HPC applications. The authors defined the resource allocation problem considering a number of underlying assumptions and determined the complexity of each one. They also proposed a more general approach by eliminating some assumptions. The problem of resource allocation was formally defined as Mixed Integer Programming (MIP) model. The design of this model was based on two assumptions: first, the application required only one VM instance, and second, the computational power and memory requirements needed by the application were static. However, estimating accurate job resource requirements for jobs is a weak point in this work.
In [
34], the authors investigated the problem of high power consumption in cloud data centers. They proposed an energy-aware resource provisioning that mapped the resources to the applications while meeting the SLA between the provider and the users. The study proposed visions, challenges, principles, and an architectural framework for energy-efficient model in virtualized data center environments. It focused on providing resources dynamically and how allocation algorithms could be improved via managing consumed energy among various data center infrastructures. The proposed framework consisted of three levels;
User level: at this level, users or their brokers submit their service requests to the cloud.
Allocator level: which acts as an interface between users and the cloud infrastructure.
Data center level: represents the VMs and PMs.
The VM allocation process was divided into two parts. The first part was receiving new requests for VM provisioning. All VMs were sorted in decreasing order according to their current utilization, and then allocated each VM to a PM that expanded the minimum amount of increment in the consumed energy. The second part was optimizing the current allocation of VMs, which was further divided into two steps: selecting the VMs to be migrated and placing the selected VMs on new PMs.
In that study, the selection of the migrating VMs was heuristically achieved. Four heuristics were used; the first one was based upon setting an upper utilization threshold for PMs. The VMs were allocated to a PM when the placement process kept the total CPU utilization below that threshold. The other three heuristics were based on the idea of setting two thresholds for utilization, upper and lower. The total CPU utilization by all VMs had to remain between the setting two thresholds.
The four heuristics relied on three policies: the first was migrating the least number of VMs to minimize migration overhead. The second was migrating VMs that had the lowest usage of CPU to maintain utilization. The third policy was selecting the necessary number of VMs based on a uniformly distributed random variable.
In [
35], the authors analyzed the cost of energy in virtualized data center environments. The virtualization concept inspired the authors to propose an energy-efficient framework dedicated to cloud architecture, and they called it Green Open Cloud (GOC). GOC was proposed for the next generation of Cloud data centers that support extra facilities, such as advanced reservation. GOC has the ability to aggregate the workload by negotiating with users. In this case, the idle PMs can be switched off for a longer period of time without the need for further negotiations with the users.
In [
36], the paper presented the design and implementation of two VM management solutions: Distributed Resource Scheduler (DRS) and Distributed Power Management (DPM).
DRS managed the allocation of physical resources to a set of VMs by mapping these VMs to PMs. Additionally; it performed intelligent load balancing in order to enhance the performance. DRS provided a “what-if” mode to handle any changes in workloads or PM configuration. DRS solution performed four key resource-management operations: Computes the amount of resources requested by a VM based on the reservation, periodically balanced load across PMs by performing VM migrations, saved power by benefiting from DPM, and performed initial placement of VMs onto PMs.
DPM surpassed DRS in its ability to reduce the consumed power by consolidating VMs onto fewer number of PMs. DPM is able to power a PM off when the CPU and memory resources have low utilization. At the same time, it is able to power a PM on appropriately when demand on resources increases, or in order to maintain the constraints.
The researchers in [
37] investigated the resource fragments which resulted from the imbalanced use of the PM resources. They mentioned that the problem of VMs to PMs placement should be solved according to a resource-balanced strategy. To characterize the resource usage of each PM, the authors proposed a multi-dimensional space model, while each dimension of the space corresponds to one dimensional resource. The whole space is partitioned into three domains, each with particular features, to elucidate the appropriateness of resource utilization for each VM placement process. The proposed model can be used as a guide in designing the resource-balanced VM placement algorithms. Based on this model, the researchers proposed their own energy efficient VM placement algorithm and called it (EAGLE). EAGLE was based on a tradeoff at each time-slot between balancing multi-dimensional resource utilization and reducing the total number of PMs during VMs placement. EAGLE checks the next resource usage state for each available PM and chooses the most suitable one. A new PM could be turned on to avoid any excessive resource fragments. This would decrease excessive resource fragments and further reduce the number of PMs on the long run. This algorithm resulted in a better utilization of resources, introduced less resource fragments and saved energy.
The trade-off between energy efficiency and SLA constraints were analyzed in [
38]. The authors studied the users’ utilization patterns and introduced a dynamic resource provisioning mechanism to over-allocate capacity in cloud data centers. The core concept of the proposed mechanism was to employ the resource utilization patterns of each user to eliminate any potential waste in utilization that might result from overestimation of resources requests. The over-allocate algorithm in this mechanism considered two different parameters: the predicted resource utilization based on historical data, and dynamic occupation to determine the maximum number of resources that could be allocated over the actual PM capacity. The PM capacity was calculated based on the cost-benefit analysis of deploying a specific instance into a particular server. It also allowed consolidating additional VMs in the same PM. A compensation mechanism to adjust resource allocation in cases of underestimation was also discussed in this study.
In his thesis [
1], Anton Beloglazov proposed novel algorithms for distributed dynamic consolidation of VMs in virtualized cloud data centers. The thesis reduced amount of the total energy consumption under different workload requirements. Energy consumption was reduced by dynamically switching PMs on and off to meet the current resource demand. The author suggested a data center model and applied a VM placement algorithm that worked in this model. The data center model consisted of numerous PMs and had two types of managers to coordinate the VM management. The local manager residing on each PM as a module of the virtual machine management (VMM) and the global manager that resided as a master for a specific number of PMs. The decision to place the VM on a specific PM was made individually according to the communication between the local manager on the PM and the global managers.
The thesis also presented a novel approach that optimally solved the problem of host overload detection by maximizing mean inter-migration time. This approach was based on a Markov chain model and worked for any fixed workload and a given state configuration.
The authors in [
39] investigated the VM provisioning as an essential technique in cloud computing. VM provisioning refers to providing VMs upon users’ requests. The work proposed a power-aware VM provisioning model for both hard and soft real-time services. A real-time service (such as financial analysis, distributed databases, and image processing) was presented as real-time VM requests. It included many tasks, and each task was defined by some parameters. Therefore, when users made their requests to the cloud computing environment, appropriate VMs were allocated for executing those requests. Brokers were responsible for finding VMs for the users’ requests. The requirements of the requested VMs were called Real-Time Virtual Machine (RT-VM) in this paper. Each RT-VM
V
i
included three parameters: utilization
u
i
, Million Instruction Per Second (MIPS)
m
i
, and deadline
d
i
. Such requirements imply that the real-time service is guaranteed if the allocated VM keeps providing a processing capacity of
u
i
×
m
i
amount by the specified deadline
d
i
. After defining their power model, the authors proposed a power aware framework including the following five steps: Requesting a VM, generating the RT-VM from real-time applications, requesting a real-time VM, mapping the physical processors, and finally, executing the real-time applications.
This study suggested the variable w
i
as the remaining service time. The initial value of w
i
is defined by u
i
× m
i
× (d
i
- t
s
), at the submission time t
s
. If V
i
is provided with q
i
MIPS rate for the period t
p
, then, the value w
i
is decreased by q
i
× t
p
. In such case, V
i
finishes its service when w
i
becomes zero.
When a datacenter receives a RT-VM request from users or their brokers, it returns the price of providing the RT-VM service if it can provide real-time VMs for that request. Then, the users or brokers can select the VM with the minimum price among available VMs provided by the datacenters. Thus, the provisioning policy in this work was selecting the processing element with the minimum price to maximize user/broker profits. If the process element is able to schedule V
i
, it estimates provisioning energy and cost. As the provisioning policy aimed to provide a lower price for the user, the proposed algorithm in this paper discovered the minimum-price processor. For the same price, less energy is preferable because it produces higher profit. Finally, a VM is mapped if V
i
is schedulable on the datacenter. However, in this study, the soft real-time VM provisioning did not considered.
The authors in [
40] stated the previous real time job scheduling algorithms were running in uncertain cloud environments. Those algorithms assumed that cloud computing environments were deterministic, and there were statistical pre-computed schedule decisions to be followed during the schedule execution. So this study introduced the interval number theory to describe the uncertainty of the cloud computing environment and the impact of uncertainty on the scheduling quality in a cloud data center. Accordingly, a novel scheduling algorithm, called Proactive and Reactive Scheduling (PRS), was presented. It dynamically exploited proactive and reactive scheduling methods for scheduling real-time jobs.
The proactive scheduling was used to build baseline schedules depending on redundancy, where a protective time cushion between jobs’ finish time lower bounds and their deadlines was added to guarantee job deadlines. The reactive scheduling was triggered to generate proactive baseline schedules in order to account for various disruptions during the course of executions.
Some strategies were presented to scale the system’s computing resources up and down according to workload to improve resource utilization and to reduce energy consumption for the cloud data center. These strategies were proposed to treat the following five events as disruptions:
2)
the system becomes overloaded;
3)
a new urgent job arrives or the waiting jobs become urgent;
5)
some VMs’ idle time exceeds the pre-established threshold.
However, estimating job execution time is main factor in the scheduling model proposed in this work. So, improving the precision of estimated job execution time may lead to better scheduling decisions.
The authors in [
41] presented two scheduling algorithms for precedence-constrained parallel VMs in a virtualized data center. The proposed algorithms used a new insertion policy to insert VMs among previously scheduled ones. The new policy inserted VMs into already switched on low utilized PMs to increase their utilization, thus reducing the total number of switched on PMs that served the VMs, and therefore enhancing energy efficiency.
The first algorithm, called Virtualized Homogeneous Earliest Start Time (VHEST), was an extension of the well-known HEFT algorithm. HEFT scheduled VMs according to the non-overlapping insertion policy. VHEST was modified to use overlapping insertion policy to minimize the makespan.
It had two major stages: VM selection and PM Selection. At the first stage, VMs were sorted according to their priority, and then the VM with the highest priority was selected to be placed on PM. At the second stage, the selected VM was placed in the best PM that minimized the VM’s start time by applying an overlapping-insertion policy.
The second algorithm, called Energy-Aware Scheduling Algorithm (EASA), solved a multi-objective problem. It improved the utilization of PMs and minimized the makespan. EASA also had two major stages: Local optimization and Global Optimization. The local optimization stage improved the utilization of the PMs. The global optimization stage reduced the number of switched on PMs by switching off the underutilized ones. However, the work did not support heterogeneous data centers.
An energy-aware resource provisioning framework for cloud data centers was proposed in [
42]. The main functions of the proposed framework can be summarized into three points:
1)
Predicting the upcoming number of the (VM) requests that would arrive to the cloud data center in a certain future period, associated with the requirements for each VM. The prediction approach relied upon monitoring past workload variations during a period of time. It combines machine learning clustering and stochastic theory to predict the number of the upcoming VM requests along with required resources associated with each request.
2)
Estimating the number of PMs needed in the data center that will serve the upcoming users’ requests. This estimation is based on the predictions of such requests.
3)
Turning the unneeded PMs in the data center to the sleep mode by applying intelligent power management decisions in order to reduce the consumed energy.
In addition to the previous studies, various optimization methods such as, Ant Colony Optimization (ACO), Particle Swarm Optimization (PSO), and Genetic Algorithms (GA)) were used to improve resource utilization and reduce energy consumption in the virtualized data centers.
In [
43], the researchers presented a framework to manage the VM placement in an IaaS environment. They defined an initial VM placement strategy and proposed multi-objective optimization algorithm based on (ACO) to determine the initial VMs placement. The proposed algorithm was an optimization method. It was able to achieve an optimal solution through efficient convergence by the constantly updated pheromone. The optimal solution was selected from a set of solutions using the exclusion method.
In [
44], the authors designed a distributed ACO-based algorithm for solving the VM consolidation problem (VMCP). The algorithm iterated over a finite number of cycle. At each cycle, an item was selected for each ant to be packed in a bin. If the bin did not have enough space according to defined constraints, another bin was selected. At the end of each cycle, the best solution found was saved and pheromone evaporation was triggered to achieve the VM consolidation.
In [
45], the authors proposed a multi-objective ACO to solve the problem of VM placement. The formulated multi-objective VM placement problem represented a permutation of VM assignment. The goal was to efficiently obtain a set of non-dominated solutions that reduced total power consumption resulting from resource wastage. This proposed model has two phases; Initialization phase: where the parameters and the pheromone trails were initialized. Iterative phase: in this phase, all VM requests were sent to the ants to start assigning VMs to the selected PMs. This was done by using a pseudo-random-proportional rule, which described the way each ant selected a particular next one VM pack into its current PM. Ants moved towards the most promising VMs based on information about the current pheromone concentration during the movement. Local and global pheromones were frequently updated. A local pheromone update was performed once an artificial ant built a movement. Then, a global update was performed with each solution of the current Pareto set after all ants had constructed their solutions.
The thesis in [
46] focused on the IaaS cloud service model. This model offered compute infrastructure to cloud consumers by provisioning VMs on-demand. The thesis investigated the challenge of designing, implementing, and evaluating an energy-efficient IaaS cloud management system for private clouds. The author proposed four contributions:
1)
IaaS Cloud Management system (Snooze): It was based on a self-configuring hierarchical architecture and performed VM management for distributed large-scale virtualized data centers by splitting the data center into independently managed groups of PMs and VMs. Snooze provided a holistic energy-efficient VM management solution. Particularly, it integrated a power management mechanism which automatically detected idle PMs, transitioned them into a power-saving state, and woke them up on demand.
2)
VM Placement via ACO: an ACO-based VM placement algorithm was used to solve the problem of considering only a single resource to evaluate the PM load and VM resource demands while ignoring the other resources. In addition, ACO appealed to the VM placement problem due to its polynomial time worst-case complexity and convenience of parallelization.
3)
VM Consolidation via Ant Colony Optimization: VM consolidation algorithms are required in order to enable continuous consolidation of already placed VMs on fewer PMs. This consolidation helped avoid resource fragmentation and further increases the data center resource utilization. Therefore, the researcher proposed a consolidation algorithm based on ACO to achieve both scalability and high data center utilization by applying VM consolidation.
4)
Fully decentralized consolidation system based on an unstructured peer-to-peer network.
In [
47], the authors proposed a VM consolidation scheme that focused on balanced resource utilization of servers across different computing resources with the goal of minimizing power consumption and resource wastage. This study presented an adaptation and integration of the ACO met heuristic with a balanced usage of computing resources. The degree of imbalance in the current resource utilization of a PM was captured and represented as a multi-dimensional server resource utilization, and then resources utilization are balanced using vector algebra.
The study in [
48] proposed a PSO-based algorithm, which could successfully reduce the energy cost and the time for searching feasible solutions. The authors presented an environment of heterogeneous multiprocessors, which is similar to the environment of cloud data center, and they proposed a job to the processor assignment model that would work in that environment. During assigning jobs to the processors, the velocity of the particles (that represented the jobs) determined their positions. This velocity will affect the overall convergence of the PSO algorithm and the efficiency of the algorithm’s global searching. The particle’s position updates present the next position of the job. As the particle position updates, it indicated that it needed to adjust the number of processors that fit its requirements. Then, the proposed algorithm optimized the most feasible solution to in order to reduce energy consumption by assigning jobs to new processors or by exchanging their corresponding processors. However, there are many constraint conditions in the work, reducing them will make the work more convenient for solving the problem of real-time job scheduling.
In [
49], the authors proposed a genetic algorithm for power-aware (GAPA) scheduling to find the optimal solution for the problem of VM allocation. In the proposed algorithm, a tree structure was used to encode chromosome of an individual job. The fitness function of GA calculated the evaluation value of each chromosome. The tree had three levels; Level 1: Consisted of a root node that did not have a significant meaning, Level 2: Consisted of a collection of nodes that represented a set of PMs, Level 3: Consisted of a collection of nodes that represented a set of virtual machines. Using this model, each instance of tree structure showed the VM to PM Allocation. However, the computational time of the GAPA is high, also deadline of jobs did not considered in the work.
In [
50], the authors proposed a distributed parallel genetic algorithm (DPGA) of placement strategy for VMs deployment on cloud platform. The proposed algorithm had two stages: It executed the genetic algorithm in parallel and in a distributed manner on several selected PMs in the first stage to get several solutions. Then, it continued to execute the genetic algorithm of the second stage with solutions obtained from the first stage as the initial population. A relatively optimal job to VM mapping was obtained as a result of the second stage. The fitness value of GA chosen here was performance per watt.
In [
51], the authors introduced a power efficient resource allocation algorithm for jobs in cloud computing data centers. The developed approach was also based on GA. Resource allocation was performed to optimize job completion time and data center power consumption. It considered a static scheduling of independent jobs on homogeneous single-core resources. The proposed algorithm, called Non-dominated Sorting Genetic Algorithm II (NSGA-II), was applied to cloud environments to explore space solutions and efficiently search for the optimal solution. The data center was modeled as three-tier fat-tree layers architecture: access, aggregation, and core layers. The access layer provided connection to servers which were arranged into racks with each rack being served by a single Top of the rack switch. Two fitness functions were used here: task completion time and data center power consumption. When the execution of the algorithm was completed and optimal Pareto solutions were obtained, it became possible to fine tune the trade-off between power consumption and execution time. Then, by using a procedure called ranking, the population of solutions were sorted heuristically into different non-domination levels. This procedure was repeated for every solution creating different groups or non-domination; an integer value called rank was assigned to each non-domination level. When applying selection and sorting, NSGA-II was able to deal with constraints. The solution with less constraint violation had a better rank.
Table
3 [
1,
23-
51] summarizes all the studies and techniques illustrated in this survey. The table compares the studies from many perspectives. It shows if the studies are applied in single or multiple data centers, Homogenous or heterogeneous environment, Specific or all types of jobs are used, if any other parameters are considered with enhancing the energy efficiency, the involved resources in enhancing the energy efficiency, HW or SW approach is applied with the study, the scheduling is online or offline, and finally, what is the used technology in each study?
Table 3
Studies of power efficiency improvement in virtualized data centers
| √ | | √ | | √ | | | √ | | | | | √ | √ | | √ | DVFS, SW-based Model, VM Consolidation, On/Off Switching. |
| √ | | | √ | √ | | | √ | | | | | √ | √ | | √ | DVFS, SW-based Model, VM Consolidation, On/Off Switching. |
| √ | | √ | | √ | √ | | √ | | | | | √ | √ | | √ | DVFS, SW-based Model, VM Consolidation, On/Off Switching. |
| √ | | | √ | √ | √ | | √ | | | | | √ | √ | | √ | DVFS, SW-based Model, VM Consolidation, On/Off Switching. |
| √ | | √ | | √ | | | √ | √ | | | | √ | | | √ | WS-based Model, On/Off Switching. |
| √ | | √ | | √ | | | √ | √ | | | | √ | | | √ | VM Consolidation, On/Off Switching. |
| √ | | √ | | | | | √ | | | | | √ | √ | | √ | DVFS, SW-based Model. |
| √ | | √ | | √ | | √ | √ | √ | | √ | | √ | | √ | | SW-based Model, VM Consolidation |
| √ | | √ | | √ | √ | √ | √ | √ | | | | √ | | √ | | SW-based Model |
| √ | | √ | | √ | | | √ | √ | | | | √ | | | √ | SW-based Model |
| √ | | | √ | | | | √ | | | | | √ | | | √ | SW-based Model, VM Consolidation |
| √ | | | √ | √ | √ | | √ | | | | | √ | √ | | √ | DVFS, SW-based Model. |
| √ | | √ | | √ | | | √ | | | | | √ | | | √ | SW-based Model, On/Off Switching. |
| √ | | √ | | √ | | | √ | √ | | | | √ | | | √ | SW-based Model, VM Consolidation, On/Off Switching. |
| √ | | √ | | √ | | | √ | | | | | √ | | | √ | SW-based Model, VM Consolidation. |
| √ | | √ | | √ | | | √ | | | | | √ | | | √ | SW-based Model, VM Consolidation, On/Off Switching. |
| | √ | | √ | √ | | | √ | | | | | √ | √ | | √ | DVFS, SW-based Model, VM Consolidation, On/Off Switching. |
| √ | | | √ | √ | √ | | √ | | | | | √ | √ | | √ | DVFS, SW-based Model. |
| | √ | √ | | √ | | √ | √ | √ | | | | √ | | | √ | SW-based Model. |
| √ | | | √ | √ | | √ | √ | | | | | √ | | | √ | SW-based Model, VM Consolidation. |
| √ | | | √ | √ | | | √ | √ | | | | √ | | | √ | SW-based Model. |
| √ | | √ | | √ | | √ | √ | √ | | √ | | √ | | | | ACO |
| √ | | | √ | √ | | √ | √ | √ | √ | | | √ | | | √ | ACO, VM Consolidation |
| √ | | √ | | √ | | | √ | √ | | | | √ | | | √ | ACO, VM Consolidation |
| √ | | | √ | √ | | | √ | | | | | √ | | √ | | ACO, VM Consolidation, On/Off Switching. |
| √ | | √ | | √ | | | √ | √ | | √ | | √ | | | √ | ACO, VM Consolidation, On/Off Switching. |
| √ | | | √ | √ | | √ | √ | | | | | √ | | | √ | PSO |
| √ | | √ | | √ | | | √ | | | | | √ | | | √ | GA |
| √ | | √ | | √ | √ | | √ | | | | | √ | | | √ | GA |
| √ | | √ | | √ | | √ | √ | | | √ | | √ | | | √ | GA |