Energy Efficient Thermal Management of Data Centers

This chapter provides an introduction to the emerging trends in the growth of data centers. It is seen that projected growth in functionality and performance in information technology and communications equipment is resulting in sharply increasing power dissipation per unit facility footprint. Increased energy usage for powering and thermal management of data centers, which is a potential concern for continued growth of data centers is examined. The guidelines for environmental control of data centers to assure satisfactory equipment performance are discussed. Thermal management of data centers involves multiple length scales. The approaches currently in use and under exploration at various scales are presented. Thermal modeling approaches for data centers are also discussed. Data centers are generally expected to operate continuously, yet there have been documented failure events that have lead to usage interruption. The tier classification of data centers based on redundancy is introduced.

Airflow management is probably the most important aspect of data center thermal management. It is an intricate and challenging process, influenced by many factors. This chapter presents some of the fundamental concepts governing airflows in today’s data centers. As such, it provides a foundation necessary for understanding the remaining topics discussed in the book. The chapter begins by introducing the concept of system pressure drop and its influence on the computer room air conditioning (CRAC) unit performance. Various factors contributing to the overall pressure drop, such as plenum design, perforated tile open area, and aisle layouts, are described. Some of the key aspects of room and rack airflows are also discussed. The second part of the chapter highlights the importance of temperature and humidity control in data centers. The basic concepts of psychrometrics are introduced. Specific examples on data center cooling processes, such as sensible cooling, humidification/dehumidification and evaporative cooling are illustrated with the help of the psychrometric chart. The concept of airside and waterside economizers for data center cooling are introduced. The third and final part of the chapter describes an ensemble COP model, for assessing the overall thermal efficiency and performance of the data center.

As the concept of cloud computing is gaining popularity, more data centers are built to support the needs. The data centers, which have consumed 1.5% of the total electrical energy generated in the USA in 2006, are paying the majority of their maintenance cost to the electricity bills. Reducing power consumption in the data centers is now a must not only for seizing sustainable development but also for preserving our planet green. Along the effort of building power-efficient data centers, this chapter will start by answering the ultimate question—where did the power go? By taking a top–down approach from the data center level all the way down to the microarchitectural level, this chapter visualizes the power breakdowns and discusses the power optimization techniques for each layer.

The continuing, unsustainable increase in datacenter power consumption is causing researchers in industry and academia to be heavily invested in addressing power management challenges. This chapter presents the basic elements of a measurement-based approach toward managing distributed datacenter and cloud computing systems to meet both application and end-user needs and to obtain improved efficiency and sustainability in their operation. The main components of the approach presented include (1) continuous online monitoring, measurement and assessment of systems and applications behaviors and power consumption, including for online estimation of the power usage of virtual machines running application components in these virtualized systems; (2) the ability to perform these tasks efficiently at scale, so as to deal with the ever-increasing sizes and complexity of modern datacenter infrastructures; and (3) the importance of “coordinated” management methods that operate across multiple levels of abstraction and multiple layers of the management stack in an orchestrated manner.

Since the early days of data centers, the data center operators have been challenged with device placement, capacity planning, equipment maintenance, and failure & downtime. Lately, things have become even harder for them due to the spotlight on low energy efficiency and low utilization of available capacity. The idea is that more needs to be done with less, while maintaining the same level of service. While piecemeal and inadequate attempts have been made from time to time, to address some of these challenges, the key to solving all of these problems lies in combining IT and facility monitoring, or even just facility monitoring. Therefore, in this chapter, first we briefly discuss some of the aforementioned issues. Then we get into the details of what physical quantities should be monitored in a data center and what value that can bring. We focus on the cooling distribution chain and the power distribution chain. Since power monitoring is a very popular topic and a major portion of it deals with power quality, a separate section has been dedicated on brief explanations of different power quality issues. Finally, we discuss the requirements for the software infrastructure that is needed to support this type of holistic monitoring.

In this chapter, metrics for measuring and improving data center efficiencies are explored. Metrics at varying levels from the infrastructure components to the entire data center are reviewed. The primary data center efficiency metric, PUE is discussed at length as well as variants of PUE. The challenges of defining a metric around computing output or data center useful work are also considered. The chapter includes discussions on a variety of other related topics such as codes, standards, and rating systems. Through a thorough review of the chapter, the reader will also gain a strong insight into some of the fundamental issues in data center design and operations.

This chapter describes data center measurement systems and supporting modeling methods. The first part concerns techniques and systems for taking relevant physical measurements to characterize the environmental conditions in data centers. This includes a brief discussion about how design choices, for example, sensor placement, sensor density, and measurement frequency, depend on the supporting modeling approach. Wireless and wired sensing solutions and the role of internal and external sensors are addressed as well. The second part of the chapter deals with how these measurements can be utilized as inputs for subsequent heat transfer modeling in data centers. Two different modeling approaches are discussed, namely a simplified physics-based model (Laplacian model), where the measurements are used to provide the required boundary data, and a reduced order modeling approach using proper-orthogonal decomposition. Case studies for these two different techniques are presented.

This chapter deals with the numerical modeling of data centers. The chapter presents an overview of the fundamental equations governing the conservation of mass, energy, and momentum, with an emphasis on the most widely used numerical approaches used for discretizing the equations and solving them. The specific simplifications and assumptions that are typically used in modeling data centers are reviewed. Turbulent modeling is covered in some detail, with an emphasis on the suitability of different models for data centers. A review of recent numerical studies of data centers is presented and compared to available measurements and characterization studies. Results for different air cooling protocols are presented and ranked according to their overall performance. A detailed discussion of the impact of blockages in the plenum, due to wiring and cooling water pipes, is presented and general design guidelines are made pertaining to placement of such blockages. Specific attention is given to the modeling of data centers during dynamic fluctuations in power, airflow, and temperature. This is of particular relevance for the establishment of dynamic self-regulating data centers that may be optimized to operate at the lowest possible energy level while they are meeting specific performance metrics. A case is made for verified reduced order modeling of dynamic data centers. Such an approach may be the most suitable and pragmatic one to achieve real-time holistic models that are capable of predicting and optimizing the overall performance of complex data centers.

Data center thermal management systems exist to maintain the computer equipment within acceptable operating temperatures. As power densities have increased in data centers, however, the energy used by the cooling infrastructure has become a matter of growing concern. Most existing data center thermal management metrics provide information about either the energy efficiency or the thermal state of the data center. There is a gap around a metric that fuses information about each of these goals into a single measure. This chapter addresses this limitation through an exergy analysis of the data center thermal management system. The approach recognizes that the mixing of hot and cold streams in the data center airspace, which is often a primary driver of thermal inefficiency in the data center, is an irreversible process and must therefore lead to the destruction of exergy. Experimental validation in a test data center confirms that such an exergy-based characterization in the cold aisle reflects the same recirculation trends as suggested by traditional temperature-based metrics. Further, by extending the exergy-based model to include irreversibilities from other components of the thermal architecture, it becomes possible to quantify the amount of available energy supplied to the cooling system which is being utilized for thermal management purposes. The energy efficiency of the entire data center cooling system can then be collapsed into the single metric of net exergy consumption. When evaluated against a ground state of the external ambience, this metric enables an estimate of how much of the energy emitted into the environment could potentially be harnessed in the form of useful work. The insights availed from the above analysis include a wide range of considerations, such as the viability of workload placement within the data center; the appropriateness of airside economization as well as containment; the potential benefits of reusing waste heat from the data center; as well as the potential to install additional compute capacity without needing to increase the data center cooling capacity. In addition, the analysis provides insight about how local thermal management inefficiencies in the data center can be mitigated. The chapter concludes by suggesting that the proposed exergy-based approach can provide a foundation upon which the data center cooling system can be simultaneously evaluated for thermal manageability and energy efficiency.

In this chapter, the sustainable and reliable operations of the electronic equipment in data centers are shown to be possible through a reduced order modeling based design. First, the literature on simulation-based design of data centers using computational fluid dynamics/heat transfer (CFD/HT) and low-dimensional modeling are reviewed. Then, two recent proper orthogonal decomposition (POD) based reduced order thermal modeling methods are explained to simulate multiparameter-dependent temperature field in multiscale thermal/fluid systems such as data centers. The methods result in average error norm of ~6% for different sets of design parameters, while they can be up to ~250 times faster than CFD/HT simulation in an iterative optimization technique for a sample data center cell. The POD-based modeling approach is applied along with multiobjective design principles to systematically achieve an energy efficient, adaptable, and robust thermal management system for data centers. The framework allows for intelligent dynamic changes in the rack heat loads, required cooling airflow rates, and supply air temperature based on the actual momentary center heat loads, rather than planned occupancy, to extend the limits of air cooling and/or increase energy efficiency. This optimization has shown energy consumption reduction by 12–46% in a data center cell.

A data center is an integrated IT system housing multiple-unit servers intended for providing various application services. A significant portion of the costs associated with operating and maintaining a data center is used for heat removal. A growing trend in the IT industry is to use computer experiments to study thermal properties of data centers because the corresponding physical experimentation can be time consuming, costly, or even infeasible. This chapter presents useful statistical methods for the design and analysis of data center computer experiments.

Cooling of data centers is estimated to have an annual electricity cost of 1.4 billion dollars in the USA and 3.6 billion dollars worldwide. Currently, refrigerated air is the most widely used means of cooling data center’s servers. According to recent articles published at the ASHRAE Winter Annual Meeting at Dallas, typically 40% or more of the refrigerated airflow bypasses the server racks in data centers. The cost of energy to operate a server for 4 years is now on the same order as the initial cost to purchase the server itself, meaning that the choice of future servers should be evaluated on their total 4-year cost, not just their initial cost. Based on the above issues, thermal designers of data centers and server manufacturers now seem to agree that there is an immediate need to improve the server cooling process, especially considering that modern data centers require the dissipation of 5–15 MW of heat, and the fact that 40–45% of the total energy consumed in a data center is for the cooling of servers. Thus, the manner in which servers are cooled and the potential of recovery of the dissipated heat are all more important, if one wishes to reduce the overall CO₂ footprint of the data center. Recent publications show the development of primarily four competing technologies for cooling chips: microchannel single-phase (water) flow, porous media flow, jet impingement cooling and microchannel two-phase flow. The first three technologies are characterized negatively for the relatively high pumping power to keep the temperature gradient in the fluid from inlet to outlet within acceptable limits, i.e., to minimize the axial temperature gradient along the chip and the associated differential expansion of the thermal interface material with the silicon created by it. Two-phase flow in microchannels, i.e., evaporation of dielectric refrigerants, is a promising solution, despite the higher complexity involved. The present chapter presents the thermo-hydrodynamic fundamentals of such a new green technology. Two potential cooling cycles making use of microchannel evaporators are also demonstrated. A case study was developed showing the main advantages of each cycle, and a comparison between single-phase (water and brine) and two-phase (HFC134a and HFO1234ze) cooling is given. Finally, an additional case study demonstrating a potential application for the waste heat of data centers is developed. The main aspects considered were reduction of CO₂ footprint, increase of efficiency (data centers and secondary application of waste heat), and economic gains.

This chapter introduces a number of emerging topics in data center design and operation. The use of ambient air, water, or ground for heat rejection is attractive for many facilities whenever the environmental conditions are conducive. Changes in equipment layout and real-time control of cooling and information technology (IT) resources also offer opportunities for savings in cooling energy costs. As rack level powers continue to increase for several equipment classes, there is increasing interest in hybrid liquid/air cooling and liquid cooling approaches. Safety issues arising due to elevated temperatures and ambient noise are also receiving increasing attention. Broadening of equipment operation temperature and humidity ranges is resulting in concerns for wiring corrosion. Need for on-demand, rapidly deployable, and expandable computing resources has resulted in rapid development of modular data centers.