High Performance Computing in Power and Energy Systems

This chapter presents a review of the main application areas of high performance computing in electrical energy systems and introduces some results obtained with the implementation of parallel computing to some relevant problems encountered in the planning, operation and control of such systems. The research topics described in the paper are optimization of hydrothermal systems operation, tunning of multiple power system stabilizers, security constrained optimal power flow, composite reliability evaluation, dynamic security assessment, and distribution network reconfiguration. Most of the work described includes results obtained with tests conducted using actual models of electrical energy systems.

High-speed extended term (HSET) time domain simulation (TDS) is intended to provide very fast computational capability to predict extendedterm dynamic system response to disturbances and identify corrective actions. The extended-term dynamic simulation of a power system is valuable because it provides ability for the rigorous evaluation and analysis of outages which may include cascading. It is important for secure power grid expansion, enhances power system security and reliability, both under normal and abnormal conditions. In this chapter the design of the envisioned future dynamic security assessment processing system (DSAPS) is presented where HSETTDS forms the core module. The power system is mathematically represented by a system of differential and algebraic equations (DAEs). These DAEs arise out of the modeling of the dynamic components such as generators, exciters, governors, automatic generation control, load tap changers, induction motors, network modeling and so on. To provide very fast computational capability within the HSET-TDS, this chapter motivates the need for high performance computing (HPC) for power system dynamic simulations through detailed modeling of power system components and efficient numerical algorithms to solve the resulting DAEs. The developed HSET-TDS is first validated for accuracy against commercial power simulators (PSSE, DSA Tools, Power- World) and then it is compared for computational efficiency. The chapter investigates some of the promising direct sparse linear solver for fast extended term time domain simulation and makes recommendation for the modern power grid computations. The results provide very important insights with regards to the impact of the different numerical linear solver algorithms for enhancing the power system TDS.

The information technology (IT) world has changed fundamentally and drastically from running software applications on a single computer with a single CPU to now running software as services in distributed and parallel computing environment. Power system operation has been also shifted from being solely based on off-line planning study to more and more real-time market-driven. Facing these challenges, we will discuss in this chapter how to design power system analysis and simulation software to take advantage of the new IT technologies and to meet the real-time application requirements, using the InterPSS project as a concrete example.

To meet the requirement of dispatch and control in Smart Grid environment, a MAS-Based cluster computing platform is developed for modern Energy Management System(EMS). Multi-Agent technology is used to develop the high-performance computing platform. “Double-Dimensional Federal Architecture” is proposed and used to implement the advanced application software of modern EMS. A middleware named “WLGrid” is implemented to simplify software development and to integrate distributed computers in Electric Power Control Center(EPCC) into a “Super Computer”. Methods of “Task Partition”, “Concurrent Dispatching Strategy”, “Dynamic Load Balancing” and “Priority-based Resource Allocation” are introduced to accelerate the computation and to improve the real-time property of the whole system. Onsite tests are done and the results are reported to show the effectiveness of these methods.

Power system computation software tools are traditionally designed as serial codes and optimized for single-processor computers. They are becoming inadequate in terms of computational efficiency for the ever increasing complexity of the power grid. The power grid has served us remarkably well but is likely to see more changes over the next decade than it has seen over the past century. In particular, the widespread deployment of renewable generation, smart-grid controls, energy storage, plug-in hybrids, and other emerging technologies will require fundamental changes in the operational concepts and principal components of the grid. The grid is in an unprecedented transition that poses significant challenges in power grid operation. Central to this transition, power system computation needs to evolve accordingly to provide fast results for power grid management.

On the other hand, power system computation should and has to take advantage of ubiquitous parallel computers. To bring HPC to power grid applications is not simply putting more computing units against the problem. It requires careful design and coding to match an application with computing hardware. Sometimes, alternative or new algorithms need to be used to maximize the benefit of HPC.

This chapter demonstrates the benefits of HPC for power grid applications with several examples such as state estimation, contingency analysis, and dynamic simulation. These examples represent the major categories of power grid applications. Each of the applications has its own problem structure and data dependency requirements. The approach to apply HPC to these problems has different challenges. The HPC-enhanced state estimation, contingency analysis, and dynamic simulation presented in this chapter are suitable for today’s power grid operation.

Power system simulations involving solution of thousands of stiff differential and algebraic equations (DAE) are extremely computationally intensive and yet crucial for grid security and reliability. Online simulation of minutes to hours for a large number of contingencies requires computational efficiency several orders of magnitude greater than what is todays state-of-the-art. We have developed an optimized simulator for single contingency analysis using efficient numerical algorithms implementation for solving DAE, and scaled it up for large-scale contingency analysis using MPI. A prototype parallel high speed extended term simulator (HSET) on in-house high performance computing (HPC) resources at Iowa State University (ISU) (namely Cystorm Supercomputer) is being developed. Since the simulation times across contingencies vary considerably, we have focused our efforts towards development of efficient scheduling algorithms through work stealing for maximal resource utilization and minimum overhead to perform faster than real time analysis. This chapter introduces a novel implementation of dynamic load balancing algorithm for dynamic contingency analysis. Results indicate potential for significant improvements over the state-of-the-art methods especially master-slave based load balancing typically used in power system community. Simulations of thousands of contingencies on a large real system were conducted and computational savings and scalability results are reported.

This chapter reviews designs and prototypes of reconfigurable hardware implemented on a Field Programmable Gate Array (FPGA) to speedup ubiquitous linear algebra subroutines used in system security analysis. The grid operators use Energy Management System (EMS) software to analyze system security to assure normal operating state. EMS consists of three main computations: 1) state estimation, 2) contingency analysis, and 3) optimal power flow. These computations involve sparse linear algebra algorithms such as matrix orthogonal (QR) decomposition, Lower-Upper (LU) decomposition and matrix multiplication. Currently, EMS computations are performed on a general-purpose processor system. A benchmark study of several state-of-the-art sparse linear solver packages running on these systems reveals inefficient utilization of the floating-point computational throughput. A custom hardware sparse linear solver that maximizes floating-point hardware utilization based on pipeline architecture and efficient data caching offers an alternative. A prototype on reconfigurable hardware demonstrated that despite more than an order of magnitude deficit in clock speed as compared to general purpose processor based systems, a specialized sparse LU hardware running on FPGA is capable of an order of magnitude speedup relative to these systems for power system Jacobian matrix sparse LU decomposition. Performance analysis of sparse QR decomposition hardware showed a similar potential speedup over general-purpose processors.

Programmable graphics processing units (GPUs) currently offer the best ratio of floating point computational throughput to price for commodity processors, outdistancing same-generation CPUs by an order of magnitude, which has in turn led to their widespread adoption in a variety of computationally demanding fields. Adapting power system simulations to these processors is complicated by the unique hardware architecture of GPUs, which precludes the usage of direct linear system solvers based on Gaussian elimination. Krylov subspace methods are better suited to the GPU architecture, yet the ill-conditioned nature of power system matrices requires substantial preconditioning to ensure robustness of these methods. To reduce the time spent on preconditioning, we have developed a GPU-based preconditioner designed specifically to handle the large, sparse matrices typically encountered in power system simulations. The preconditioning technique used, based on Chebyshev polynomials, is described in detail, as are the design decisions made in adapting this algorithm to the GPU. Evaluation of the performance of the GPU-based preconditioner on a variety of sparse matrices, ranging in size from 30 x 30 to 3948 x 3948, shows significant computational savings relative to a CPU-based implementation of the same preconditioner and a typical incomplete LU (ILU) preconditioner.

Reference Network Models (RNMs) are large-scale distribution network planning tools. RNMs can be used by policy makers and regulators to estimate efficient distribution costs. This is a very challenging task, particularly being network planning a combinatorial problem, which is especially difficult to solve due to the vast size of the distribution areas, and the use of several voltage levels. This chapter presents the main features of RNMs developed by the authors, including high performance requirements related to the type and size of the problem. The model can be used to plan distribution networks either from scratch or incrementally from existing grids. Different case studies illustrate the applicability of these models to the assessment of the impact of massive deployment of renewable distributed generation, demand response actions, and plug-in electric vehicle penetration on distribution costs. The results obtained provide valuable information to guide strategic policy-making decisions regarding the implementation of renewable energy programs and smart grid initiatives.

Electricity consumer demand response and load control are playing an increasingly important role in the development of a smart grid. Smart grid load management technologies such as Grid Friendly^TM controls and real-time pricing are making their way into the conventional model of grid planning and operations. However, the behavior of load both affects, and is affected by load control strategies that are designed to support electric grid planning and operations. This chapter discussed the natural behavior of electric loads, how it interacts with various load control and demand response strategies, what the consequences are for new grid operation concepts and the computing issues these new technologies raise.

This paper describes an effective methodology for on-line screening and ranking of a large set of contingencies. An evaluation study of the on-line methodology in a real-time environment as a transient stability analysis (TSA) screening tool is presented. Requirements for an on-line screening and ranking tools are presented. The methodology of BCU classifiers implemented in the TEPCO-BCU package was evaluated on the PJM system as a fast screening tool to improve on-line performance of the PJM TSA system. This evaluation study is the largest in terms of system size, 14,500-bus, 3000 generators, for a practical application of direct methods for on-line TSA. The total number of contingencies involved in this evaluation is about 5.3 million. The evaluation results were very promising and confirm the practicality of the methodology based on direct methods, in particular the BCU method for on-line TSA of large-scale systems with a large set of contingencies.

Development and application of SmartGrids or Intelligrids, including roll-out of smart meters and electrical vehicles, is of a great importance if the UK and other countries are to achieve significant carbon emission reductions and realize sustainable energy systems. These new grids will offer the opportunity to increase the level of renewable energy integrated into the system. They will also allow customers, including small households, to actively participate and adjust their demand depending on energy availability and price. This will further lead towards improved energy efficiency, as well as offer possibilities to reduce overall consumption and reduce or postpone investments into new large generation and infrastructure facilities.

To achieve these goals, a number of technical, economical and policy issues need to be addressed and resolved. The development of new generations of extremely fast software tools that can solve power system problems with large number of nodes will also be important to help resolve these issues. For example, distribution system and network operators, as well as trading entities such as aggregators, will get a better coordination of system operation though the possibility to engage with even smaller generators, and especially smaller customers. This control at lower voltage levels will allow for the aggregation of responses which will then propagate to higher voltage levels.

Currently, the discussion regarding the operation of future power systems is looking into two different options. One is to develop methodologies that will allow decentralization of network operation with the reduced level of coordination at the high level of system operation. However, the new software developed to exploit the benefits of the HPC architecture may open a possibility for businesses and policy makers to investigate and compare operation of centralized vs. decentralized operation over areas with large number of participants.. These new HPC power system analysis tools will enable more frequent price signal calculations and bring the possibility to define policies which will ensure engagement with customers to reduce their energy consumption or shift it towards offpeak periods, as well as allow for the coordination of charging of electric vehicles and their use as storage devices. Such tools will be useful for both decentralized and centralized operation, however they will be crucial for the latter.

This chapter will first give an overview of the changes in future power system operation and then outline power system analysis tools such as power flow, optimal power flow, generation scheduling and the security assessment. It will then discuss current status of the parallel techniques and HPC applications for the power system operation tools. It will also discuss the formulation requirements, achievements and possible obstacles in the application of techniques suitable for HPC and for power system operation problems such as power flow, optimal power flow (OPF), security constrained OPF. Finally, it will look how new developments in the HPC/Numerical Analysis area, and even more powerful Extreme Computing together with new algorithms developed for this next-step class of machines may help improve power system operation and electricity markets tools.