Fast, energy-efficient electronic structure simulations for multi-million atomic systems with GPU devices

We report that speed and energy efficiency of large-scale electronic structure simulations, which target realistically sized systems consisting of multi-million atoms, can be hugely improved with offload-computing using graphics processing unit (GPU) devices. For simulations of quantum dot devices that have \(\sim \) 1.5 million atoms and are described by a \(sp^3d^5s^*\) tight-binding (TB) model, a remarkable performance enhancement is obtained with asynchronous sharing of computing load in host CPUs and Tesla K40 devices. Compared to the case when only host CPUs are used, sparse matrix-vector multiplications, the core operation needed to solve Schrödinger equations, become remarkably faster leading \(\sim \) 1.5\(\times \) speed-up of end-to-end simulations with GPU devices. Asynchronous streams accelerate data-transfer reducing the associated overhead below \(\sim \) 15% of the total wall-time. The speed and energy efficiency of TB simulations also turn out to be better with Tesla GPU devices than those obtained with Intel Xeon Phi Knights Corner (KNC) coprocessors, such that Tesla K40 GPU devices save \(\sim \) 10% of the wall-time and \(\sim \) 40% of the total energy consumed with KNC 7120 coprocessors for the target simulation. With technical details of offload-computing that can be also applied to accelerate other numerical problems involving large-scale sparse matrix operations, this work delivers practical information regarding the efficiency of GPU computing that has not been well covered for empirical modeling of large-scale electronic structures.