8. Parallel Computing Considerations

This chapter discusses the computational cost of machine learning model. To reduce its training time is a requisite of its industrial applications since a production process usually requires real-time responses. The commonly used method to accelerate the training process is to develop a parallel computing framework. In literature, two kinds of popular methods speeding up the training involves the one with a computer equipped with graphics processor unit (GPU) and the one with computer cluster including a number of computers. This chapter firstly introduces the basic ideas of GPU acceleration (e.g., the compute unified device architecture (CUDA) created by NVIDIA™) and the computer cluster framework (e.g., the MapReduce framework), then gives some specified examples of them. When training an EKF-based Elman network, the inversion operation of a Jacobian matrix is the most time-consuming procedure; a parallel computing strategy for such an operation is therefore proposed by using the CUDA-based GPU acceleration. Besides, with regard to the LSSVM modeling, a CUDA-based parallel PSO is then introduced for its hyper-parameters optimization. As for the computer cluster version, we design a parallelized EKF based on ESN by using MapReduce framework for acceleration. At the end, we also present a series of experimental analysis by using the practical energy data in steel industry to validate the performance of the accelerating approaches.