OMBM-ML: efficient memory bandwidth management for ensuring QoS and improving server utilization

As cloud data centers are dramatically growing, various applications are moved to cloud data centers owing to cost benefits for maintenance and hardware resources. However, latency-critical workloads among them suffer from some problems to fully achieve the cost-effectiveness. The latency-critical workloads should show latencies in a stable manner, to be predicted, for strictly meeting QoSs. However, if they are executed with other workloads to save the cost, they experience QoS violation due to the contention for the hardware resources shared with co-location workloads. In order to guarantee QoSs and to improve the hardware resource utilization, we proposed a memory bandwidth management method with an effective prediction model using machine learning. The prediction model estimates the amount of memory bandwidth that will be allocated to the latency-critical workload based on a REP decision tree. To construct this model, we first collect data and train the model with the data. The generated model can estimate the amount of memory bandwidth for meeting the SLO of the latency-critical workload no matter what batch processing workloads are collocated. The use of our approach achieves up to 99% SLO assurance and improves the server utilization up to 6.8\(\times\) on average.