Data Caching Based Transfer Optimization in Large Scale Networks

With the tremendous increased scale of networks, data transferring plays a very important role in high performance computing. Traditionally, when a task is completed, the data associated with the task will be released to free the space for the following tasks. However, the following task may require the same data, which will be fetched back and the efficiency will be affected too. In this paper, we implement a data caching method so as to improve the performance of the system. When a task is completed, the data will be stored for a longer time. If the previous used data is requested before releasing, there will be no transferring cost. With the help of data caching, the performance of the network in handling real-time tasks is significantly improved while the traffic jamming is also decreased. We extend the previous single data model to multiple type of data among servers. Tasks arrive in an online fashion, and when handling a task on some network server, we have no information about the future tasks. Assuming the storage cost is submodular, which is well adopted in reality, we propose an online algorithm to minimize the total transfer and caching costs of the system. Via theoretical analysis, the competitive ratio of the algorithm is proved to be a constant.