Deep reinforcement learning mechanism for deadline-aware cache placement in device-to-device mobile edge networks

Caching the most likely to be requested content at the mobile devices in a cooperative manner can facilitate direct content delivery without fetching content from the remote content server and thus alleviate the user-perceived latency, reduce the burden on backhaul and minimize the duplicated content transmissions. In addition to content popularity, it is also essential to consider the users’ dynamic behaviour for real-time applications, which can further improve communication between user devices, leading to efficient content service time. Most previous studies consider stationary network topologies, in which all users remain stationary during data transmission, and the user can receive desired content from the corresponding base station. In this work, we study an essential issue: caching content by taking advantage of user mobility and the randomness of user interaction time. We consider a realistic scenario in a cooperative caching problem with user devices moving at various velocities. We formulate the cache placement problems as maximization of saved delay with capacity and deadline constraints by considering the contact duration and inter-contact time among the user devices. A deep reinforcement learning-based caching scheme is presented to solve the high dimensionality of the proposed Integer linear programming problem. The proposed caching schemes improve the long-term reward and higher convergence rate than the Q-learning mechanism. Extensive simulation results demonstrate that the proposed cooperative caching mechanism significantly improves up to 23 % on hit ratio, 24 % on acceleration ratio and 25 % on offloading ratio compared with existing mechanisms.