A Joint Caching and Offloading Strategy Using Reinforcement Learning for Multi-access Edge Computing Users

The increasing adoption of UHD video applications in connected car mobile terminals is placing significant demands on the real-time processing capabilities of in-vehicle devices. Previous studies have proposed a combination of caching and offloading techniques to ensure efficient processing at the edge network. However, these studies often handle edge caching and computation offloading decisions concurrently, which adds complexity to the decision-making process and increases the update overhead of the pre-cache. In this paper, we present a novel joint control scheme that establishes a dual time-scale resource management framework for cache updates and offload scheduling. In the first phase, the edge pre-cache strategy is executed in a longer time scale, and we employ a DQN-based edge cache service replacement algorithm to ensure that local pre-cache services can be hit after offload requests. In the second phase, resource offload scheduling occurs in shorter time slots, and we guarantee low latency using a DDPG-based intra-domain computational resource allocation algorithm. Simulation results demonstrate that our proposed scheme effectively maintains a high hit rate of requests without frequent updates of the intra-domain cache, thereby reducing the overall latency of task processing. Compared to the benchmark scheme without intra-domain collaboration, our proposed scheme achieves a 50% reduction in average total processing latency and a 20% improvement in average resource utilization.