Joint design of routing selection and user association in multi-hop mmWave IABN: a multi-agent and double DQN framework

The increasing demand for high-speed data services from users has promoted the exploration of high-frequency spectrum resources, and the introduction of millimeter wave (mmWave) technology provides enormous bandwidth resources for the development of 5 G and future networks. Although the densely deployed multi-hop mmWave integrated access and backhaul network (IABN) has reduced operation costs and improved spectrum utilization through efficient design, the intensification of network interference and the complexity of resource management have also been followed. Especially, how to maintain a balance of rate between the access and backhaul parts is crucial for improving the overall network throughput. Therefore, this paper investigates the joint optimization problem of routing selection and user association in multi-hop mmWave IABN. This formulated problem is essentially a mixed integer nonlinear programming (MINLP) problem under consideration of the dynamic changes in the IABN environment. Deep reinforcement learning has become a emerging method for handling complex decision problems. To this end, we propose an improved multi-agent double deep Q-network based joint optimization (MDDQJO) scheme of routing selection and user association, which aims to maximize the end-to-end throughput of the multi-hop mmWave IABN. The MDDQJO scheme adopts a load based spectrum allocation strategy to adaptively meet the traffic requirements of different nodes, and dynamically optimizes routing selection and user association decisions through parallel processing and distributed training mechanisms of all agents to respond to real-time changes in the network environment. Finally, through experimental verification, the scheme not only improves the system spectrum utilization and user service quality, but also significantly enhances the link transmission efficiency.