Actor-critic deep learning for efficient user association and bandwidth allocation in dense mobile networks with green base stations

In this paper, we introduce an efficient user-association and bandwidth-allocation scheme based on an actor-critic deep learning framework for downlink data transmission in dense mobile networks. In this kind of network, small cells are densely deployed in a single macrocell, and share the same spectrum band with the macrocell. The small-cell base stations are also called green base stations since they are powered solely by solar-energy harvesters. Therefore, we propose an actor-critic deep learning (ACDL) algorithm for the purpose of maximizing long-term network performance while adhering to constraints on harvested energy and spectrum sharing. For this purpose, the agent of the ACDL algorithm tries to obtain an optimal user-association and bandwidth-allocation policy by interacting with the network’s environment. We first formulate the optimization problem in this paper as a Markov decision process, during which the agent learns about the evolution of the environment through trial and error experience. Then, we use a deep neural network to model the policy function and the value function in the actor and in the critic of the agent, respectively. The actor selects an action based on the output of the policy network. Meanwhile, the critic uses the output of the value network to help the actor evaluate the taken action. Numerical results demonstrate that the proposed algorithm can enhance network performance in the long run.