Anti-jamming transmission in softwarization UAV network: a federated deep reinforcement learning approach

This paper presents the concept and studies of federated deep reinforcement learning (DRL) based anti-jamming communication in softwarization UAV network (SUNET). First, task-driven SUNET framework is presented and joint beamforming and power control (JBPC) based multidomain anti-jamming model is built to balance the spectrum efficiency (SE) and energy efficiency (EE) of the SUENT. Then a weighted dueling DQN (wDDQN) learning algorithm with upper confidence bound (UCB) action exploration is provided to handle the formulated model. Further, we propose federated wDDQN-UCB (F-wDDQN) based JBPC anti-jamming strategy to tackle the challenge of agent training would consume a large of communication resources, and design the link state aware quantity of service (LSAQ) routing of SUNET to reduce transmission delay of the model parameters during F-wDDQN training. Simulation results validate that the spectrum-energy efficiency and convergence performance achieved by the F-wDDQN based JBPC anti-jamming strategy is superior to existing DRL methods, and the LSAQ routing is benefit to reduce the transmission delay of the F-wDDQN learning strategy and accelerate its convergence.