A Reinforcement-Learning Based Cognitive Scheme for Opportunistic Spectrum Access

Cognitive Radio enables secondary users (SUs) to access communication channels allocated to primary users (PUs). As prior knowledge of the channel characteristics is not available in practice, the SUs attempting to get communication access in a geographical area must act autonomously and fast in order to detect vacant communication channels. Addressing the SUs need for autonomous operation, this article proposes a reinforcement learning scheme that determines the sensing order of the available channels employing two alternative update rules. Under both alternative options, the SUs operate as independent agents processing information acquired solely from their own sensing mechanisms in order to assess the channels with respect to (i) the occupancy probability and (ii) the mean duration of vacant periods. The scheme capability of accurately estimating the various channel characteristics without any prior knowledge of the traffic pattern followed by the PUs is thoroughly investigated with regard to critical performance metrics in both static and dynamic transmission environments. The proposed scheme is compared with two existing channel selection schemes. The simulations show that the proposed scheme manages to prioritize channel selection according to the channel characteristics and that it outperforms both schemes under comparison in terms of channel utilization and energy efficiency.