A Q-learning approach for machine-type communication random access in LTE-Advanced

Due to the wide proliferation of the 3GPP long term evolution (LTE) and LTE-Advanced systems as the air interface for 4th generation (4G) wireless systems and beyond, telecommunication operators became more interested in using the LTE infrastructure to meet the projected surge in demand for machine-to-machine (M2M) and IoT communications. As a result, the LTE-Advanced system must evolve to provide these services as an overlay over the original network that is originally designed for human-to-human (H2H) communication. In this setup, M2M communication devices share the same random access channel (RACH) with the H2H communication devices. Since M2M communication is expected to be massive, consequently the RACH is a new bottleneck in the LTE-A system. This triggered the need to enhance the operation of the RACH to meet the needs to support the low power and low data rates of M2M devices. The current standardized scheme to request access to the system is known to suffer from congestion and overloading in the presence of a huge number of devices. Accordingly, recent research pointed to the need of designing more efficient ways to manage the random access channel in such setups. Most of previous works focused on solving the congestion problem in RACH in the presence of M2M solely, but not with the existence of both M2M and H2H services. In this work we propose a new random access channel scheme based on Q-learning approach to reduce the congestion problem. The scheme adaptively divides the available preambles between both M2M and H2H devices in a way that provides an acceptable service for the H2H devices and maximizes the number of active M2M devices. The adaptation is done based on current demand levels from both the H2H and M2M devices and the observed service levels. The results indicate that the proposed approach provides high random access channel success probability for both M2M and H2H devices even with the huge number of M2M devices.