Reinforcement Learning-Based Routing in Underwater Acoustic Sensor Networks

Underwater acoustic sensor networks (UASNs) play an essential role in exploring the marine world applications in rivers, ponds, and oceans. In recent years, prominent developments have addressed and resolved the numerous issues of critical underwater applications. Routing is one of the vital problems discussed widely, and various protocols are designed to solve the routing-related issues effectively. In this paper, we propose reinforcement-learning based routing using Steiner-points to minimize energy consumption and enhance the network lifetime of UASNs. Initially, the sensor nodes form a group of three nodes and calculate the Steiner point for each such group. The isolated nodes and the Steiner points in the network are vertices of the Steiner tree, which is the routing path to be constructed dynamically by the sink node agent to travel with the assistance of AUV. The sink node agent in the unknown environment initiates the Steiner tree construction process by visiting either the isolated neighbor node or neighbor sensor node’s Steiner point to collect the data from sensor nodes based on the reward function and the environmental conditions. If agent visits the isolated node, then it collects the data from that node itself. If the agent visits the Steiner point, the agent broadcasts its arrival through the beacon message and intended members of that group communicate to the agent and forwards the sensed data. The agent processes the data to remove the data’s redundancy and selects the next point to be visited and constructs the Steiner tree dynamically. The performance evaluation of the proposed scheme minimized each sensor node’s energy consumption and enhanced the network’s lifetime.