Lightweight Energy-and Link-Aware Q-Learning Routing for Underwater Acoustic Sensor Networks

Abstract

The paper presents LEARQ (Lightweight Energy-and Link-Aware Q-Learning Routing) an adaptive, reinforcement-learning-based routing protocol that is tailored to Underwater Acoustic Sensor Networks (UASNs). The protocol involves the combination of the residual energy measurements, acoustic link quality (measured by signal-to-noise ratio) and depth gradient to dynamically pick the best next-hop node using a distributed Q-learning system. An underwater energy consumption model is realistic and includes both spreading and absorption losses. The protocol gets tested by comparing it with traditional techniques, that is, Depth-Based Routing (DBR), and Vector-Based Forwarding (VBF) through a large-scale MATLAB simulation. The outcomes of twelve performance studies have been conducted, such as the ratio of packet delivery, power consumption, network lifetime, dead node development, energy distribution, visualization of heat-map, packet loss rate, routing drop rate, and energy savings, which prove that LEARQ is always better than both DBR and VBF. Precisely, LEARQ results in a 1720% faster increase in the delivery of packets, a 22% lower energy consumption, over 25 percent decrease in the packet loss and drop rates, and a 28 percent longer network lifetime. Besides, its energy distribution is more even among the network nodes, thus reducing the early termination of nodes. All these results support the conclusion that LEARQ is a feasible and scalable solution to underwater acoustic routing in real-time, energy-saving, and reliable mode.