Soft actor-critic with automatically adjusted entropy for autonomous exploring in unknown environments

Abstract

This paper presents an in-depth analysis of the soft actor-critic (SAC) algorithm, specifically focusing on its automatically adjusted temperature (alpha) variant, within the context of autonomous navigation in unexplored environments. We developed an exploration system that integrates the SAC-alpha algorithm to autonomously navigate and map unfamiliar terrains, employing a strategic waypoint selection mechanism to optimise movement toward target objectives. The SAC-alpha framework was customised and enhanced to address the challenges posed by the lack of pre-existing environmental maps and the presence of complex obstacles. The efficacy of SAC-alpha was rigorously evaluated in simulated environments, utilising metrics such as success rate, navigation efficiency, and obstacle avoidance performance. Results from these evaluations demonstrate that SAC-alpha outperforms the twin delayed deep deterministic policy gradient (TD3) algorithm, exhibiting superior capability in overcoming local optima, efficiently navigating complex environments, and ensuring safety throughout the navigation process.