Adaptive Decentralized Policies With Attention for Large-Scale Multiagent Environments

Abstract

Multiagent reinforcement learning (MARL) poses unique challenges in real-world applications, demanding the adaptation of reinforcement learning principles to scenarios where agents interact in dynamically changing environments. This article presents a novel approach, “decentralized policy with attention” (ADPA), designed to address these challenges in large-scale multiagent environments. ADPA leverages an attention mechanism to dynamically select relevant information for estimating critics while training decentralized policies. This enables effective and scalable learning, supporting both cooperative and competitive settings, and scenarios with nonglobal states. In this work, we conduct a comprehensive evaluation of ADPA across a range of multiagent environments, including cooperative treasure collection and rover-tower communication. We compare ADPA with existing centralized training methods and ablated variants to showcase its advantages in terms of scalability, adaptability to various environments, and robustness. Our results demonstrate that ADPA offers a promising solution for addressing the complexities of large-scale MARL, providing the flexibility to handle diverse multiagent scenarios. By combining decentralized policies with attention mechanisms, we contribute to the advancement of MARL techniques, offering a powerful tool for real-world applications in dynamic and interactive multiagent systems.