A New Learning Technique for Planning in Cooperative Multi-Agent Systems

Abstract

This presentation introduces a new learning technique for planning in cooperative multi-agent systems (MAS), proposing a taxonomy for MAS based on rationality and optimality, and formally defining cooperative, competitive, and mixed matrix games (MGs). It presents the Cooperative Multi-agent Markov Decision Process (CMMDP) as a mathematical framework and introduces the Extended-Q algorithm, which integrates reinforcement learning with game-theoretic equilibrium concepts like Nash equilibrium to solve coordination problems. The algorithm is extended to handle weakly competitive scenarios and is enhanced with neural network-based generalization (Neuro-Extended-Q) for large state spaces. Experimental validation using grid games demonstrates its effectiveness, while future work includes convergence proofs, extensions to competitive MAS, partial observability, and improved exploration techniques.