Intelligent Caching in IoT Sensing Networks: A Decentralized Multi-Agent Reinforcement Learning Approach with Entropy-Driven Exploration

Abstract

This paper presents a decentralized soft actor-critic (DSAC) framework to optimize cache decisions in IoT edge networks, addressing critical trade-offs between data freshness (Age of Information), energy efficiency and fronthaul traffic reduction. Our DSAC framework driven by entropy demonstrates superior performance compared to the Deep Q-network (DQN) in three key indicators: (1) the achievement of a cumulative reward of 14.5% (−21.28 vs. −24.415 for DQN), (2) the reduction of energy consumption by adjusting battery updates, and (3) the maintenance of a more stable fronthaul traffic load (0.15–0.4 vs. DQN’s erratic 0.15–0.25 range). The integration of entropy regularization (α=0.2) proved particularly powerful, accelerating learning convergence, preventing policy stagnation, and yielding 15.6% rewards increase relative to non-entropy versions. The results of the dynamic simulation confirm the robustness of the DSAC in balancing the freshness of real-time data with resource constraints and are particularly suitable for large-scale IoT deployments requiring decentralized coordination.