Model‐Free Multi‐Agent Reinforcement Learning with PID Fusion for Nonlinear Process Control: Application to Hybrid Power Converters and Photobioreactors

Abstract

ABSTRACT This paper presents a hybrid control strategy combining model‐free Multi‐Agent Proximal Policy Optimization (MAPPO) with classical Proportional‐Integral‐Derivative (PID) controllers for the regulation of complex nonlinear systems. Unlike pure reinforcement learning approaches, the proposed framework leverages PID control as a stabilizing baseline while employing learned policy corrections to compensate for nonlinearities and uncertainties. The methodology is validated on two distinct benchmark systems: (1) a hybrid mode‐redundant (MR) switched power converter with continuous‐discrete dynamics, and (2) a multi‐variable photobioreactor model involving coupled biological, thermal, and chemical processes. In both applications, decentralized MAPPO agents coordinate to regulate multiple outputs simultaneously without requiring explicit system models. Simulation results demonstrate that the MAPPO‐PID fusion architecture consistently outperforms standalone PID and fuzzy logic controllers, particularly under time‐varying references and parametric uncertainties. Extended Monte Carlo validation with realistic sensor noise, actuator delays, and parameter variations confirms the robustness and practical applicability of the proposed model‐free control strategy. Quantitative metrics including root‐mean‐square error (RMSE), integral absolute error (IAE), and settling time reveal improved tracking accuracy, faster convergence, and enhanced disturbance rejection across both case studies.