Multi-objective inventory optimization using reinforcement learning: a comparative study on profitability and carbon emissions

Abstract

Inventory management is a core part of supply chains, and over the years it has been increasingly challenged by the need to balance economic performance with environmental considerations. While prior reinforcement learning (RL) studies have incorporated carbon emissions indirectly through cost penalties or regulatory constraints, this work addresses an existing gap by treating emissions as an independent optimization objective. This study examines RL as an adaptive decision‑making approach for inventory optimization with two objectives: maximizing profit and minimizing carbon emissions. The problem is formulated as a Markov Decision Process, and four RL algorithms Proximal Policy Optimization (PPO), Phasic Policy Gradient (PPG), Advantage Actor‑Critic (A2C), and Double Deep Q‑Network (DDQN) are evaluated under identical experimental conditions. Carbon emissions are explicitly modeled in the reward function rather than embedded within operating costs. The results show that PPG achieves the highest profitability with only a modest increase in emissions, while DDQN converges faster but yields lower profit overall. Sensitivity analysis indicates that reward weighting strongly influences policy behavior, with PPO providing the most stable trade‑off between profitability and emissions.