Reinforcement Learning for Warehouse Management Using a Scenario-Based Simulation Testbed

Warehouse operations involve dynamic item flows, fluctuating demand, and heterogeneous layouts, making adaptive decision-making essential for efficient storage and order fulfillment. In this context, reinforcement learning (RL) provides a promising approach for learning adaptive warehouse control policies under stochastic environments. However, evaluating RL-based solutions in real warehouse settings is often costly and time-consuming, motivating the need for realistic and reproducible simulation environments. In this paper, we introduce a configurable warehouse simulation environment modeling stochastic item arrivals, order generation, and internal logistics operations across diverse layouts and workload conditions. Based on this environment, we construct a reproducible experimental testbed composed of multiple scenarios ranging from low-load to highly congested settings. The testbed is publicly released to support reproducible research and comparative evaluation within the research community. We formulate the warehouse management problem as a Markov decision process (MDP) and apply a Maskable Proximal Policy Optimization (Maskable PPO) agent to learn adaptive control policies. The RL-based approach is evaluated across the defined scenarios and compared against heuristic baseline strategies. Experimental results show that the proposed solution achieves performance comparable to a strong greedy first-in, first-out (FIFO) heuristic while improving order fulfillment by up to 13.5 percentage points under challenging workload conditions. These results demonstrate the ability of RL to learn robust warehouse control policies that adaptively optimize performance and maintain operational stability across a wide spectrum of distinct scenarios.