Reinforcement Learning-Based Tension Control for Marine Winches Under JONSWAP Irregular Wave Disturbances

Active tension control in marine cable winch systems is challenged by vessel heave motion under irregular wave excitation, which generates dynamic disturbances that fixed-gain proportional-integral (PI) controllers cannot fully compensate. This study investigates whether a recurrent reinforcement-learning policy can address this limitation by learning an adaptive, temporally informed control strategy from simulation. A controller based on Proximal Policy Optimization with Long Short-Term Memory (PPO-LSTM) is trained in a Unity ML-Agents environment under stochastic Joint North Sea Wave Project (JONSWAP) wave disturbances and compared with a tuned PI baseline over an 8–20 kN operating range. This is the first application of PPO-LSTM to marine winch tension control and the first ablation study isolating the contributions of recurrent memory and wave-phase sensing in this context. The principal benefit of the recurrent architecture is not a reduction in mean tracking error which remains statistically comparable to the PI baseline but a substantial improvement in control consistency: tension-error variability is markedly reduced, particularly at higher setpoints. Both LSTM and wave-phase observations are shown to contribute independently to this improvement. Key limitations include curriculum bias at low setpoints, catastrophic forgetting during fine-tuning, and the simulation-to-reality gap, for which a staged hardware validation pathway is proposed.