Performance Analysis of Q-Learning; DQN-Enhanced PID Controllers for Aircraft Trajectory Tracking Under Wind Disturbances

Traditional fixed-gain proportional–integral–derivative (PID) controllers often exhibit limited adaptability when regulating the longitudinal motion of passenger aircraft under varying flight conditions and external disturbances. This study proposes an adaptive control framework that integrates reinforcement learning (RL) techniques—specifically tabular Q-learning and Deep Q-Networks (DQN)—to optimize the PID gains for improved trajectory tracking. The longitudinal dynamics of the aircraft are modeled using a set of nonlinear state equations, while the RL agents adjust the proportional, integral, and derivative gains based on tracking error and control effort. The Q-learning agent adapts these gains continuously at each simulation time step, whereas the DQN agent learns to select an optimized gain configuration at the trajectory level through an ε-greedy exploration strategy and carefully designed reward functions. Performance is evaluated via numerical simulations for both trapezoidal and sinusoidal reference trajectories, with and without gust disturbances. Results demonstrate that the RL-enhanced PID controllers achieve substantially lower tracking errors, measured by mean absolute error (MAE), root mean square error (RMSE), and maximum error, compared to conventionally tuned PID controllers. In particular, the DQN-based approach yields the most consistent robustness and accuracy across different reference profiles and disturbance levels. These findings highlight the potential of RL-augmented adaptive PID control for enhancing stability and precision in intelligent flight control systems operating in dynamic aerospace environments.