Joint Beamforming and Trajectory Optimization Algorithm for RSMA-UAV-Enabled Integrated Sensing and Communication System

Integrated sensing and communication (ISAC) enables concurrent wireless communication and target sensing using shared hardware and spectrum resources, and is regarded as a key technology for next-generation mobile networks. To address the challenge of coordinating sensing and communication in multiuser scenarios, this paper investigates an unmanned aerial vehicle (UAV)-enabled ISAC system employing rate-splitting multiple access (RSMA) and proposes a joint beamforming and trajectory optimization framework. Specifically, ground users are first clustered subject to a capacity–diameter constraint, and the UAV employs RSMA to serve users within each cluster. The sensing performance is characterized by the composite transmit beampattern gain in the target direction contributed by the common stream and all private streams. Under constraints on users’ downlink rates, transmit power, and sensing quality, we formulate an optimization problem to maximize the average downlink rate. By adopting a block coordinate descent (BCD) framework, the resulting non-convex problem is decomposed into three subproblems, namely cluster scheduling, rate allocation and beamforming, and UAV trajectory optimization. These subproblems are then solved via linear programming, semidefinite relaxation, and successive convex approximation, respectively. Numerical results demonstrate that, while satisfying the sensing-quality requirement, the proposed algorithm significantly improves the achievable system downlink rate.