From Language to Aerial Performance: Interactive UAV Swarm Trajectory Generation via Domain-Enhanced Vectorized Representations

Designing complex UAV swarm performances like drone light shows and aerial displays demands both geometric precision and expressive flexibility, yet existing approaches rely on manually crafted trajectories, requiring substantial manual effort and limiting creative flexibility. While recent large language models (LLMs) offer an opportunity to bridge natural language intent and trajectory generation, general-purpose LLMs lack the geometric reasoning needed to produce spatially consistent paths. We propose an interactive end-to-end framework that translates natural language and speech instructions into executable multi-agent flight trajectories, in which language-specified 2D patterns are mapped to 3D flight coordinates at assigned altitudes. Our approach introduces SVG path commands as a vectorized representation that is both LLM-friendly and geometrically precise, serving as a semantic–geometric bridge. To equip LLMs with geometric awareness, we construct a difficulty-tiered dataset and perform supervised fine-tuning (SFT) with LoRA, complemented by greedy nearest-neighbor target assignment and BallTree-accelerated artificial potential field (APF) deconfliction for scalable execution. Experiments across ten frontier LLMs reveal a pronounced capability cliff: the best Tier 1 accuracy ranges from 88% to 92% across the two judges, whereas Tier 2 accuracies range from 0% to 26% across models and judges. After domain enhancement, our SFT-enhanced 4B-parameter model with few-shot prompting achieves 80% on basic shapes and 22% on text outlines, giving 51% overall accuracy versus 43% for the 9B model under the tested configurations. Different LoRA ranks and training token cutoffs confound this scale comparison. The Llama comparison also shows that fine-tuning gains do not generalize across the tested model families and settings. The 2 m minimum-separation threshold is satisfied in 95–100% of randomized-seed runs for swarms up to N=50 under the tested grid-launch protocol and in all tested phased-deployment runs for the N=100 dense text formation; these are empirical results rather than formal safety guarantees. The structural validity of the recommended fine-tuned configuration reaches 100% on both tiers (versus 0–34% for the two Qwen pretrained baseline configurations), and AirSim physics-based simulation validation reports a mean formation error below 1 m for five of six tested patterns, averaged from the minimum-error frame to the end of each run, with one transient separation violation observed in a single pattern. For small and moderate swarms, the offline design cycle takes approximately 30–74 s and is dominated by one LLM inference call; the densest N=200 case additionally requires up to 48.7 s for APF convergence.