Enhanced Text-Driven Directional Editing for Marine Dynamic Data Generation

The generation of high-quality maritime samples is gradually becoming a key and challenging issue, due to the data thirst for training maritime intelligent models. However, existing methods mainly focus on static sample generation, which cannot meet the requirements of algorithms for dynamic decision. In this paper, an innovative method for generating high-quality marine dynamic data is proposed based on diffusion models. Considering the sensitivity of the diffusion model to prompts, a text enhancement module is first designed to perform semantic enhancement on the input text from the perspective of an expert in maritime climatology. Meanwhile, a directional image editing module is proposed to extract masks of interest from the input image, resulting in separate sea surface and sky regions. Then the image, mask and the enhanced text are sent together into the diffusion model to generate a high-quality directionally edited image. Finally, a video generation diffusion model is designed to convert the edited image into a dynamic data sequence. The entire framework has a clear sense of hierarchy and stable generation effect. We performed quantitative and qualitative experiments to prove that our method has significant advantages in data quality and controllability against existing SOTA methods.