Sequential SAR-to-Optical Image Translation

There is a common need for optical sequence images with high spatiotemporal resolution. As a solution, Synthetic Aperture Radar (SAR)-to-optical translation tends to bring high temporal continuity of optical images and low interpretation difficulty of SAR images. Existing studies have been focused on converting a single SAR image into a single optical image, failing to utilize the advantages of repeated observations from SAR satellites. To make full use of periodic SAR images, it is proposed to investigate the sequential SAR-to-optical translation, which represents the first effort in this topic. To achieve this, a model based on a diffusion framework has been constructed, with twelve Transformer blocks utilized to effectively capture spatial and temporal features alternatively. A variational autoencoder is employed to encode and decode images, enabling the diffusion model to learn the distribution of features within optical image sequences. A conditional branch is specifically designed for SAR sequences to facilitate feature extraction. Additionally, the capture time is encoded and embedded into the Transformers. Two sequence datasets for the sequence translation task were created, comprising Sentinel-1 Ground Range Detected data and Sentinel-2 red/green/blue data. Our method was tested on new datasets and compared with three state-of-the-art single translation methods. Quantitative and qualitative comparisons validate the effectiveness of the proposed method in maintaining radiometric and spectral consistency.