Semi-Supervised Pre-trained Foundation Model for 3D Geological Feature Analysis of Seismic Images

Abstract

Geological feature analysis from seismic images plays a vital role in geological assessments, resource exploration, natural disaster prediction and assessment, and carbon capture and storage. Traditional deep learning-based geological feature recognition methods often suffer from a high demand for labeled training samples and limited model generalization. In recent years, foundational models pre-trained on large-scale unlabeled data through self-supervised learning have gained significant attention in the computer vision community owing to their strong generalization and robustness, and initial attempts have been made to extend such models to seismic image analysis. Nevertheless, most existing efforts rely on 2D slices and focus primarily on general seismic tasks, with little emphasis on the specialized optimization needed for identifying diverse geological features. This study proposes a dual pre-training semisupervised framework to develop SS-UNETR, a foundational model that leverages a 3D Swin Transformer backbone. SS-UNETR is designed to concurrently segment multiple geological features in seismic volumes through multi-task learning. The training procedure consists of two sequential stages. In the first stage, the encoder of SS-UNETR is pre-trained on 75,651 field seismic images via three proxy tasks (image inpainting, rotation prediction, and contrastive learning) to learn underlying representations and basic patterns without explicit labeling. In the second stage, SS-UNETR is fine-tuned using 2,860 synthetic seismic images with corresponding geological annotations to adapt the model to geological feature segmentation. Experimental results demonstrate that SS-UNETR excels in analyzing geological features of seismic images compared to methods based on seismic attributes and neural networks trained for specific tasks. SS-UNETR achieves effective simultaneous identification of multiple geological features, even when these features exhibit highly similar seismic responses, for such features as faults, channels, and caves. Furthermore, SS-UNETR outperforms conventional deep learning models without pre-training and general-purpose seismic foundational models in few-shot geological feature identification tasks. These results indicate that SS-UNETR can serve as a valuable tool in the industry to enhance the efficiency and accuracy of geological interpretation, reduce the burden of manual labeling, and reduce the carbon footprint of model training.