Hyperspectral Video Object Segmentation Based on Spectral–Spatial Interaction Enhancement and Adaptive Correlation Memory Selection

Hyperspectral imaging (HSI) is an important imaging modality in the field of optical remote sensing, capable of simultaneously capturing spatial structures and continuous spectral responses related to material properties. Therefore, compared with conventional red-green-blue (RGB) images, it can provide richer representations for understanding dynamic scenes. However, most existing video object segmentation methods have been developed for RGB videos, while high-quality hyperspectral video datasets with frame-by-frame pixel-level annotations remain scarce. To address this gap, this study constructed a hyperspectral video object segmentation dataset, comprising hyperspectral videos and their corresponding three-channel false-color RGB videos derived from the same HSI data, with pixel-level annotations provided for each frame. The dataset contains three spectral subsets: visible (VIS), near-infrared (NIR), and red-edge near-infrared (RedNIR). Using this dataset, we developed SSAM-SAM2, a hyperspectral video object segmentation network built on Segment Anything Model 2 (SAM 2). The network includes a spectral–spatial interaction enhancement (SSIE) module designed to identify informative spectral bands, suppress interference from redundant and noisy spectral channels, and integrate hyperspectral cues with the spatial-structure features of the corresponding false-color RGB images. In addition, the network introduces an Adaptive Correlation Memory Selection (ACMS) module that regulates memory sampling and selection according to variations in target state and inter-frame feature correlation. This design reduces the influence of redundant or unreliable historical information and improves temporal memory reliability under target deformation, motion variations, and background interference. Extensive experiments show that SSAM-SAM2 outperforms the selected comparison methods on the VIS, NIR, and RedNIR subsets under the evaluated settings. These results demonstrate the value of the proposed dataset and indicate that spectral–spatial enhancement and temporal adaptive memory mechanisms contribute to improving hyperspectral video object segmentation performance.