Camera-Consistent RGB Generation for Medical Hyperspectral Reconstruction

Abstract

Hyperspectral imaging (HSI) provides spatially and spectrally resolved information that can be used for tissue characterization and multiple medical applications. However, conventional HSI systems remain costly, complex, and slow. RGB-based hyperspectral reconstruction (HSR) offers a promising alternative, yet its practical use is limited by the ill-posed nature of the problem and poor generalization abilities. This work investigates the generalization ability of HSR models.We propose a novel approach using camera-consistent RGB image generation from hyperspectral reflectance data based on an illuminant- and sensor-dependent forward model and sigmoid-based normalization. MST++, a deep network for HSR, and partial least squares regression (PLSR) were trained on Hyper-Skin and validated on both internal test data and the HSI-Perfusion as external test data. On HSI-Perfusion, adapted RGB generation improved mean relative absolute error (MRAE) for MST++ from 0.3591 to 0.3250 (9.5 %) and reduced spectral angle mapper (SAM) from 0.1841 to 0.1782 (3.2 %). For PLSR, adapted RGB with sigmoid normalization reduced MRAE from 0.5855 to 0.3019 (48.4 %) and SAM from 0.3882 to 0.0781 (79.9 %). These results show that adapted RGB generation improves generalization to real data.