Optimization of the fast layer transmittance algorithm in RTTOV v13.1 for strong water vapor absorption channels of the FY-3F HIRAS-II instrument using LBLRTM v12.11

Fast and accurate calculation of atmospheric transmittance is essential for infrared atmospheric remote sensing and satellite data assimilation. However, fast radiative transfer models show significant errors in strong water vapor absorption channels (e.g., near 6.7 µm). An important reason is the numerical instability encountered during the regression of transmittance coefficients when dealing with lower and middle atmosphere. To address this, this study proposes an optimized scheme to calculate atmospheric transmittance vertical profiles for the RTTOV (Radiative Transfer for TOVS) fast transmittance algorithm. The method introduces a physically motivated transmittance threshold to sub-select training samples and employs a cumulative transmittance-based weighting factor within a weighted least squares regression to recalibrate the transmittance coefficients. It aims to optimize the calculation scheme for transmittance coefficients of the Hyperspectral Infrared Atmospheric Sounder-II (HIRAS-II) instrument onboard the Fengyun-3F satellite (FY-3F). The method is assessed by calculations on the training profile datasets provided within the RTTOV model framework. By comparing transmittance and brightness temperature calculations at 6.7 µm from this method with those from a line-by-line model and observations from HIRAS-II, the results show that the accuracy of the forward model for the 6.7 µm absorption channel is significantly enhanced by applying a threshold-based noise reduction method. This improvement enhances the stability and reliability of the transmittance calculations for this strong absorption band. Further accuracy enhancements are obtained by incorporating weighting corrections into the calculations of transmittance coefficients. The root mean square error (RMSE) and bias of the observation minus background (OMB) time series for FY-3F HIRAS-II demonstrate that the transmittance coefficient calculation scheme with weighting factor correction improves the forward model accuracy, which is more consistent with RTTOV simulation results. The mean OMB bias at the 6.7 µm absorption-peak channel is closer to zero than that of RTTOV, whereas the OMB biases at the channels on either side of the absorption peak remain consistent with those of RTTOV.