Mixed-Precision SVD on GPUs via Ogita–Aishima Iterative Refinement

The Ogita–Aishima iterative refinement algorithm for SVD is based on matrix–matrix multiplication, making it an attractive approach for GPUs. We extend the algorithm to complex arithmetic and add cluster handling. The latter is based on Wedin's perturbation theory and a per-pair version of Ogita–Aishima's convergence analysis. Earlier implementations require well-separated singular values; the proposed cluster handling removes this restriction. Our implementation refines an FP32-quality SVD computed with cuSolver's

to FP64-quality, while matching the accuracy of the corresponding FP64 routine on all test problems—twelve synthetic singular-value distributions and 38 matrices from the SuiteSparse collection. The performance depends on the spectrum and the FP32-to-FP64 ratio. Under Ozaki-II FP64 emulation on an NVIDIA RTX PRO 6000 Blackwell workstation GPU, which has a \(64\) : \(1\) FP32-to-FP64 ratio, the refinement is up to \(11.1\times\) faster than on spectra whose adjacent gaps are resolved by the FP32 SVD, and at most \(1.9\times\) slower on spectra for which they are not. The advantage is at most \(1.5\times\) on data-center GPUs with a \(2\) : \(1\) FP32-to-FP64 ratio, such as the NVIDIA B200. These results show the algorithm's potential on hardware with strong mixed-precision capabilities.