Synthetic‐to‐realistic domain adaptation for cold‐start of rail inspection systems

Abstract

Rail surface defects are potential danger factors for railway systems, and visual inspection of surface defects plays a vital role in rail maintenance. Recently, the methods based on deep learning have been widely used in rail inspection systems, but such systems often face the problem of a lack of defect samples for training deep learning models at start‐up, which is called the cold‐start problem. It is challenging to obtain sufficient defect samples since defects are sparse and even non‐existent for a running railway system. Therefore, a synthetic‐to‐realistic domain adaptation (SRDA) method is proposed for real‐world rail inspection. SRDA adapts synthetic images to look more realistic for reducing the domain gap between synthetic images and realistic rail surface images and obtains translated images that consist of synthetic defect information and realistic rail background. After that, the translated images are used to train a detector for rail inspection of rail surface defects. In order to make the detector more robust to complex backgrounds, SRDA generates images with the same defect‐level semantics but with different texture appearances and makes the detector align these images in the learned feature space. In addition, the synthetic and realistic rail surface defects (SRRSD) dataset containing 20,662 images is built. The experimental results on SRRSD show that SRDA achieves higher detection performance than other established domain adaption methods with 19.0% for and 26.7% for average precision.