Enhanced Mask R-CNN with PANet and PointRend for Detection and Instance Segmentation of Shadow Puppet Characters

The segmentation of Hehuang shadow puppet images holds significant value for artistic creation and cultural preservation. However, it still faces challenges such as the subjective nature of manual segmentation and inaccurate boundary delineation in complex images. To address these challenges, an improved Mask R-CNN model is proposed for precise segmentation of shadow puppet image boundaries. Specifically, inspired by PANet, only its bottom-up path augmentation is adopted after the FPN outputs to shorten the feature fusion path between low-level contour features and high-level semantics. Additionally, Concat replaces residual connections to preserve feature diversity. Finally, the PointRend module is integrated into the mask branch to prevent overly smooth transitions along segmentation mask edges and enhance pixel-level refinement of image boundary segmentation. In this study, a novel COCO-format dataset with 2700 images and seven instance categories was constructed for comprehensive segmentation evaluation. Compared with the baseline, the proposed model achieves relative improvements of 10.1% in bbox_AP for detection and 10.6% in segm_AP for instance segmentation, while segm_AP50 rises from 92.1% to 95.0%. Furthermore, we developed an interactive segmentation system that refines complex boundaries, enhancing the extraction capability of shadow puppet elements and facilitating its application in digital preservation scenarios of shadow puppet heritage.