HSMTL-Net: A Multi-Task Learning Framework for Human Segmentation with Integrated Regression and Classification Modules

Human segmentation is a fundamental task in computer vision, with applications in surveillance, sports analytics, autonomous systems, and human–computer interaction. This paper proposes HSMTL-Net, an attention-guided and boundary-aware multi-task learning framework for fine-grained human segmentation. The framework is built on a lightweight U-Net–inspired encoder–decoder backbone and jointly optimizes three complementary tasks: pixel-level human segmentation (Seg-Module), auxiliary gender classification (Gender-Module), and geometric regression of region descriptors (Seg-Tuner-Module). The two auxiliary heads provide semantic and geometric supervision signals that regularize the shared encoder, with the explicit aim of improving boundary fidelity and feature generalization under limited-data conditions. On the Supervise.ly Filtered Segmentation Person Dataset, HSMTL-Net achieves a Dice score of 97.83%, a mean IoU of 96.21%, a gender-classification accuracy of 97.96%, and a regression mean squared error (MSE) of 0.00995. A task-wise ablation shows that adding the regression and classification heads improves segmentation over the single-task baseline, with the largest gain attributable to the geometric regression head. The principal contribution of this work is the use of mask-derived geometric regression as auxiliary supervision for boundary-sensitive human segmentation within a single lightweight multi-task learning framework, together with an explicitly defined multi-task weighting strategy for joint optimization.