FedSwin-LHTP: Structure-Aware Hessian-Inspired Token Pruning for Efficient Federated Skin Lesion Classification

Background: Skin cancer encompasses a diverse range of malignancies and remains a significant global health challenge. Accurate machine-learning-assisted diagnosis can substantially improve patient outcomes through early detection and timely clinical intervention. Federated Learning (FL) enables privacy-preserving collaborative model training across multiple healthcare institutions while ensuring that sensitive patient data remain decentralized. However, deploying advanced architectures such as Vision Transformers (ViTs) in clinical environments is challenging due to the high computational demands of self-attention mechanisms. Methods: This work proposes FedSwin-LHTP, an efficient federated learning framework for skin lesion classification that integrates a Swin Transformer backbone with a Lightweight Hessian-Inspired Token Pruning (LHTP) mechanism. LHTP estimates token importance using a second-order Taylor approximation around converged local model parameters to identify less informative patch tokens, enabling the early pruning of redundant representations without explicitly computing the Hessian matrix. Furthermore, the framework incorporates the FedProx optimization objective to mitigate client drift under heterogeneous non-IID data distributions. The proposed framework is evaluated on the HAM10000 and ISIC datasets under realistic non-IID federated settings. Results: Experimental results demonstrate stable convergence, effective knowledge aggregation, and robust diagnostic discrimination across distributed clients. By adaptively pruning approximately 60% of Stage-1 tokens, the proposed framework substantially reduces the computational burden of local transformer processing while maintaining high multiclass classification performance, achieving an accuracy of up to 96.1% on the evaluated datasets. Conclusions: These results highlight the potential of FedSwin-LHTP as a practical, privacy-preserving, and resource-efficient solution for collaborative healthcare intelligence.