Heteroscedastic nearest shrunken centroid classification: methods and applications to high-dimensional data

Abstract

Despite of recent advances in machine learning methods with huge training data, classification of high-dimensional, low-sample-size (HDLSS) data remains challenging because limited sample sizes increase the risk of overfitting and hinder reliable parameter estimation. The nearest shrunken centroid classifier, also known as Prediction Analysis of Microarrays (PAM), combines centroid-based discrimination with feature selection through thresholding. However, its homoscedasticity assumption may limit classification performance when feature variances differ across classes, as commonly observed in gene expression data. We propose a heteroscedastic extension of the nearest shrunken centroid framework that incorporates class-specific variances into both the test statistic and discriminant function. The proposed method retains the simplicity and interpretability of the original classifier while accommodating variance heterogeneity. By integrating the framework with soft, hard, and order thresholding strategies, we develop three new classifiers. We evaluate their performance through comprehensive simulation studies and applications to multiple multiclass cancer gene expression datasets. The results show that accounting for heteroscedasticity substantially improves classification accuracy, particularly with hard and order thresholding, while maintaining competitive feature sparsity. These findings highlight the importance of modeling variance heterogeneity in high-dimensional classification and provide practical tools for improved cancer subtype prediction.