Automatic Oral Cancer Detection Using Improved Honey Badger Algorithm-Based Feature Selection

Background/Objectives: Oral cancer is one of the most common types of cancer, with high mortality rates if not detected early. Traditional diagnostic methods based on clinical examination rely on experience, leading to delays in early and reliable diagnosis. In recent years, medical imaging and AI-based computer-aided diagnostic systems have shown promising results in the automated identification of oral cancer. In particular, the efficient management of high-dimensional feature spaces in machine learning and deep learning approaches directly impacts classification performance. In this context, metaheuristic-based feature selection technics is a critical component because of eliminating redundant and irrelevant features. To address these challenges, this study proposes a metaheuristic-based feature selection method to reduce feature dimensionality and enhance the classification performance of oral cancer detection. Methods: This study proposes an improved Honey Badger Algorithm-based feature selection approach for the automated detection of oral cancer. In the proposed method, the distance vector used in the HBA method has been redefined to improve the balance between exploration and exploitation. Additionally, a new Cauchy mutation-based migration strategy was integrated into the proposed method to increase diversity in the search space and avoid getting stuck in local minima. The continuous-valued iHBA method was discretized with a modified sin–cos transfer function for feature selection. Oral cancer images were filtered using the CLAHE method, and after extracting deep features with the ResNet50 architecture, the proposed metaheuristic-based method was used to select discriminative features. Results: The proposed method was first tested for reliability and limitations through repeated runs on problems with different characteristics, such as unimodal and multimodal classical test functions. Then, the method was applied to extract significant features for oral cancer detection using a Histopathological Imaging Database containing 1224 histopathological oral tissue images at 100× and 400× magnification levels from 230 patients. The proposed approach was assessed in terms of accuracy, precision, recall, F1-score, and convergence curves in comparison with various classical feature selection techniques, such as wrapper-based, filter-based, and embedded-based methods, as well as other metaheuristic-based methods. The experimental results demonstrated that the suggested strategy outperformed both traditional feature selection techniques and alternative metaheuristic approaches. Conclusions: The effectiveness of the proposed method in improving diagnostic accuracy was evaluated through comprehensive experimental analyses. The obtained findings show that the proposed iHBA-based feature selection approach can reduce feature dimensionality, eliminate redundant and irrelevant features, and improve the classification performance of oral cancer detection. Therefore, the proposed method provides an effective and competitive computer-aided diagnostic framework for the automated classification of histopathological oral cancer images.