Mercer-compliant kernels in machine learning: comprehensive theoretical analysis and empirical validation

Abstract

This paper presents a rigorous investigation of Mercer-compliant kernels through both theoretical analysis and extensive empirical validation. We present a unified theoretical analysis of Mercer’s condition across six fundamental kernel classes, including radial basis function (RBF), Matérn ( ν = 0.5, 1.5, 2.5), Nyström-based approximations (Nyström-RBF and Nyström-Matérn), and Gaussian Process formulations. This framework integrates established proofs and spectral properties from the literature into a coherent analytical perspective. Our analysis further provides characterization of eigenvalue behavior and examination of positive definiteness properties for each kernel type. Through comprehensive SVM classification experiments on seven diverse datasets (Breast Cancer Wisconsin, Wine, Digits, Iris, Spiral, Moons, and Circles), we demonstrate how theoretical kernel properties translate into practical performance. Key findings indicate that Matérn ( ν = 0.5) kernels achieve strong and often near-perfect accuracy on datasets with complex nonlinear geometric structures (e.g., Moons and Spiral), while RBF kernels excel (99.8 %) on relatively well-separated data. The Nyström approximation provides significant computational speed-up and maintains competitive accuracy on synthetic datasets, though its performance degrades on more complex real-world data. Statistical validation using the Friedman test and post-hoc Nemenyi tests indicates statistically significant differences among kernel types ( p < 0.001). This work bridges the gap between kernel theory and practical machine learning by providing: (1) a unified analysis of Mercer condition properties across kernel types, (2) empirical performance benchmarks, and (3) practical insights for kernel selection based on data characteristics and computational constraints.