A Configurable Framework for Quantifying and Comparing Interpretability Across ML Models and Methods

Machine learning (ML) models have achieved remarkable success in areas ranging from healthcare to autonomous systems. Yet, their inherent complexity frequently obscures the reasoning behind their decisions, undermining transparency, accountability, and user trust. Compounding this issue is the absence of a universal methodology for comparing and ranking interpretability across diverse ML models and interpretability techniques. This paper addresses this gap by introducing a configurable framework of quantitative metrics to evaluate and rank interpretability. Our approach offers a structured, heuristic basis for assessing model clarity, decision logic, and accessibility, enabling practitioners to systematically compare interpretability across a wide range of algorithms and techniques. The resulting scores are intended as a practical, domain-configurable heuristic guide for comparison rather than a universal notion of interpretability.