Certifiable Deep Importance Sampling for Rare-Event Simulation of Black Box Systems

AI-Enhanced Monte Carlo Methodology Achieves More Reliable Testing and Efficiency Certificate for Safety-Critical Systems

As artificial intelligence (AI) increasingly drives human-interacting intelligent physical systems such as self-driving vehicles, ensuring their safety against rare catastrophic events has also become a critical challenge. Traditional simulation-based testing methods are powerful in quantifying risks before real deployment. However, for black box systems that generically arise in AI-embedded applications, these methods can consistently underestimate the probabilities of rare failures and, dangerously, without diagnostically detectable signals. To address this perilous gap, the paper "Certifiable Deep Importance Sampling for Rare-Event Simulation of Black Box Systems," by Arief, Bai, Ding, He, Huang, Lam, and Zhao, introduces a framework based on an integration of neural networks into importance sampling, a variance reduction technique that has been found useful in rare-event simulation. This framework, which is called deep probabilistic accelerated evaluation (Deep-PrAE), uses deep neural network classifiers to learn rare-event geometries, which then calibrates importance samplers to estimate rare events with certifiable statistical guarantees. Moreover, these guarantees are designed to go beyond the conventional certifiable notions in rare-event simulation to effectively balance the applicability to black box settings with the required sampling efficiency gain that pertains to rare-event computation. These findings demonstrate a vital step forward in the reliable and certifiable deployment of complex autonomous technologies.