Reliable Identification and Out‐of‐Library Detection in Mass Spectra

ABSTRACT

Accurate molecular identification from mass spectra is essential in analytical workflows, yet conventional library search typically returns the closest match even when the true compound is absent, leading to overconfident false positives. We propose a probabilistic framework that supports both reliable identification of compounds represented in a reference library and model‐based flagging of spectra that may not be adequately represented in the reference library. Spectra are modeled within a Bayesian nonparametric method that does not predefine the number of clusters; instead, the model can allocate new clusters when incoming spectra are insufficiently explained by existing references. This property provides a statistical mechanism for flagging potentially unseen compounds while maintaining coherent grouping of known ones. Experiments on large‐scale electron ionization libraries demonstrate stable performance under diverse noise conditions, consistent grouping of replicate spectra, and the tendency of spectra absent from the reference database to form new clusters. The framework supports reliable identification while reducing forced matches for spectra that are poorly represented in the reference data.