Calibrated term presence modelling for CCUS patent abstracts: a case study with variational autoencoders and retrieval baselines

Abstract

Patent codes may lack granularity when carbon capture, utilization and storage (CCUS) inventions combine climate and chemical engineering concepts. We evaluate term presence models on 10 603 English patent abstracts through document completion, calibration and cooperative patent classification (CPC) prefix retrieval. We compare two variational autoencoders (VAEs), a product of the experts latent Dirichlet allocation model, a unigram prior, term frequency-inverse document frequency (TF-IDF), PatentSBERTa and PaECTER. With a target masking rate of 50 per cent for positive terms, TF-IDF nearest neighbours give the best ranked completion. The direct Bernoulli VAE ranks slightly above the negative binomial to Bernoulli VAE (NB VAE), while the latter has lower expected calibration error (ECE). Conditional posterior predictive draws retain positive rank associations for document and term counts, but the NB VAE and direct Bernoulli VAE predict 1.47 and 1.63 times the observed positive count, respectively. PaECTER and the TF-IDF representation after singular value decomposition (SVD) give the strongest CPC prefix retrieval, while VAE probabilities could support inspectable term screening.