Large-scale semantic indexing of Spanish biomedical literature using contrastive transfer learning
Ronghui You, Tianyang Huang, Ziye Wang, Yuxuan Liu, Hong Zhou, Shanfeng ZhuAbstract
Motivation
The exponential growth of biomedical literature has made automatic indexing essential for advancing biomedical research. While automatic indexing has made strides in English biomedical literature, there has been limited research on non-English biomedical texts due to insufficient high-quality training data.
Results
We propose BERTDeCS, a novel deep learning framework for automatically indexing Spanish biomedical literature using contrastive transfer learning. BERTDeCS utilizes a multilingual BERT (M-BERT) to generate multi-language representations and adapts M-BERT for Spanish biomedical literature domain through contrastive learning. Additionally, BERTDeCS enhances its semantic indexing capabilities on Spanish biomedical literature by leveraging enriched English annotated literature in MEDLINE through transfer learning. Experimental results on Spanish datasets demonstrate that BERTDeCS outperforms state-of-the-art indexing methods, achieving top performance in the MESINESP and MESINESP2 Tasks on medical semantic indexing in Spanish within the BioASQ challenge. Additional experiments on Portuguese provide preliminary evidence that BERTDeCS can transfer to a closely related language sharing substantial lexical and biomedical terminology with Spanish.
Availability
The source code is available at https://github.com/yourh/BERTDeCS and at https://doi.org/10.5281/zenodo.23212468.