Symphony-Bind: Prediction of Protein Binding Sites for 11 Representative Small Molecules and Ions via Fine-Tuning Protein Language Models and Grouped Multi-Task Learning

Abstract

Accurately identifying protein binding sites for small molecules and ions is crucial for understanding biological processes and advancing drug discovery. Pretrained protein language models (pLMs) have emerged as powerful tools for this purpose, but existing prediction models often face a trade-off when using pLMs: freezing pLMs limits their adaptability, while fully fine-tuning them requires high computational costs. To address this trade-off, we attempted to introduce Parameter-Efficient Fine-Tuning (PEFT) as a promising solution to balance efficiency and performance. Specifically, we applied five PEFT strategies (i.e., LoRA, QLoRA, DoRA, AdaLoRA, and IA3) to fine-tune four pLMs (i.e., ProtT5, ProtBERT, ESM2–150M, and ESM2–650M) and assessed their predictive performance across protein binding site tasks for 11 representative small molecules and ions. The results clearly indicate that the LoRA-enhanced ESM2–650M consistently outperforms all other combinations. Despite this robust baseline, training independent models for specific small molecules remains challenging due to the scarcity of high-quality binding data. To bridge this gap, we implemented a Grouped Multi-Task Learning (GMTL) strategy, allowing the model to capture shared binding patterns among ligands with similar biological significance. Experimental results demonstrate that this strategy significantly enhances predictive performance. Building upon these insights, we present Symphony-Bind. It is a GMTL framework that leverages LoRA-enhanced ESM2–650M to extract embeddings, which are subsequently refined by a shared ConvBERT module and then processed by ligand-specific MLPs for precise binding site prediction. Performance evaluation on 11 representative ligand tasks shows that Symphony-Bind achieves average MCC values of 0.561, 0.629, and 0.324 for the nucleotide, cofactor, and inorganic ion groups, surpassing evaluated sequence-based state-of-the-art methods while remaining competitive with structure-based models.