SAC-XLM: Structure Discovery Cross-Lingual Transfer Through Advantage Actor–Critic Algorithm for Multilingual Model

Multilingual language models have attracted increasing attention as language modeling has advanced. However, the uneven availability of high-quality labeled data across languages can lead to disparities in downstream performance. Struct-XLM addresses this challenge by using reinforcement learning to discover shared syntactic structures, reducing its reliance on labeled corpora. However, its use of REINFORCE can produce high-variance policy gradient estimates, resulting in unstable training. This paper proposes SAC-XLM, an optimization framework for multilingual structure discovery based on advantage actor–critic (A2C) learning. SAC-XLM combines a pretrained structural encoder, an actor network, a critic network, and a reward acquirer to learn structural representations across languages. The critic network provides a state-dependent value baseline for advantage estimation, while the reward acquirer evaluates the contribution of structural actions to cross-lingual alignment. Together, these components support more stable policy optimization. Experiments on seven XTREME tasks show that SAC-XLM improves training stability and efficiency compared with Struct-XLM. It reduces pretraining time to approximately 40% of that required by Struct-XLM while preserving overall downstream performance.