DOI: 10.18485/judig.2026.2.ch22 ISSN:

Exploring In-Context Learning for LLM Machine Translation for Macedonian

Jana Kuzmanova, Sonja Gievska, Katerina Zdravkova
Show PDF Cite
Despite the increasing popularity of large language models for machine translation and their good performance across many language pairs, there is still considerable room for improvement, particularly for low-resource languages and translation into these languages. In-context learning approaches have shown promise in enhancing translation quality in such settings. Providing translations of words from the source text, similar parallel sentences, and grammatical information has resulted in improvements across multiple language pairs. In this paper, we explore whether and how different in-context learning approaches, including parallel sentences and dictionary entries, can improve translation from English into Macedonian. We examine different numbers of examples and different degrees of similarity to the source sentences, phrases, and words to determine which configurations are most effective. Our results show that the effectiveness of in-context learning depends strongly on the model, prompt configuration, and type of additional information provided. Greater example similarity does not consistently improve translation quality, while dictionary-based prompting generally yields poorer results, particularly when out-of-domain entries are used. Although improvements are observed for individual models and translations, all evaluated LLMs perform below the encoder-decoder NLLB baseline across all experimental settings.

More from our Archive