DOI: 10.1145/3860324 ISSN: 1049-331X

What to Retrieve? Investigating Effective Call Information in Repository-level Code Generation

Siyu Nan, Jian Wang, Yaling Luo, Zhihao Wang, Neng Zhang, Yutao Ma, Bing Li

Retrieval-Augmented Generation (RAG) emerges as a widely adopted solution in repository-level code generation by retrieving functions for direct reuse or learning similar usage patterns. Nevertheless, it remains unexplored which types of contexts effectively assist LLMs. In this paper, we categorize the context of function calls and empirically analyze the performance of LLMs. Specifically, we define call information along two dimensions: call depth , including direct-level , inward expansion (to callees), and outward expansion (to callers), as well as call granularity , including function-level , module-level , and file-level . We investigate the utility of the context in two distinct scenarios: precision-oriented call extraction and relevance-oriented call retrieval , targeting precise call information and relevant call information , respectively. We conduct comprehensive experiments with three mainstream LLMs on two widely used benchmarks, CoderEval and DevEval. When precise call information is provided, the additional inclusion of inward expansion, outward expansion, module-level, and file-level context enhances the average Pass@1 of the studied LLMs by 2.11%, 79.59%, 37.54%, and 82.54%, respectively. When the relevant call information is provided, incorporating the same set of strategies leads to improvements of 18.00%, 4.77%, 26.07%, and 44.60% in average Pass@1, respectively. This empirical study offers practical insights for optimizing RAG strategies in repository-level code generation.

More from our Archive