A Dynamic-Selection-Based, Retrieval-Augmented Generation Framework: Enhancing Multi-Document Question-Answering for Commercial Applications

Commercial multi-document question-answering (QA) applications require a high multi-document retrieval performance, while simultaneously minimizing Application Programming Interface (API) usage costs of large language models (LLMs) and system complexity. To address this need, we designed the Dynamic-Selection-based, Retrieval-Augmented Generation (DS-RAG) framework, which consists of two key modules: an Entity-Preserving Question Decomposition (EPQD) module that effectively decomposes questions while preserving the entities of the original user’s question to reduce unnecessary retrieval and enhance performance, and a Dynamic Input Context Selection (DICS) module that optimizes the LLM input context based on the content of the user’s question, thereby minimizing API usage. We evaluated the proposed framework on a newly constructed dataset containing questions that require up to four multi-document retrievals. Experimental results demonstrated the new framework’s superior performance in terms of retrieval quality, input context optimization, and final answer generation compared to existing approaches. Consequently, the DS-RAG framework can be leveraged to develop domain-specific commercial QA applications in the future.