GraphRAG Retrieval of Regulatory Knowledge: An Empirical Evaluation for Ecuadorian Tax Legislation Query Answering
Daniel Vinicio Cuatín Flores, Jorge Andrés Mosquera Sotomayor, Carlos Iván Díaz Hidalgo, Pablo Marcel Recalde Varela, Paúl Francisco Baldeón-Egas, Monserrate Intriago-Pazmiño, Yadira QuiñonezTax regulation in Ecuador changes constantly, and the resulting information is scattered across laws, regulations, resolutions, forms, and annexes with strong ties across meaning and time. We compare four retrieval-augmented generation (RAG) strategies for question answering over Ecuadorian tax legislation (vector-only, GraphRAG, hybrid, and agentic retrieval) in a system that supports text, speech, and image inputs and combines a tax knowledge graph, dense vector search, and three agents that refine, validate, and plan each query’s strategy. Following a Design Science Research approach informed by a SALSA literature review, the system was tested on a curated 51-document corpus using 100 text-only tax queries across the four retrieval modes and two cloud generator models (gemma4:31b-cloud and gpt-oss:20b-cloud), for 800 RAGAS-judged executions, plus a Likert-scale review by five domain specialists. Speech- and image-input performance was not evaluated in this benchmark. Graph-only retrieval was weakest on all five RAGAS metrics (Context Precision = 0.02), while hybrid gave the best balance (Faithfulness = 0.67, Answer Relevancy = 0.64, Context Precision = 0.52, Context Recall = 0.76) at the lowest latency (2.66 s). Faithfulness depended more on the generator model than on retrieval mode (0.78 vs. 0.47). No strategy won on every metric, so architecture choice in dynamic regulatory domains should follow operational priority rather than a single universally optimal setup.