DOI: 10.3390/fi18080442 ISSN: 1999-5903

An AI Answer-Validation Method Using Agentic RAG for Datasheet Inquiry for IoT Application System Deployment

Dezheng Kong, Nobuo Funabiki, Htoo Htoo Sandi Kyaw, I Nyoman Darma Kotama, Zihao Zhu

Internet of Things (IoT) application systems are increasingly adopted in factories, shops, offices, and governments. However, building such systems using various devices and modules remains difficult for non-experts, because they must confirm specifications, communication interfaces, voltage ranges, and operating conditions from technical datasheets before connecting devices. In previous studies, we have explored a generative AI-based answering tool for datasheet inquiry using Retrieval-Augmented Generation (RAG) for technical guidance of IoT application system deployment. However, the adopted top-kRAG pipeline often retrieves multiple related text chunks, which can cause the AI to confuse technically different specifications, such as power output voltage, signal output voltage, and input voltage range, and produce inaccurate answers. In addition, the AI may generate a hallucinated answer if the datasheet does not provide sufficient source information. In this paper, we propose an AI answer-validation method using agentic RAG for datasheet inquiry for IoT application system deployment. The method organizes datasheet information into structured specification data, including device models, field types, values, units, conditions, and source information. For question-answering, the agent coordinates structured fact query, top-k text retrieval, source checking, and rule-based compatibility comparison according to the question type. Instead of fully relying on the LLM to interpret retrieved chunks, this method adopts structured specifications and deterministic source checks before accepting the final answer. For evaluation, we constructed a dataset from 20 IoT datasheets, including 1000 question-answering tasks with three difficulty levels. Compared with conventional top-k RAG, the proposed method improved the correct answer rate from 0.686 to 0.958 for easy questions, from 0.549 to 0.969 for medium questions, and from 0.273 to 0.613 for hard questions, which confirms the effectiveness of the proposed method.

More from our Archive