Can LLMs automate data extraction in agronomic meta‐analyses? A feasibility study on cover crops and soybean yield

Abstract

Large language models (LLMs) can streamline article screening and data extraction to accelerate meta‐analysis synthesis; however, their use remains limited in agrisciences. In this study, we evaluated the potential of LLMs to accurately extract data from articles of a published meta‐analysis. We developed a multi‐agent LLM (GPT and Gemini) pipeline that screened and extracted data from a set of 50 peer‐reviewed studies. When the LLMs disagreed, a structured human‐in‐the‐loop protocol was implemented to resolve the inconsistencies. Then, the pooled effect size estimates generated from the LLMs and human‑in‑the‑loop workflow were analyzed and compared to the meta‑analysis. Of 671 effect sizes reported in the published meta‐analysis, GPT recovered 180 effect sizes, and Gemini extracted 244 effect sizes, while the human‐in‐the‐loop recovered 293 effect sizes. Human reviewers had to verify 71.5% of all extracted effect sizes. The remaining 28.5% of the extracted effect size cases did not require manual review. These results demonstrate limited efficiency of GPT and Gemini to synthesize and extract data from articles regarding the effect of cover crop termination timing on soybean [ Glycine max (L.) Merr.] yield. Accurate data extraction remains dependent on literature reporting practices, highlighting the need for standardized, machine‐readable formats to support reliable AI‐assisted evidence synthesis in agricultural science.