LLMs in the Laboratory: Benchmarking Conversational AI for Multivariate Analysis of NMR Spectroscopic Data

Abstract

This study presents a comparative evaluation of five state-of-the-art large language models (LLMs) (Claude Opus 4.6, Gemini 3.1 Pro, Grok-4, ChatGPT (GPT-5.4 Thinking), and Qwen 3.5) for their ability to perform key chemometric tasks through natural language interaction. Using a low-field nuclear magnetic resonance (LF-NMR) dataset of edible vegetable oils, the models were prompted to execute a complete multivariate analysis workflow, including data exploration, unsupervised and supervised classification, model validation, and interpretation. Results reveal a clear performance hierarchy: Claude Opus 4.6 was the only model to autonomously execute the entire pipeline with the highest analytical rigor, correctly predicting all nine unlabeled samples. Gemini 3.1 Pro and ChatGPT (GPT-5.4 Thinking) also demonstrated strong capabilities, achieving 100% or near-100% classification accuracy within fallback or native workflows, positioning them as effective analytical co-pilots. Grok-4 produced near-perfect classification results but showed limitations in data handling autonomy and exploratory depth. Qwen 3.5, the only open-source model, suffered a critical, unrecoverable failure at the supervised classification stage after repeated attempts, preventing workflow completion. Overall, the results indicate that linguistic fluency should not be taken as a reliable indicator of analytical competence and that general-purpose benchmarks may have limitations when assessing domain-specific chemometric performance. Critical validation and human oversight remain essential for the trustworthy deployment of these tools in spectroscopy-based workflows.