Generative AI in financial mathematics: Evaluating LLMs on a bond-focused benchmark dataset

This paper presents a comprehensive evaluation of the performance of large language models (LLMs) on a newly created set of 50 tasks in financial mathematics, with a focus on bond valuation. Models o3, Claude Opus 4, DeepSeek-R1, and Gemini were tested on three types of tasks (practical, real-life, and theoretical) and three difficulty levels (easy, medium, and hard). Model o3 achieved the highest average accuracy (80.9%), while DeepSeek-R1 showed the weakest results (70.6%). The detailed analysis included error classification (methodological, numerical, conceptual), measuring solution time as well as the application of ANOVA and post-hoc tests, which confirmed statistically significant differences between the models. The results indicate that more complex and theoretical tasks remain challenging for all models. The proposed methodology and the set of created tasks provide a basis for further research and for the development of specialized LLM systems for financial analysis and educational purposes.