Localizing failures in AI agent pipelines with structured execution traces

Failures in applications built around large language models may emerge several steps after an incorrect decision, making final responses inadequate for diagnosis. This study evaluates whether preserving the identity of execution-trace fields improves lightweight failure localization beyond their lexical content. A frozen public TraceElephant dataset was analyzed using source artifacts available by 30 April 2026. The eligible cohort contained 219 failed executions, 5,936 candidate steps, and 173 task groups across three agent systems. Regularized logistic rankers were evaluated using five task-grouped folds, matched text budgets, and a shared training-only vocabulary. Separate-field encoding achieved 24.7% exact Top-1 localization, compared with 23.3% for a flat representation containing the same lexical evidence. The paired difference was 1.4 percentage points (95% task-cluster bootstrap interval −1.9 to 5.2). Top-3 coverage increased from 46.6% to 50.7%, although its advantage varied across sensitivity analyses. When an entire agent system was withheld, separate-field Top-1 accuracy fell to 12.8%. A combined ranker placed the annotated step within the selected quarter of records in 67.6% of traces, compared with 18.3% for selecting the latest records. These findings support structured traces as inputs to inexpensive diagnostic triage, while providing no clear evidence of improved exact localization from field separation alone. The results concern agreement with benchmark annotations and do not establish causal diagnosis or reductions in human debugging time.