Martingale Doppelgänger-Eval: A Specification-Driven Interventional Audit Algorithm for Visual Evidence Use in Vision–Language Models

Assessing chart understanding requires distinguishing responses to local visual evidence from associations with a chart’s overall shape. We present Martingale Doppelgänger-Eval, an interventional audit framework that connects executable edit specifications to benchmark generation, validity checks and response estimation. Matched charts support separate measurements of evidence response and regional sensitivity. Each result carries its uncertainty, response coverage and identification status. We characterize conditions for identifying paired effects and specify a sequential procedure for audits that stop after inspecting accumulating evidence. We directly evaluate nine frozen vision–language models on 12,000 pairs generated with the corrected renderer. Complete-pair evidence scores range from 0.4362 to 0.4990, while supervised pixel controls generalize the learned rules to disjoint held-out symbols. A matched volume-only experiment reveals model-specific responses, and magnitude-matched regional edits quantify oracle-region sensitivity separately from the faithfulness of reported regions. The framework links interpretable visual comparisons to the conditions needed for their statistical evaluation.