CI-DIOR-7: A Task-Oriented Benchmark and Failure Analysis for Critical Infrastructure Detection in Remote Sensing Imagery

Accurate detection of critical infrastructure in high-resolution remote sensing imagery is important for infrastructure inventory, regional monitoring, and risk assessment. However, differences in object scale, spatial density, geometric form, and background complexity make it difficult to evaluate detectors using a single aggregate metric. This study develops a diagnostic evaluation framework utilizing CI-DIOR-7, a task-oriented benchmark derived from the DIOR dataset by retaining horizontal bounding-box annotations for seven infrastructure categories: Airport, Harbor, Bridge, Dam, Train Station, Storage Tank, and Windmill. The benchmark contains 23,463 images and 44,579 instances and preserves the original training, validation, and test splits. Four representative detectors—YOLO11s, RT-DETR-L, Faster R-CNN R50-FPN v2, and RetinaNet R50-FPN v2—were evaluated under a unified COCO-style evaluation protocol, modified with a maximum detection limit of 500 (maxDets = 500) to accurately evaluate highly dense infrastructure scenes. On the official test set of 11,738 images and 33,766 instances, RT-DETR-L achieved the highest mAP@0.5:0.95 of 0.3212 and the smallest validation-to-test performance drop. YOLO11s obtained an mAP@0.5:0.95 of 0.2785 while providing the lowest inference latency and the lowest false-positive burden on zero-GT images, at 0.168 false positives per image. Class-wise and difficulty-oriented analyses showed that Bridge was the most challenging category, with a mean class AP of 0.1514 and small-bridge recall ranging from 0.105 to 0.161 across the four models. Storage-tank detection was strongest in moderately dense scenes but deteriorated when more than 20 instances occurred in one image. These results show that no single architecture simultaneously achieved the highest detection accuracy, the lowest measured inference latency, and the lowest false-positive burden under the zero-GT protocol. CI-DIOR-7 therefore provides a reproducible benchmark and diagnostic framework for evaluating critical infrastructure detectors beyond overall mAP.