Region of Interest Extraction Algorithm Development in Video Surveillance Systems: A Review

Video surveillance systems play a vital role in modern security, transportation, and public safety infrastructure. Most of the analytical functions of an intelligent surveillance system, from detection and tracking to ROI-aware compression, begin by isolating a Region of Interest (ROI), yet the algorithms that perform this step are scattered across the computer-vision, video-coding and sensor-network literatures, use the term “ROI” in different senses and report results on metrics that cannot be compared directly. This narrative review classifies and critically analyzes ROI extraction algorithms for video surveillance published between 2020 and 2026. A comparison with survey papers published between 2021 and 2026 shows that none is centered on the extraction step itself. The reviewed methods are organized by explicit criteria (hand-crafted or learned features, learned or online-estimated decision rules, annotation dependence and output granularity) into background-subtraction, classical machine-learning, deep-learning and hybrid families, and the deep-learning family is further divided into six groups. Five semantic variants of the ROI are distinguished, and reported results are compared only within the metric family appropriate to each variant. Widely used video surveillance datasets are compiled, described and compared. The analysis shows that the evaluation base of the field is narrow, with fourteen of the compared studies reporting their principal results on CDnet 2014 and none reporting run-to-run variance. Because the compared studies differ in dataset, subset, training split and averaging, the reported values indicate operating ranges and do not show that any method family, deep learning included, is generally superior to another. Research directions are derived from these limitations, including cross-dataset evaluation with temporal-stability measures, codec-aligned ROI extraction for edge devices, motion-gated promptable segmentation, extraction from moving cameras, and the joint design of ROI extraction and privacy protection.