HRM-Crime: A Hierarchical Relationship Model for Weakly Supervised Crime Anomaly Detection in Surveillance Video

Detecting anomalous behaviour in surveillance video is a high-impact application of machine learning, but supervision is available only at the video level: a set of temporal segments carries a single anomaly or normal label, with no per-segment ground truth. This is the Multiple Instance Learning setting. On the UCF-Crime benchmark, published weakly-supervised methods have converged to a narrow band of Receiver Operating Characteristic Area Under the Curve between 0.85 and 0.87 over the past three years, despite the adoption of progressively larger Vision Transformer backbones. We investigate whether this ceiling reflects a limit on feature quality or a limit on prediction diversity. We introduce HRM-Crime, a compact hierarchical score model of approximately 270 thousand trainable parameters that couples windowed self-attention for local temporal structure with dilated one-dimensional convolutions for long-range temporal context. Segment features are pre-extracted with a frozen ResNet-50 for appearance and a frozen R3D-18 for motion. On this base we evaluate two orthogonal interventions: aggregating each model’s segment scores by the mean of its K highest values rather than the maximum, and fusing three models trained on different feature subsets through rank normalisation. Across 98 training runs a single model attains a mean Area Under the Curve of 0.8934 with standard deviation 0.0087. The three-stream ensemble attains 0.9300, an improvement of 0.0213 over the strongest single model, with a bootstrap 95 percent confidence interval of 0.0079 to 0.0360 and a one-sided p-value of 0.0008. Five negative results are documented. We note that the reported gain confounds architecture, feature representation and ensembling, and we identify the experiment required to separate them.