Nested Spatiotemporal Anomaly Detection with Semantic Augmentation: A Case Study in Heritage Conservation

Cultural heritage (CH) sites are continuously exposed to pressures that can lead to deterioration. Continuous monitoring and anomaly detection (AD) enable early damage detection for preventive conservation. We propose a late-fusion AD framework combining NST-Net, a nested spatiotemporal neural network, with a semantic module encoding expert conservation rules, and evaluate it on five real-world CH monitoring datasets. NST-Net outperforms six state-of-the-art baselines on most sites, achieving, on average, 42% higher Average Precision than the best baseline per site under a synthetic evaluation protocol emphasizing sharp, short-duration anomalies. Dataset length serves as an important performance factor: the combined architecture and preprocessing pipeline particularly benefit longer campaigns, whereas they add little value on shorter ones. NST-Net also requires approximately 13 times less peak memory than baseline detectors, at a higher but still millisecond-scale latency. Further analysis shows that performance depends strongly on anomaly density, type, and severity. The semantic module passes label-free sanity checks and complements NST-Net effectively (complementarity index > 0.87), identifying conservation-relevant events missed by the deep model. These findings support late-fusion statistical–semantic frameworks for AD in CH monitoring.