Event‐Based Video Reconstruction via Bidirectional Spatiotemporal Learning

ABSTRACT

As event cameras capture dynamic scenes with high temporal resolution and high dynamic range, event‐based video reconstruction has become a popular research focus. Most existing methods primarily adopt unidirectional recurrent networks to integrate temporal information, but they suffer from suboptimal performance due to error accumulation across stages. Additionally, the inherent sparsity of event streams leads to incomplete spatial coverage, degrading frame quality in low‐texture regions. To address these challenges, we propose a novel hierarchical framework (ST‐E2V) that integrates bidirectional temporal context with the spatial complementarity of multiple event representations and historical prediction, enabling joint prediction of multiple frames within a sliding temporal window. Specifically, we design a temporal context collaboration (TCC) module capable of utilizing both past and future temporal context to mitigate error accumulation across time. Meanwhile, we introduce a historical prediction‐guided spatial complementary (HSC) module that improves detail preservation in low‐texture regions by leveraging visual information provided by historical predictions to compensate for the sparsity of event streams. Subsequently, a spatiotemporal attention (STA) module is introduced to integrate long‐term spatiotemporal information. Experiments on real datasets demonstrate that our method achieves state‐of‐the‐art performance. Furthermore, comprehensive evaluations involving practical applications validate the effectiveness and superior performance of our method.