ALP-Net: Adaptive and Long-Range Processing Network for Temporal Action Localization

Processing and understanding untrimmed videos require a much larger receptive field than trimmed videos. Temporal action localization (TAL) methods generally utilize depthwise convolutional layers or transformer blocks to extract a representation that distinguishes action segments of different classes from one another and from background segments. Convolutional layers provide a limited receptive field unless equipped with large kernels. While transformers provide a large receptive field, they are more computationally complex and harder to optimize. Even though dilated convolutional layers provide a larger receptive field than standard convolutional layers, they have a rigid kernel structure with fixed spacing that can lower performance. Unlike standard dilated convolutional layers, dilated convolution with learnable spacings (DCLS) overcomes the problem of rigidity by formulating both weights and spacings of the grid as learnable parameters. This new formulation provides a more adaptive and efficient approach for increasing the receptive field. In this paper, we propose ALP-Net (Adaptive and Long-Range Processing Network), the first TAL method to utilize DCLS layers. ALP-Net is built around ALP blocks, which combine large-kernel convolutional layers, DCLS layers, and a temporal attention mechanism that emphasizes crucial temporal points. Together, these components yield an adaptive and efficient architecture with a large receptive field. We evaluate ALP-Net on Thumos14, ActivityNet 1.3, Epic-Kitchens-100, HACS Segments and FineAction datasets. The experiments show that ALP-Net achieves results competitive with state-of-the-art methods. For instance, it obtains an average mAP of 73.10% on the Thumos14 dataset when coupled with InternVideo2-6B features.