An Attention Aware Pooling Strategy for Efficient Multi‐Label Classification of Aerial Forest Imagery

ABSTRACT

Accurate multi‐label classification of aerial forest imagery is a critical component of large‐scale forest monitoring, biodiversity assessment and climate‐related decision making. Unlike single‐label scenarios, natural forest scenes exhibit strong class co‐occurrence, severe class imbalance and spatially heterogeneous label relevance, which challenge conventional deep learning pipelines. Commonly, fixed pooling strategies are adopted such as global average pooling that implicitly assumes uniform spatial importance of features, an assumption that is poorly aligned with the multi‐label nature of complex aerial imagery. In this work, we investigate the role of feature aggregation and adapt an established CLS‐token‐driven attention pooling strategy in transformer for multi‐label classification of aerial images and propose a task‐adaptive attention‐based token pooling framework. Built upon a Swin‐Base Transformer backbone, the proposed method replaces static global pooling with a learnable Classification (CLS) token driven multi‐head self‐attention pooling mechanism that adaptively aggregates spatial features according to their semantic relevance. The adaptive attention‐based aggregation may help preserve discriminative signals associated with minority and co‐occurring classes that can otherwise be diluted by fixed pooling. Comprehensive experiments were performed on TreeSatAI benchmark to demonstrate the effectiveness of the proposed attention‐based pooling strategy. Moreover, it improves the Recall and F1‐score compared to both the standard Swin‐Base architecture and a wide range of Convolutional Neural Network (CNN) as well as transformer baselines under the same training conditions for the models reproduced in the present study. The detailed class‐wise analysis further provides evidence of the potential suitability of adaptive feature aggregation for multi‐label aerial forest classification. The results suggest that pooling strategy is a critical yet under‐explored design choice for transformer‐based remote sensing models and that task‐adaptive token aggregation provides a simple and potentially effective means to improve multi‐label classification performance in the evaluated TreeSatAI setting.