Agricultural Field Crop Type Semantic Segmentation and Boundary Extraction from Sentinel-2 Time Series Using Multitask Learning with Directional Feature-Sharing

Precise analysis of crop fields is essential for agricultural management. Various remote sensing tasks comprise the characterization of agricultural lands. The automatic extraction of boundaries and the crop type semantic segmentation tasks, both benefiting from satellite image time-series analysis, are crucial for continuous field monitoring. Recent models explore the multitask setting for boundary detection, but they do not incorporate class information that is crucial when several kinds of agricultural crops co-exist. Inspired by the idea that crop type segmentation and boundary extraction are closely connected, this study exploits a multitask learning framework for both agricultural segmentation and boundary detection tasks. The model uses a shared encoder with 3D convolutional blocks and task-specific decoders with cross-task feature exchange to simultaneously solve the two main tasks of crop type segmentation and boundary extraction. The model also leverages on estimating the distance to the closest border as an auxiliary task to improve training. The model effectively shares features across tasks and solves them simultaneously, achieving detailed crop-field characterization. To validate the method performance and examine inter-task relationships, two datasets composed of time series of Sentinel-2 images acquired from two agricultural areas in Austria, spanning two different years, were used.