Deep learning architectures for tennis action recognition: Benchmark for cross-sport video analysis

Sports video analysis has evolved from niche methodologies to expansive technological frameworks that integrate multiple sports disciplines. This research provides a comprehensive comparative analysis of four deep learning architectures: ConvLSTM, Long-term Recurrent Convolutional Networks (LRCNN), Optical Flow with LSTM, and C3D-Lite, specifically for detailed tennis action recognition using the Three dimEnsional TennIs Shots THETIS dataset. The experiments were performed over five random seeds and three stratifications of data into three skill sets: S1 Mixed (all players), BS2 Beginner-Only (Players 1–31), and BS3 Expert-Only (Players 32–55), using common train-validation-test splits to ensure the reproducibility of results over different architectures. The LRCNN provided the best performance (Verified with results in Table 55 65%) and a generalization gap of 14 percentage points due to the use of pretrained features of the InceptionV3 architecture. The C3D-Lite architecture demonstrated the most accurate-efficiency trade-off in terms of mean test accuracy 55.25% ± 3.5% for S1. The Optical Flow + LSTM combination provided validation accuracy of 57.8% (S1), 58.2% (BS2), and 63.8% (BS3), whereas training accuracy was 100% for each seed and strategy, which implies full memorization of the training set. ConvLSTM suffered severe overfitting (∼40% training-to-test gap) and is not recommended for limited-scale video datasets. Cross-seed McNemar analysis confirmed result reproducibility across all strategies. Performance remained consistent across skill-stratified partitions, indicating that mixed-skill datasets are sufficient for deployment. This work establishes the first multi-architecture benchmark on the THETIS dataset with full computational profiling, cross-sport applicability analysis, and deployment guidance for racquet sports analytics.