Intrusion Detection in Sensor-Cloud Networks: A Reproducible Multi-Seed Comparison of XGBoost and a CNN–LSTM–Transformer Model

Intrusion detection (IDS) is needed for sensor-cloud networks (SCN) and other distributed computing systems. Many of the results published, however, may have been misleading in part because they were based on the same data records, information leakage, partitioning of the datasets with unequal sample sizes, and on the execution of only one experiment. This paper provides a replicable experiment that compares the performance of XGBoost and a hybrid CNN–LSTM–Transformer model on CICIDS2017 flow-level dataset. 2867 rows were removed because they contained non-finite values, and duplicate flows were removed to yield 2,520,798 samples with 77 input features. The data was randomly split, using 1,764,558 samples for training, 252,080 samples for validation, and 504,160 samples for testing, with no samples common to more than one partition. The scaling parameters were fitted only to the training set; the validation and test sets were not resampled. The validation set was used to select thresholds by maximizing the Matthews correlation coefficient (MCC) and the test set was used for reporting. Five randomly seeded experiments were used to evaluate the models. XGBoost had a mean accuracy of 99.9132% and a mean MCC of 0.996914, while the hybrid model had a mean accuracy of 98.8220% and a mean MCC of 0.958924. The mean training time for XGBoost and hybrid models was 16.31 s and 1552.13 s, respectively. These results suggest that the more complex model did not improve the performance of the simpler model on the given tabular flow data. The analysis in general was centered on duplicate data, partitioning, multiple randomly seeded experiments, computational complexity and the sensitivity of the results when classes are under-represented in the test set.