Fake Review Detection from Classical to Transformers: Gains and Limits

ABSTRACT

Reported fake review detection performance is hard to interpret when model families are evaluated under different data splits and development procedures. This study compares six traditional machine learning systems, five deep learning models, seven pretrained Transformer encoders, and seven sentiment‐augmented Transformer variants on the Amazon‐based fake reviews dataset (FRD) under one leakage‐controlled protocol: a duplicate‐safe 70/10/20 partition; validation‐only selection of checkpoints, thresholds, models, and ensembles; and a test set reserved for final evaluation and paired inference. Accuracy rises from 94.32% (TF‐IDF + SVM) and 95.00% (Word2Vec + LSTM) to 98.23% with the validation‐selected plain Transformer (ModernBERT), the largest stage‐level gain. A validation‐selected soft‐voting ensemble adds a small but significant improvement, to 98.75% (ROC‐AUC 0.9991). Sentiment augmentation, an explicit sentiment profile added through a dedicated fusion head, did not improve accuracy consistently: Over five training runs per backbone, its effect changed sign for all seven encoders, including the three with a significant first‐run effect. Zero‐shot evaluation on AiGen‐FoodReview, which replaces Amazon/GPT‐2 with Yelp/GPT‐4‐Turbo reviews, shows that the in‐domain operating performance does not carry over: Every transferred Transformer‐based system labels nearly all machine‐generated reviews as authentic, and the final ensemble ranks the two classes in reverse (ROC‐AUC 0.4357). No configuration ranks generated reviews above authentic ones in all five runs; plain BERT reaches a ROC‐AUC of 0.9055 in one run but only 0.30–0.77 in the others. Controlled in‐domain gains and external validity must be established separately.