Fine‐tuning XLNet for Amazon review sentiment analysis: A comparative evaluation of transformer models

Abstract

Transfer learning in large language models adapts pretrained models to new tasks by leveraging their existing linguistic knowledge for domain‐specific applications. A fine‐tuned XLNet, base‐cased model is proposed for classifying Amazon product reviews. Two datasets are used to evaluate the approach: Amazon earphone and Amazon personal computer reviews. Model performance is benchmarked against transformer models including ELECTRA, BERT, RoBERTa, ALBERT, and DistilBERT. In addition, hybrid models such as CNN‐LSTM and CNN‐BiLSTM are considered in conjunction with single models such as CNN, BiGRU, and BiLSTM. The XLNet model achieved accuracies of 95.2% for Amazon earphone reviews and 95% for Amazon personal computer reviews. The accuracy of ELECTRA is slightly lower than that of XLNet. The exact match ratio values for XLNet on the AE and AP datasets are 0.95 and 0.94, respectively. The proposed model achieved exceptional accuracy and F1 scores, outperforming all other models. The XLNet model was fine‐tuned with different learning rates, optimizers (such as Nadam and Adam), and batch sizes (4, 8, and 16). This analysis underscores the effectiveness of the XLNet approach for sentiment analysis tasks.