DOI: 10.35377/saucis...1951015 ISSN: 2636-8129

Multimodal Aspect-Based Sentiment Analysis for Vietnamese Reviews using Learnable Aspect Queries

Sneha Kaware, Rajeshri Shelke, Poonam Manjare
Show PDF Cite
The field of Multimodal Aspect-Based Sentiment Analysis (MABSA) has attracted considerable interest due to its unique ability to jointly leverage both textual and visual modalities for better aspect-level sentiment prediction. Nonetheless, there is no consideration of the dependence on particular modalities in practical application, especially when textual information is vague or missing in the data. Such cases are typical in many areas, including hotel reviews and e-commerce, where people use multimodal data to express their opinions. In this study, we introduce an aspect-centric MABSA approach that combines a transformer-based text encoder (PhoBERT) with a Vision Transformer (ViT) in a single end-to-end system. To enable fine-grained interaction between modalities, we apply aspect queries that dynamically attend to modality-dependent features, implicitly aligning them to obtain aspect-related information. Additionally, our model is tested under uncertainty in textual content by applying several text masking methods, including random token-level masking, sentiment-based masking, and evaluation-time gating. Experimental results on the ViMACSA dataset showed promising results, with a Macro-F1 score of 0.87. Additionally, we perform experiments by ablating modality, assessing aspects individually, analyzing robustness to noisy inputs, and comparing against current multimodal architectures. Results suggest that the proposed approach achieves competitive performance compared with current methods, especially for aspects based on visual grounds.

More from our Archive