Document-Level Transformer-Based Text Semantic Communication System

Given the emergence of new applications with stringent networking requirements, traditional bit-level communication may reach theoretical Shannon capacity and struggle to support such applications. Therefore, the idea of semantic communication (SC) has been proposed in literature. SC involves the use of artificial intelligence and a shared knowledge base to send a representation of data and reconstruct it at the receiver. In this paper, a SC system for document transmission is proposed. The system uses a transformer and a context encoder module to preserve the document’s knowledge and context information by quantifying the relationship between a current sentence and the preceding sentences. The semantic information is sent over a noisy channel, and a mutual information maximization model is used to reduce the effect of noise on the transmitted signal. The proposed system is trained in two stages; the first stage involves training the sentence-level transformers while the second stage involves training the sentence-level and the context-level transformers. The performance of the system is evaluated using two publicly available datasets. The results show that the performance of the proposed system is better than a sentence-level SC model in terms of BLEU score and sentence similarity over a Rayleigh fading channel.