Breaking the Sign Symmetry of Attention: A Conflict-Aware Vision–Language Fusion Framework for Privacy-Sensitive Information Detection

As image data are shared ever more openly, they increasingly leak privacy-sensitive information, yet existing detectors seldom model how the text embedded in an image relates to its visual content and tend to fail precisely when the two modalities disagree. We note that the conventional softmax attention used for multimodal fusion carries an implicit sign symmetry: Every source token contributes only additively, so conflicting evidence is averaged away rather than resolved. We propose a symmetric dual-source fusion framework whose decoder deliberately breaks this sign symmetry. Image and text are first encoded by a Swin Transformer and a policy knowledge-enhanced BERT (KL-BERT) and projected into a common space to form a permutation-symmetric dual-source memory. A Signed Cross-attention Auto-compressing Decoder (SCAD) then fuses the two sources through an attention map that factorizes into a sign-symmetric (even) magnitude term and a sign-antisymmetric (odd) polarity term, allowing the model to either reinforce or actively subtract cross-modal evidence. Experiments on a self-constructed privacy-sensitive image dataset show that the proposed method attains an accuracy of 97.69% and an F1-score of 96.70%, with its largest gains on the face category, the most conflict-prone class in our data, suggesting that controlled symmetry breaking is an effective principle for cross-modal fusion.