DOI: 10.3390/sym18111789 ISSN: 2073-8994

DualBridgeMem: Structured Visual Semantic Consolidation for Persistent Multimodal Conversational Memory

Hanyu Ding, Zhengyu Xia, Yawei Du, Mingming Chen

Persistent multimodal conversational memory requires unified access to textual and visual evidence across sessions, yet differences between image content and textual queries make relevant sources harder to locate. We propose DualBridgeMem, a structured visual semantic consolidation method that generates long and short visual descriptions at memory-writing time using a nine-aspect observation checklist, preserving detailed facts and compact discriminative information, respectively. The descriptions, source text, and original-media references are packaged into MemoryNotes for evidence retrieval through a shared semantic and lexical scoring interface, with access to original images and source traceability. We evaluate the method on LoCoMo and Memory-QA through retrieval-input ablations, comparisons with original-image embedding baselines, and paired statistical analysis. Relative to source text alone, the configuration combining source text with long and short visual descriptions has a higher point estimate for each of R@1, R@10, and answer F1 on both datasets. The respective differences are 7.14, 7.30, and 1.56 percentage points on LoCoMo, and 1.53, 0.45, and 1.14 percentage points on Memory-QA. Comparisons with original-image embedding baselines, including image-only CLIP, further support the effectiveness of this retrieval path for evidence localization. These findings provide empirical support for structured visual descriptions as an interpretable and traceable retrieval interface for persistent multimodal memory.

More from our Archive