Discover the use of multimodal language models for idea detailing in human-AI collaborative design

ABSTRACT:

In this work, we propose a multimodal, language-model–based design assistance framework for the design ideation stage. The framework leverages large language models (LLMs) to interpret user intentions with mood boards, enrich initial ideas with essential contextual details, and produce structured instructions for visual language models (VLMs) to enhance the accuracy and consistency of visual feedback.