Point‐StagePrompt: Layer‐Adaptive Three‐Stage Prompt Learning for Cross‐Modal Point Cloud Understanding

ABSTRACT

Recent advances in prompt learning have shown promising results in adapting large multimodal models to 3D point cloud understanding in a parameter‐efficient manner. However, existing approaches typically employ uniform and layer‐agnostic prompting strategies, overlooking the hierarchical nature of Transformer representations and the distinct roles of different layers in multimodal feature learning. As a result, their performance degrades noticeably under distribution shifts and unseen‐class settings. To address this limitation, we propose Point‐StagePrompt, a layer‐adaptive three‐stage prompt learning framework that explicitly aligns prompt design with the hierarchical representation structure of large multimodal models. Our method progressively introduces prompts across shallow, intermediate and deep layers to preserve foundational knowledge, refine intra‐modal representations and enable effective cross‐modal alignment. Furthermore, we design a Point‐to‐Text Prompt Mapper to facilitate flexible and efficient cross‐modal interaction. Extensive experiments on five benchmark datasets demonstrate that Point‐StagePrompt achieves state‐of‐the‐art performance in base‐to‐new generalisation, cross‐dataset transfer and few‐shot learning, surpassing previous methods such as Point‐PRC by up to 9.99% harmonic mean under challenging base‐to‐new evaluation settings while maintaining high parameter efficiency. These results validate the effectiveness of structured, layer‐aware prompt learning for robust 3D point cloud understanding.