A Survey on Parallel Reasoning
Ziqi Wang, Boye Niu, Zipeng Gao, Zhi Zheng, Tong Xu, Linghui Meng, Zhongli Li, Jing Liu, Yilong Chen, Chen Zhu, Hua Wu, Haifeng Wang, Enhong ChenAbstract
As Large Language Models (LLMs) evolve, parallel reasoning has emerged as a vital inference paradigm that enhances robustness by concurrently exploring multiple thought trajectories. Unlike fragile sequential methods, parallel reasoning expands inference breadth to significantly improve problem-solving performance. This paper provides a comprehensive survey of the progress and challenges in this burgeoning field. We first formally define parallel reasoning and distinguish it from sequential paradigms like Chain-of-Thought. Then, we propose a novel taxonomy to categorize advanced techniques into non-interactive reasoning, interactive collaboration, and efficiency-oriented decoding strategies. Furthermore, we examine diverse application scenarios, including complex problem-solving and reliability enhancement. Finally, we identify core challenges and outline future research directions. This work serves as a strategic roadmap to foster further innovation in parallel reasoning.