Code-driven planning in grid worlds with large language models

Abstract

We propose an iterative programmatic planning (IPP) framework that solves grid-based tasks by using large language models (LLMs) to synthesize interpretable, executable programs that act as agent policies mapping environment states to action sequences, rather than relying on traditional search or reinforcement learning. Alongside prompting strategies such as direct code generation, pseudocode conditioned refinement and curriculum-based prompting, IPP centres on an iterative refinement mechanism that updates code from coarse, task-level feedback. We evaluate our approach using six leading LLMs and two challenging grid-based benchmarks (grid-based reasoning for assessing spatial planning (GRASP) and MiniGrid). Our IPP framework demonstrates improvements over direct code generation for five of the six models, with net energy gains ranging from a few per cent to roughly threefold and, for Gemini-2.5-Pro, a sign flip from a negative baseline to a strongly positive score, establishes a new state-of-the-art result for GRASP and significantly outperforms direct elicitation from GPT-o3-mini (by 43% on the hardest MiniGrid task to 116% on GRASP). While code generation has a higher initial prompting cost compared to direct solution elicitation ($0.08 per task versus $0.002 per instance for GPT-o3-mini), the code can be reused for any number of instances, making the amortized cost significantly lower (by 400 times on GPT-o3-mini across the complete GRASP benchmark). Beyond performance gains, IPP produces reusable, human-readable policies that support transparent inspection and modification, underscoring the advantages of planning as program synthesis over per-instance inference. The code and results can be found at http://github.com/Mystic-Slice/LLM-Planning-CodeGen.