ResEdit: Residual embeddings for precise generative image editing

Abstract

Conditional diffusion image generators can be repurposed for editing through inversion, without the need for large‐scale paired fine‐tuning data. However, producing high‐quality, targeted edits while maintaining image identity and global consistency remains challenging, as weakly conditioned inversion often embeds conflicting image features into the noise. We demonstrate that incorporating a residual image encoding as additional conditioning enables both improved identity preservation and better editability. We optimize this residual encoding to provide a strong conditioning signal for reconstruction, thereby reducing the reliance on inversion and susceptibility to its aforementioned pitfalls. To ensure this residual does not interfere with desired edits, we incorporate a gradient reversal‐based optimization strategy that disentangles the residual from the edited condition. We illustrate our method's ability to produce high‐fidelity results across precise intrinsic‐based editing and relighting, and show proof‐of‐concept text‐guided manipulation. Project page: johnberg1.github.io/resedit