NFH‐26‐03: Data‐efficient robot learning for contact‐rich manipulation

Abstract

Robots have the potential to assist people in homes, hospitals, warehouses, and factories, but today's systems still struggle to learn robust manipulation skills from limited data, especially in contact‐rich settings involving clutter, deformable objects, dexterous hands, and coordinated multi‐arm behaviors. My research aims to make robot learning more data‐efficient and physically grounded by combining three complementary directions: synthetic data augmentation, multimodal sensing, and semantic reasoning with foundation models. First, we develop diffusion‐based augmentation methods that expand scarce robot demonstration datasets while preserving geometric and contact consistency. Second, we design learning algorithms and robotic platforms that use vision, touch, force, and other sensory signals to reason about complex physical interactions. Third, we leverage and benchmark Vision‐Language Models for manipulation, using their semantic knowledge to guide contact‐rich planning while identifying their limitations in low‐level physical reasoning. Together, these efforts move toward robots that can learn from fewer demonstrations, understand when and how to make contact, and operate more reliably across cluttered and unstructured real‐world environments.