Virtual-Dynamics-Based Motion Planning for Industrial Manipulators via Integrating Information from Multiple High-Speed Sensors

High-speed sensors, such as high-speed cameras and optical proximity sensors, enable the detailed temporal measurements of physical phenomena that exceed the dynamic capabilities of conventional industrial robots. However, effectively leveraging this sensor information for robot motion planning remains challenging because of the temporal-scale gap between sensors and robots. This paper proposes a motion planning method that extracts the task-relevant meta-information of target phenomena from high-speed sensor data and generates feasible trajectories by considering robot constraints. The information extraction process identifies task-relevant characteristics from high-speed sensor data. To integrate heterogeneous sensor information and enable trajectory adaptation, we employed multiple virtual-dynamics-based control (MVDC), which can asynchronously integrate heterogeneous sensors with different measurement principles. To validate the proposed method, we conducted a case study in which a conventional industrial manipulator grasped a pendulum at its equilibrium point, the most challenging position. The system integrated global measurements from a 1 kHz high-speed camera with local measurements from proximity sensors using MVDC to predict the pendulum period and optimal grasping timing. Experimental results demonstrated that the proposed method enables successful grasping by bridging the temporal-scale gap between high-speed sensors and conventional robots through information integration.