Design and development of a palletizing robot with an intelligent 3D vision system based on RF-DETR model

This paper presents the design and implementation of a 4-degree-of-freedom (4-DOF) palletizing robotic arm integrated with an intelligent 3D vision system based on RF-DETR. The system targets the challenges of object detection and manipulation in unstructured industrial environments characterized by clutter, occlusion, and object variability. The robot arm features a lightweight, high-stiffness structure driven by AC servo motors and harmonic gearboxes, controlled by a Mitsubishi FX5U PLC. High-level vision processing is performed on an external computer connected via the RS-485 protocol. The RF-DETR model is trained on a custom dataset of common retail and logistics objects, including boxes, bottles, and deformable packages with diverse shapes, colors, and materials. Bounding box outputs are projected into 3D grasp coordinates using depth data from an Intel RealSense camera and converted to robot coordinates through calibrated transformations. Compared with YOLOv12 and LW-DETR, the proposed model achieved superior detection performance (mAP@0.5:0.95 = 0.78), particularly under partial occlusion, where YOLOv12 failed. The system was validated through 2,000 grasping trials across ten object categories, achieving an overall success rate of 98.75%. All rectangular objects were grasped with 100% accuracy, while minor errors occurred with cylindrical items due to slippage. These results demonstrate the proposed system's robustness in complex scenarios and its practical viability for cost-effective, AI-driven palletizing in real-world industrial settings.