InstructGrasp: A Real-Time Language-Guided Grasp Pose Detection for Robotic Manipulation

초록

Language-guided robotic grasping is a key enabler of intuitive human-robot interaction in unstructured environments. Although current state-of-the-art (SOTA) methods achieve strong grounding accuracy, they increasingly rely on Vision-Language Models (VLMs) with hundreds of millions of parameters, imposing computational burdens that hinder real-time deployment on resource-constrained robotic edge devices. We present InstructGrasp, a parameter-efficient framework for real-time language-guided grasp detection. InstructGrasp introduces Query-Centric Grouped-Query Fusion (QGQF) and an Instruction-Modulated Decoder (IMD) to achieve precise visual-semantic alignment. It attains SOTA success rates of 95.0% (Seen) and 92.0% (Unseen) on Jacquard and 58.0% (Unseen) on Grasp-Anything++. On OCID-VLG, InstructGrasp reaches a 91.08% Seen and 86.49% zero-shot Unseen success rate, matching or exceeding fully supervised heavy-weight models such as OGRG (90.81% Seen) and LAVT (87.55% Seen), while using only 98.3 M total parameters (32.31 M trainable) versus 147M–240 M for these baselines. Operating at 31.04 ms, InstructGrasp attains this accuracy while preserving real-time inference speed. Beyond benchmark evaluation, we validate InstructGrasp on a real Franka Emika Panda robot, achieving a 77% success rate in cluttered visual-only grasping and 43% in instruction-driven cluttered grasping, outperforming all compared baselines in both settings.

키워드