Kinematic Control for the Motion Generation of Robot Manipulators Using MoMask LLM

In recent years, numerous Large Language Models have been developed, offering a wide variety of applications. Among these, motion generation for 3D human representations has gained attention due to its potential in simulations, animations, and video games. However, the application of LLMs to robot motion is still in its early stages. This paper presents the use of an LLM to generate motion for a robot manipulator simulated in RViz within ROS. We obtain human animated motion from a pre-trained LLM and retarget it to a manipulator robot in the task space, using as example the UR5 and Sawyer robots. This reference is then used as input to a nulls pace hierarchical kinematic controller that follows the hand and elbow motion. The results show that the proposed approach generates motion that resembles the movements of a human character, achieving a maximum error of 0.04 m for the UR5 and 0.03 m for the Sawyer Robot.