Humanoid Tennis Robot Arrives! Developed by Tsinghua University and Galaxea AI
Recently, a research achievement on humanoid robots playing tennis has attracted attention. This robot, named LATENT, was jointly developed by a team from Galaxea AI, Tsinghua University, Peking University, Shanghai Qi Zhi Institute, and Shanghai Artificial Intelligence Laboratory.
LATENT stands for "Learning Athletic Humanoid Tennis Skills from Imperfect Human Motion Data," meaning "learning athletic humanoid tennis skills from imperfect human motion data."
The research team pointed out in the paper that human athletes can perform high-speed tennis rallies thanks to diverse and highly dynamic motor skills. However, replicating such behaviors on humanoid robots is extremely difficult, with one core bottleneck being the lack of perfect human-like motion data or human kinematic data specific to tennis scenarios as reference.
To address this, the team proposed the LATENT system, which learns athletic humanoid tennis skills from imperfect human motion data. These imperfect data are not complete action sequences from real matches, but only motion segments containing the raw skills used when playing tennis, greatly reducing the difficulty of data collection.
The team's core insight is that although these data are imperfect, such "quasi-real" data are still valuable, providing raw skill information from humans in tennis scenarios. After correction and combination, the strategies learned by the robot can continuously hit incoming balls under varying conditions, return them to target positions, while maintaining a natural motion style.
Currently, this strategy has been successfully deployed on Unitree's G1 humanoid robot, achieving impressive real-world performance—the robot can now stably engage in multi-shot rallies with human players.
Zhikai Zhang, the first author of the paper and a PhD student at Tsinghua University, shared his development experience on social media:
"On the first day of field deployment, it couldn't catch any of the balls I served. By the last day of the project, I couldn't beat it anymore (though it's still far from professional athletes)."
Haofei Lu, the second author of the paper and a master's student at Tsinghua University, also reflected on the project's journey:
"LATENT explores the dynamic hitting ability of humanoid robots in real tennis scenarios. In a real court of about 200 square meters, the robot needs to perform large-range running, hitting decisions, whole-body coordination control, and stably return high-speed tennis balls with a diameter of less than 6.5 cm. Such high-dynamic tasks not only require effective learning algorithms but also place extremely high demands on system engineering and real-world deployment.
In this work, we not only focused on strategy learning itself but also invested a lot of effort in the engineering implementation of the real system. From generative strategy training to real robot deployment, we built and continuously iterated a complete Sim2Real pipeline, including strategy inference deployment, communication system stability, real-data-driven Real2Sim iteration, system calibration, and environment modeling, so that strategies learned in simulation can run stably on real robots.
After about five months of system development and experimental iteration, we conducted extensive tests on real-world courts, enabling the robot to perform running, footwork adjustment, and hitting actions in continuous incoming balls, achieving stable returns. We hope this work demonstrates that in humanoid robot research, algorithmic breakthroughs and engineering systems are equally important. Only through solid real-world deployment and system iteration can robot capabilities truly step out of simulation environments and into the real world."
| Image source: Video screenshot
| Information source: GitHub; Authors' personal social media accounts