Ping pong bot returns shots with high-speed precision

The new table tennis bot comprises a multijointed robotic arm that is fixed to one end of a ping pong table and wields a standard ping pong paddle. Aided by several high-speed cameras and a high-bandwidth predictive control system, the robot quickly estimates the speed and trajectory of an incoming ball and executes one of several swing types — loop, drive, or chop — to precisely hit the ball to a desired location on the table with various types of spin.

In tests, the engineers threw 150 balls at the robot, one after the other, from across the ping pong table. The bot successfully returned the balls with a hit rate of about 88 percent across all three swing types. The robot’s strike speed approaches the top return speeds of human players and is faster than that of other robotic table tennis designs.

Now, the team is looking to increase the robot’s playing radius so that it can return a wider variety of shots. Then, they envision the setup could be a viable competitor in the growing field of smart robotic training systems.

Beyond the game, the team says the table tennis tech could be adapted to improve the speed and responsiveness of humanoid robots, particularly for search-and-rescue scenarios, and situations in a which a robot would need to quickly react or anticipate.

“The problems that we’re solving, specifically related to intercepting objects really quickly and precisely, could potentially be useful in scenarios where a robot has to carry out dynamic maneuvers and plan where its end effector will meet an object, in real-time,” says MIT graduate student David Nguyen.

Nguyen is a co-author of the new study, along with MIT graduate student Kendrick Cancio and Sangbae Kim, associate professor of mechanical engineering and head of the MIT Biomimetics Robotics Lab. The researchers will present the results of those experiments in a paper at the IEEE International Conference on Robotics and Automation (ICRA) this month.

Precise play

Building robots to play ping pong is a challenge that researchers have taken up since the 1980s. The problem requires a unique combination of technologies, including high-speed machine vision, fast and nimble motors and actuators, precise manipulator control, and accurate, real-time prediction, as well as higher-level planning of game strategy.

“If you think of the spectrum of control problems in robotics, we have on one end manipulation, which is usually slow and very precise, such as picking up an object and making sure you’re grasping it well. On the other end, you have locomotion, which is about being dynamic and adapting to perturbations in your system,” Nguyen explains. “Ping pong sits in between those. You’re still doing manipulation, in that you have to be precise in hitting the ball, but you have to hit it within 300 milliseconds. So, it balances similar problems of dynamic locomotion and precise manipulation.”

Ping pong robots have come a long way since the 1980s, most recently with designs by Omron and Google DeepMind that employ artificial intelligence techniques to “learn” from previous ping pong data, to improve a robot’s performance against an increasing variety of strokes and shots. These designs have been shown to be fast and precise enough to rally with intermediate human players.

“These are really specialized robots designed to play ping pong,” Cancio says. “With our robot, we are exploring how the techniques used in playing ping pong could translate to a more generalized system, like a humanoid or anthropomorphic robot that can do many different, useful things.”

Game control

For their new design, the researchers modified a lightweight, high-power robotic arm that Kim’s lab developed as part of the MIT Humanoid — a bipedal, two-armed robot that is about the size of a small child. The group is using the robot to test various dynamic maneuvers, including navigating uneven and varying terrain as well as jumping, running, and doing backflips, with the aim of one day deploying such robots for search-and-rescue operations.

Each of the humanoid’s arms has four joints, or degrees of freedom, which are each controlled by an electrical motor. Cancio, Nguyen, and Kim built a similar robotic arm, which they adapted for ping pong by adding an additional degree of freedom in the wrist to allow for control of a paddle.

The team fixed the robotic arm to a table at one end of a standard ping pong table and set up high-speed motion capture cameras around the table to track balls that are bounced at the robot. They also developed optimal control algorithms that predict, based on the principles of math and physics, what speed and paddle orientation the arm should execute to hit an incoming ball with a particular type of swing: loop (or topspin), drive (straight-on), or chop (backspin).

They implemented the algorithms using three computers that simultaneously processed camera images, estimated a ball’s real-time state, and translated these estimations to commands for the robot’s motors to quickly react and take a swing.

After consecutively bouncing 150 balls at the arm, they found the robot’s hit rate, or accuracy of returning the ball, was about the same for all three types of swings: 88.4 percent for loop strikes, 89.2 percent for chops, and 87.5 percent for drives. They have since tuned the robot’s reaction time and found the arm hits balls faster than existing systems, at velocities of 20 meters per second.

In their paper, the team reports that the robot’s strike speed, or the speed at which the paddle hits the ball, is on average 11 meters per second. Advanced human players have been known to return balls at speeds of between 21 to 25 meters second. Since writing up the results of their initial experiments, the researchers have further tweaked the system, and have recorded strike speeds of up to 19 meters per second (about 42 miles per hour).

“Some of the goal of this project is to say we can reach the same level of athleticism that people have,” Nguyen says. “And in terms of strike speed, we’re getting really, really close.”

Their follow-up work has also enabled the robot to aim. The team incorporated control algorithms into the system that predict not only how but where to hit an incoming ball. With its latest iteration, the researchers can set a target location on the table, and the robot will hit a ball to that same location.

Because it is fixed to the table, the robot has limited mobility and reach, and can mostly return balls that arrive within a crescent-shaped area around the midline of the table. In the future, the engineers plan to rig the bot on a gantry or wheeled platform, enabling it to cover more of the table and return a wider variety of shots.

“A big thing about table tennis is predicting the spin and trajectory of the ball, given how your opponent hit it, which is information that an automatic ball launcher won’t give you,” Cancio says. “A robot like this could mimic the maneuvers that an opponent would do in a game environment, in a way that helps humans play and improve.”

This research is supported, in part, by the Robotics and AI Institute.