An enormous amount of data demonstrating tool operation is needed to train a robot to use them. You could show a robot how to drive a nail using a hammer, but for it to remove a nail, the robot also must know to turn the hammer around and what angles and leverage are necessary for removal. But a group of researchers are working to simplify the training process. We’ll explore in the U.S. National Science Foundation’s “Discovery Files”.

There are many approaches for training robots. Some include photos and others may include simulations, or demos. Each dataset may also capture a unique task or environment. Combining all these sources into one machine-learning model is an efficiency challenge and robots trained with one specific task are often unable to perform new tasks in unfamiliar spaces.

NSF-supported researchers at MIT have developed a new technique to combine multiple sources in generative artificial intelligence known as diffusion models.

By training a diffusion model to learn one task using one specific dataset, and combining the policies learned by the diffusion models into a general policy, they demonstrate in both real-world experiments and simulations, a robot able to perform multiple tool-use tasks and adapt to new tasks it did not see during training.

The technique offered a 20 percent improvement in task performance when compared to baseline techniques and demonstrating a new method for training the robots of the future.